2023-11-24T07:51:48.821 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-24T07:51:48.821 DEBUG:teuthology.run:Teuthology command: teuthology --name rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi --archive /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585 --verbose --description fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} --owner scheduled_rishabh@teuthology -- /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/orig.config.yaml 2023-11-24T07:51:48.871 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T07:51:48.955 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585 branch: rishabh-mgr-vol-misc description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} email: null first_in_suite: false job_id: '7465585' kernel: client: branch: testing flavor: default kdb: true sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: rishabh-mgr-vol-misc ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-vol-tests sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da owner: scheduled_rishabh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9764 sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c sleep_before_teardown: 0 suite: fs:volumes suite_branch: mgr-vol-tests suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa suite_relpath: qa suite_repo: https://github.com/rishabh-d-dave/ceph suite_sha1: 73843baf45f8a92d8e3fcbea3f024c97e73036da syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi106.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDeEurGEUMjEgvIefvhfRLqIsBnEA6le92tQs8FB72yfJmmwh1/UqB6PkW7dcVCmTDQStpYE1qXQP8mt1ol3+FI= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPAasaiQ3DwUeCiBZ8JtwfQIq+3w7DB62xjr7R10augtjzRlAoThP/oLGKfHN4PsXUMZlqfzp0C6b7q90NfQJU= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots - tasks.cephfs.test_volumes.TestSubvolumeSnapshots teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-24_07:31:48 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-24T07:51:48.955 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa; will attempt to use it 2023-11-24T07:51:48.956 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks 2023-11-24T07:51:48.956 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-24T07:51:48.957 INFO:teuthology.task.internal:Checking packages... 2023-11-24T07:51:48.982 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a13ea5ff00cb4df7a9cc941166ddeb358881144c' 2023-11-24T07:51:48.982 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-24T07:51:48.982 INFO:teuthology.packaging:ref: None 2023-11-24T07:51:48.982 INFO:teuthology.packaging:tag: None 2023-11-24T07:51:48.982 INFO:teuthology.packaging:branch: rishabh-mgr-vol-misc 2023-11-24T07:51:48.982 INFO:teuthology.packaging:sha1: a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T07:51:48.982 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=rishabh-mgr-vol-misc 2023-11-24T07:51:49.209 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-7484.ga13ea5ff 2023-11-24T07:51:49.209 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-24T07:51:49.215 INFO:teuthology.task.internal:no buildpackages task found 2023-11-24T07:51:49.216 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-24T07:51:49.244 INFO:teuthology.task.internal:Saving configuration 2023-11-24T07:51:49.262 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-24T07:51:49.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T07:51:49.294 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:46:29.092082', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDeEurGEUMjEgvIefvhfRLqIsBnEA6le92tQs8FB72yfJmmwh1/UqB6PkW7dcVCmTDQStpYE1qXQP8mt1ol3+FI='} 2023-11-24T07:51:49.320 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:46:29.090411', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPAasaiQ3DwUeCiBZ8JtwfQIq+3w7DB62xjr7R10augtjzRlAoThP/oLGKfHN4PsXUMZlqfzp0C6b7q90NfQJU='} 2023-11-24T07:51:49.320 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-24T07:51:49.326 INFO:teuthology.task.internal:roles: ubuntu@smithi106.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-24T07:51:49.326 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-24T07:51:49.326 INFO:teuthology.run_tasks:Running task console_log... 2023-11-24T07:51:49.448 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f83daef0b80>, signals=[15]) 2023-11-24T07:51:49.449 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-24T07:51:49.455 INFO:teuthology.task.internal:Opening connections... 2023-11-24T07:51:49.455 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T07:51:49.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:51:49.542 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T07:51:49.543 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:51:49.628 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-24T07:51:49.635 DEBUG:teuthology.orchestra.run.smithi106:> uname -m 2023-11-24T07:51:49.652 INFO:teuthology.orchestra.run.smithi106.stdout:x86_64 2023-11-24T07:51:49.652 DEBUG:teuthology.orchestra.run.smithi106:> cat /etc/os-release 2023-11-24T07:51:49.708 INFO:teuthology.orchestra.run.smithi106.stdout:NAME="CentOS Stream" 2023-11-24T07:51:49.709 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION="9" 2023-11-24T07:51:49.709 INFO:teuthology.orchestra.run.smithi106.stdout:ID="centos" 2023-11-24T07:51:49.709 INFO:teuthology.orchestra.run.smithi106.stdout:ID_LIKE="rhel fedora" 2023-11-24T07:51:49.709 INFO:teuthology.orchestra.run.smithi106.stdout:VERSION_ID="9" 2023-11-24T07:51:49.709 INFO:teuthology.orchestra.run.smithi106.stdout:PLATFORM_ID="platform:el9" 2023-11-24T07:51:49.710 INFO:teuthology.orchestra.run.smithi106.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T07:51:49.710 INFO:teuthology.orchestra.run.smithi106.stdout:ANSI_COLOR="0;31" 2023-11-24T07:51:49.710 INFO:teuthology.orchestra.run.smithi106.stdout:LOGO="fedora-logo-icon" 2023-11-24T07:51:49.710 INFO:teuthology.orchestra.run.smithi106.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T07:51:49.710 INFO:teuthology.orchestra.run.smithi106.stdout:HOME_URL="https://centos.org/" 2023-11-24T07:51:49.710 INFO:teuthology.orchestra.run.smithi106.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:51:49.711 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T07:51:49.711 INFO:teuthology.orchestra.run.smithi106.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T07:51:49.711 INFO:teuthology.lock.ops:Updating smithi106.front.sepia.ceph.com on lock server 2023-11-24T07:51:49.791 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-11-24T07:51:49.809 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-11-24T07:51:49.810 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-11-24T07:51:49.866 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="CentOS Stream" 2023-11-24T07:51:49.866 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="9" 2023-11-24T07:51:49.866 INFO:teuthology.orchestra.run.smithi156.stdout:ID="centos" 2023-11-24T07:51:49.866 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="rhel fedora" 2023-11-24T07:51:49.866 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="9" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el9" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="CentOS Stream 9" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:LOGO="fedora-logo-icon" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://centos.org/" 2023-11-24T07:51:49.867 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-24T07:51:49.868 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-11-24T07:51:49.868 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-11-24T07:51:49.868 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-11-24T07:51:49.895 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-24T07:51:49.908 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-24T07:51:49.984 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-24T07:51:49.984 DEBUG:teuthology.orchestra.run.smithi106:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:51:49.987 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-11-24T07:51:50.004 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-24T07:51:50.010 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-24T07:51:50.010 DEBUG:teuthology.orchestra.run.smithi106:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:51:50.042 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-11-24T07:51:50.169 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-24T07:51:50.222 INFO:teuthology.run_tasks:Running task kernel... 2023-11-24T07:51:50.237 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'} 2023-11-24T07:51:50.237 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-11-24T07:51:50.238 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-11-24T07:51:50.238 DEBUG:teuthology.orchestra.run.smithi106:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:51:50.238 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-24T07:51:50.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:51:50.255 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-11-24T07:51:50.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:51:50.260 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2023-11-24T07:51:50.408 DEBUG:teuthology.task.kernel:sha1 for client.1 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:51:50.409 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T07:51:50.409 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-11-24T07:51:50.412 DEBUG:teuthology.task.kernel:sha1 for client.0 is 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:51:50.413 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "8247bbb5b470f7e2a1f3eefcc16e24cf1338f606"... 2023-11-24T07:51:50.413 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2023-11-24T07:51:50.426 INFO:teuthology.orchestra.run.smithi156.stdout:5.14.0-361.el9.x86_64 2023-11-24T07:51:50.426 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:51:50.427 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T07:51:50.427 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum upgrade -y linux-firmware 2023-11-24T07:51:50.430 INFO:teuthology.orchestra.run.smithi106.stdout:5.14.0-361.el9.x86_64 2023-11-24T07:51:50.430 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:51:50.430 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-11-24T07:51:50.430 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum upgrade -y linux-firmware 2023-11-24T07:51:52.609 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-11-24T07:51:52.615 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 13 MB/s | 6.4 MB 00:00 2023-11-24T07:51:54.293 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 2.9 MB/s | 2.0 MB 00:00 2023-11-24T07:51:54.474 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 2.4 MB/s | 2.0 MB 00:00 2023-11-24T07:51:55.091 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 7.9 MB 00:00 2023-11-24T07:51:56.961 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 24 MB/s | 18 MB 00:00 2023-11-24T07:51:56.981 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 3.6 MB/s | 7.9 MB 00:02 2023-11-24T07:52:00.699 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 18 MB 00:02 2023-11-24T07:52:00.827 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 12 MB/s | 5.8 MB 00:00 2023-11-24T07:52:02.386 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2023-11-24T07:52:04.690 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 20 MB 00:02 2023-11-24T07:52:05.741 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 3.5 MB/s | 5.8 MB 00:01 2023-11-24T07:52:08.008 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 15 kB 00:01 2023-11-24T07:52:08.837 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2023-11-24T07:52:09.487 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2023-11-24T07:52:11.874 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2023-11-24T07:52:11.885 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2023-11-24T07:52:11.886 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2023-11-24T07:52:11.886 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2023-11-24T07:52:11.886 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2023-11-24T07:52:11.886 INFO:teuthology.orchestra.run.smithi106.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T07:52:11.886 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T07:52:11.886 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2023-11-24T07:52:11.887 INFO:teuthology.orchestra.run.smithi106.stdout:================================================================================ 2023-11-24T07:52:11.887 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 1 Package 2023-11-24T07:52:11.887 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T07:52:11.887 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 328 M 2023-11-24T07:52:11.887 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2023-11-24T07:52:13.634 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2023-11-24T07:52:16.056 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 1 Package 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T07:52:16.073 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 328 M 2023-11-24T07:52:16.074 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-11-24T07:52:16.813 INFO:teuthology.orchestra.run.smithi106.stdout:linux-firmware-20231030-141.el9.noarch.rpm 67 MB/s | 328 MB 00:04 2023-11-24T07:52:16.814 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:52:16.815 INFO:teuthology.orchestra.run.smithi106.stdout:Total 67 MB/s | 328 MB 00:04 2023-11-24T07:52:16.815 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2023-11-24T07:52:16.904 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2023-11-24T07:52:16.904 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2023-11-24T07:52:19.027 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2023-11-24T07:52:19.028 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2023-11-24T07:52:21.127 INFO:teuthology.orchestra.run.smithi156.stdout:linux-firmware-20231030-141.el9.noarch.rpm 65 MB/s | 328 MB 00:05 2023-11-24T07:52:21.128 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-11-24T07:52:21.128 INFO:teuthology.orchestra.run.smithi156.stdout:Total 65 MB/s | 328 MB 00:05 2023-11-24T07:52:21.129 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-11-24T07:52:21.221 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-11-24T07:52:21.221 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-11-24T07:52:23.437 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-11-24T07:52:23.438 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-11-24T07:52:24.898 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2023-11-24T07:52:25.340 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:52:25.426 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:52:26.569 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:52:26.569 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:52:26.950 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:52:26.951 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T07:52:26.951 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2023-11-24T07:52:26.951 INFO:teuthology.orchestra.run.smithi106.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T07:52:26.951 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T07:52:26.951 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2023-11-24T07:52:27.253 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-24T07:52:27.253 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:52:27.398 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T07:52:27.398 DEBUG:teuthology.orchestra.run.smithi106:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T07:52:27.401 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi106.front.sepia.ceph.com to complete... 2023-11-24T07:52:29.341 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-11-24T07:52:29.637 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:52:29.724 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:52:30.711 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:52:30.712 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-11-24T07:52:31.088 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-11-24T07:52:31.088 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T07:52:31.089 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-11-24T07:52:31.089 INFO:teuthology.orchestra.run.smithi156.stdout: linux-firmware-20231030-141.el9.noarch 2023-11-24T07:52:31.089 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T07:52:31.089 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-11-24T07:52:31.357 INFO:teuthology.task.kernel:Downloading kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-24T07:52:31.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 2023-11-24T07:52:31.581 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ 2023-11-24T07:52:31.581 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/ --input-file=- 2023-11-24T07:52:31.583 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi156.front.sepia.ceph.com to complete... 2023-11-24T07:52:40.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24 07:52:40 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T07:52:40.596 INFO:teuthology.orchestra.run.smithi106.stderr:FINISHED --2023-11-24 07:52:40-- 2023-11-24T07:52:40.597 INFO:teuthology.orchestra.run.smithi106.stderr:Total wall clock time: 13s 2023-11-24T07:52:40.597 INFO:teuthology.orchestra.run.smithi106.stderr:Downloaded: 1 files, 306M in 13s (23.9 MB/s) 2023-11-24T07:52:40.598 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.0... 2023-11-24T07:52:40.598 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T07:52:42.440 INFO:teuthology.orchestra.run.smithi106.stdout:Verifying... ######################################## 2023-11-24T07:52:42.514 INFO:teuthology.orchestra.run.smithi106.stdout:Preparing... ######################################## 2023-11-24T07:52:42.593 INFO:teuthology.orchestra.run.smithi106.stdout:Updating / installing... 2023-11-24T07:52:49.639 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T07:52:51.910 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24 07:52:51 URL:https://3.chacra.ceph.com/r/kernel/testing/8247bbb5b470f7e2a1f3eefcc16e24cf1338f606/centos/9/flavors/default/x86_64/kernel-6.6.0_g8247bbb5b470-1.x86_64.rpm [320493731/320493731] -> "/tmp/kernel.x86_64.rpm" [1] 2023-11-24T07:52:51.910 INFO:teuthology.orchestra.run.smithi156.stderr:FINISHED --2023-11-24 07:52:51-- 2023-11-24T07:52:51.910 INFO:teuthology.orchestra.run.smithi156.stderr:Total wall clock time: 20s 2023-11-24T07:52:51.911 INFO:teuthology.orchestra.run.smithi156.stderr:Downloaded: 1 files, 306M in 20s (15.4 MB/s) 2023-11-24T07:52:51.912 INFO:teuthology.task.kernel:Installing kernel 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 on client.1... 2023-11-24T07:52:51.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-11-24T07:52:53.761 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying... ######################################## 2023-11-24T07:52:53.827 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing... ######################################## 2023-11-24T07:52:53.908 INFO:teuthology.orchestra.run.smithi156.stdout:Updating / installing... 2023-11-24T07:53:01.038 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-6.6.0_g8247bbb5b470-1 ######################################## 2023-11-24T07:53:04.615 INFO:teuthology.orchestra.run.smithi106.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T07:53:16.301 INFO:teuthology.orchestra.run.smithi156.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-11-24T07:53:25.686 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi106.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T07:53:25.686 DEBUG:teuthology.orchestra.run.smithi106:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T07:53:25.801 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T07:53:25.802 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T07:53:25.803 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T07:53:25.804 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T07:53:25.805 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T07:53:25.806 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T07:53:25.807 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T07:53:25.808 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T07:53:25.809 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T07:53:25.810 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T07:53:25.811 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T07:53:25.812 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T07:53:25.813 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T07:53:25.814 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T07:53:25.815 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T07:53:25.816 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T07:53:25.817 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T07:53:25.818 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T07:53:25.819 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T07:53:25.820 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T07:53:25.821 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T07:53:25.822 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T07:53:25.823 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T07:53:25.824 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T07:53:25.825 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T07:53:25.826 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T07:53:25.827 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T07:53:25.828 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T07:53:25.829 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T07:53:25.830 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T07:53:25.831 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T07:53:25.832 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T07:53:25.833 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T07:53:25.834 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T07:53:25.835 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T07:53:25.836 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T07:53:25.837 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T07:53:25.838 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T07:53:25.839 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T07:53:25.840 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T07:53:25.841 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T07:53:25.842 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T07:53:25.843 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T07:53:25.844 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T07:53:25.845 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T07:53:25.846 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T07:53:25.847 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T07:53:25.848 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T07:53:25.849 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T07:53:25.850 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T07:53:25.851 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T07:53:25.852 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T07:53:25.853 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T07:53:25.854 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T07:53:25.855 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T07:53:25.856 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T07:53:25.857 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T07:53:25.858 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T07:53:25.859 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T07:53:25.860 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T07:53:25.861 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T07:53:25.862 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T07:53:25.863 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T07:53:25.864 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T07:53:25.865 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T07:53:25.866 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T07:53:25.867 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T07:53:25.868 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T07:53:25.869 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T07:53:25.870 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T07:53:25.871 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T07:53:25.872 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T07:53:25.873 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T07:53:25.874 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T07:53:25.875 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T07:53:25.876 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T07:53:25.877 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T07:53:25.878 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T07:53:25.879 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T07:53:25.880 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T07:53:25.881 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T07:53:25.882 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T07:53:25.883 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T07:53:25.884 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T07:53:25.885 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T07:53:25.886 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T07:53:25.887 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T07:53:25.888 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T07:53:25.889 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T07:53:25.890 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T07:53:25.891 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T07:53:25.892 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T07:53:25.893 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T07:53:25.894 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T07:53:25.895 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T07:53:25.896 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T07:53:25.897 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T07:53:25.898 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T07:53:25.899 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T07:53:25.900 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T07:53:25.901 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T07:53:25.902 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T07:53:25.903 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T07:53:25.904 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T07:53:25.905 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T07:53:25.906 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T07:53:25.907 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T07:53:25.908 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T07:53:25.909 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T07:53:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T07:53:25.911 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T07:53:25.912 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T07:53:25.913 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T07:53:25.914 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T07:53:25.915 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T07:53:25.916 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T07:53:25.917 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T07:53:25.918 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T07:53:25.919 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T07:53:25.920 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T07:53:25.921 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T07:53:25.922 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T07:53:25.923 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T07:53:25.924 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T07:53:25.925 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T07:53:25.926 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T07:53:25.927 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T07:53:25.928 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T07:53:25.929 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T07:53:25.930 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T07:53:25.931 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T07:53:25.932 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T07:53:25.933 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T07:53:25.934 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T07:53:25.935 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T07:53:25.936 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T07:53:25.937 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T07:53:25.938 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T07:53:25.939 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T07:53:25.940 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T07:53:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T07:53:25.942 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T07:53:25.943 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T07:53:25.944 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T07:53:25.945 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T07:53:25.946 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T07:53:25.947 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T07:53:25.948 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T07:53:25.949 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T07:53:25.950 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T07:53:25.951 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T07:53:25.952 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T07:53:25.953 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T07:53:25.954 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T07:53:25.955 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T07:53:25.956 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T07:53:25.957 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T07:53:25.958 INFO:teuthology.orchestra.run.smithi106.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T07:53:25.959 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T07:53:25.959 DEBUG:teuthology.orchestra.run.smithi106:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T07:53:25.981 INFO:teuthology.orchestra.run.smithi106.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T07:53:25.981 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T07:53:25.982 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2023-11-24T07:53:25.983 INFO:teuthology.orchestra.run.smithi106.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T07:53:25.983 INFO:teuthology.orchestra.run.smithi106.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T07:53:25.983 INFO:teuthology.orchestra.run.smithi106.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T07:53:25.983 INFO:teuthology.orchestra.run.smithi106.stdout:fi 2023-11-24T07:53:25.984 DEBUG:teuthology.orchestra.run.smithi106:> sudo rpm -qi grub2-tools 2023-11-24T07:53:26.059 INFO:teuthology.orchestra.run.smithi106.stdout:Name : grub2-tools 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Epoch : 1 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Version : 2.06 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Release : 68.el9 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Architecture: x86_64 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Group : Unspecified 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Size : 8264254 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:License : GPLv3+ 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T07:53:26.060 INFO:teuthology.orchestra.run.smithi106.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:Packager : builder@centos.org 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:Vendor : CentOS 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:Summary : Support tools for GRUB. 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:Description : 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:hardware devices. 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T07:53:26.061 INFO:teuthology.orchestra.run.smithi106.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T07:53:26.062 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T07:53:26.062 INFO:teuthology.task.kernel:Updating grub on smithi106 to boot 6.6.0-g8247bbb5b470 2023-11-24T07:53:26.063 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T07:53:26.600 INFO:teuthology.orchestra.run.smithi106.stderr:Generating grub configuration file ... 2023-11-24T07:53:27.952 INFO:teuthology.orchestra.run.smithi106.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T07:53:27.991 INFO:teuthology.orchestra.run.smithi106.stderr:done 2023-11-24T07:53:27.993 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2023-11-24T07:53:28.008 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.dIhen8D3Q9 2023-11-24T07:53:28.008 DEBUG:teuthology.orchestra.run.smithi106:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.dIhen8D3Q9 2023-11-24T07:53:28.089 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.dIhen8D3Q9 2023-11-24T07:53:28.224 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.dIhen8D3Q9 is 6KB 2023-11-24T07:53:28.273 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.dIhen8D3Q9 2023-11-24T07:53:28.287 DEBUG:teuthology.orchestra.run.smithi106:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T07:53:28.359 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T07:53:28.359 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T07:53:28.359 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T07:53:28.359 INFO:teuthology.orchestra.run.smithi106.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T07:53:28.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T07:53:28.520 DEBUG:teuthology.orchestra.run.smithi106:> sudo shutdown -r now 2023-11-24T07:53:37.439 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi156.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-11-24T07:53:37.439 DEBUG:teuthology.orchestra.run.smithi156:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-11-24T07:53:37.499 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:53:37.499 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/config-6.6.0-g8247bbb5b470 2023-11-24T07:53:37.499 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/vmlinuz-6.6.0-g8247bbb5b470 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-11-24T07:53:37.500 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crc32c-intel.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-avx2.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-11-24T07:53:37.501 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-amd.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm-intel.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/arch/x86/kvm/kvm.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ansi_cprng.ko 2023-11-24T07:53:37.502 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/anubis.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/arc4.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_memcpy.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_pq.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_raid6_recov.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_tx.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/async_xor.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/async_tx/raid6test.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authenc.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/authencesn.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blake2b_generic.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_common.ko 2023-11-24T07:53:37.503 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/blowfish_generic.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/camellia_generic.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast5_generic.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast6_generic.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cast_common.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/ccm.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cmac.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crc32_generic.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_engine.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/crypto_user.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/cts.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/des_generic.ko 2023-11-24T07:53:37.504 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/dh_generic.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/echainiv.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/essiv.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/fcrypt.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/khazad.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/lz4hc.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/md4.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/michael_mic.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcbc.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/pcrypt.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/rmd160.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/seed.ko 2023-11-24T07:53:37.505 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/serpent_generic.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tcrypt.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/tea.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_common.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/twofish_generic.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/vmac.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/wp512.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xcbc.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xor.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/crypto/xxhash_generic.ko 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi 2023-11-24T07:53:37.506 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_ipmi.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/acpi_pad.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/custom_method.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/ec_sys.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/platform_profile.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbs.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/sbshc.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/acpi/video.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/acard-ahci.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ahci_platform.ko 2023-11-24T07:53:37.507 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/ata_generic.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/libahci_platform.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_acpi.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ali.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_amd.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_artop.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atiixp.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_atp867x.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd640.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cmd64x.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_cypress.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_efar.ko 2023-11-24T07:53:37.508 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt366.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt37x.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x2n.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_hpt3x3.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it8213.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_it821x.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_jmicron.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_marvell.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_mpiix.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_netcell.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ninja32.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87410.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_ns87415.ko 2023-11-24T07:53:37.509 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_oldpiix.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_opti.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_optidma.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pcmcia.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc2027x.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_piccolo.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_rdc.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sch.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_serverworks.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sil680.ko 2023-11-24T07:53:37.510 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sis.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_sl82c105.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_triflex.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pata_via.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/pdc_adma.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_inic162x.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_mv.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_nv.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_promise.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_qstor.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sil24.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sis.ko 2023-11-24T07:53:37.511 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_svw.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_sx4.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_uli.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_via.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ata/sata_vsc.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/auxdisplay/ks0108.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-i2c.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/base/regmap/regmap-mmio.ko 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma 2023-11-24T07:53:37.512 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/bcma/bcma.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/aoe/aoe.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/brd.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/drbd/drbd.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/floppy.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/loop.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/nbd.ko 2023-11-24T07:53:37.513 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/null_blk/null_blk.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/pktcdvd.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/rbd.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/virtio_blk.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/xen-blkfront.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/block/zram/zram.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hangcheck-timer.ko 2023-11-24T07:53:37.514 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/amd-rng.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/intel-rng.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/via-rng.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/hw_random/virtio-rng.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_si.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-11-24T07:53:37.515 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/lp.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/mwave/mwave.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/ppdev.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tlclk.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_atmel.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_infineon.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_nsc.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/uv_mmtimer.ko 2023-11-24T07:53:37.516 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/char/virtio_console.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/p4-clockmod.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/powernow-k8.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/cpufreq/speedstep-lib.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/ccp/ccp.ko 2023-11-24T07:53:37.517 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-aes.ko 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/padlock-sha.ko 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dca/dca.ko 2023-11-24T07:53:37.518 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/dw/dw_dmac.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/dma/ioat/ioatdma.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/amd64_edac.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/e752x_edac.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/edac_mce_amd.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3000_edac.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i3200_edac.ko 2023-11-24T07:53:37.519 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5100_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i5400_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7300_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i7core_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/i82975x_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/ie31200_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/sb_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/edac/x38_edac.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/edd.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/firmware/iscsi_ibft.ko 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio 2023-11-24T07:53:37.520 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-ich.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/gpio/gpio-viperboard.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-appleir.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-aureal.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-axff.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-dr.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elecom.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-elo.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-emsff.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gaff.ko 2023-11-24T07:53:37.521 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gt683r.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-gyration.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-kbd.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtek-mouse.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-holtekff.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-hyperv.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-icade.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-keytouch.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-kye.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lcpower.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-led.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-lenovo.ko 2023-11-24T07:53:37.522 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-dj.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-multitouch.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-ortek.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-penmount.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-petalynx.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-picolcd.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-pl.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-plantronics.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-primax.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-rmi.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-arvo.ko 2023-11-24T07:53:37.523 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-common.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-isku.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kone.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-konepure.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-lua.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-pyra.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-ryos.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat-savu.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-roccat.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-saitek.ko 2023-11-24T07:53:37.524 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-samsung.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sensor-hub.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sjoy.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sony.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-speedlink.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-steelseries.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-sunplus.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-thrustmaster.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tivo.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-tmff.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-topseed.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-twinhan.ko 2023-11-24T07:53:37.525 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-uclogic.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-waltop.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-wiimote.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-xinmo.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zpff.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/hid-zydacron.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/uhid.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hid/wacom.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_balloon.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_utils.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hv/hv_vmbus.ko 2023-11-24T07:53:37.526 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/abituguru3.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/acpi_power_meter.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7414.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ad7418.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adc128d818.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1025.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1026.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1029.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm1031.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adm9240.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ads7828.ko 2023-11-24T07:53:37.527 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7410.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7411.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7462.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7470.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7475.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/adt7x10.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/amc6821.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/applesmc.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asb100.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asc7621.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/asus_atk0110.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/atxp1.ko 2023-11-24T07:53:37.528 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/coretemp.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/dme1737.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds1621.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ds620.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc1403.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/emc6w201.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71805f.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f71882fg.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/f75375s.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fam15h_power.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/fschmd.ko 2023-11-24T07:53:37.529 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g760a.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/g762.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl518sm.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/gl520sm.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/hwmon-vid.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5500_temp.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/i5k_amb.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmaem.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ibmpex.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina209.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ina2xx.ko 2023-11-24T07:53:37.530 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/it87.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k10temp.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/k8temp.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lineage-pem.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm63.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm73.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm75.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm77.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm78.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm80.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm83.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm85.ko 2023-11-24T07:53:37.531 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm87.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm90.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm92.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm93.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95234.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95241.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/lm95245.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc2945.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4151.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4215.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4222.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4245.ko 2023-11-24T07:53:37.532 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4260.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/ltc4261.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max16065.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1619.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max1668.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max197.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6639.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6650.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/max6697.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/mcp3021.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6683.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775-core.ko 2023-11-24T07:53:37.533 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct6775.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/nct7802.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87360.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pc87427.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pcf8591.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max16064.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max34440.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/max8688.ko 2023-11-24T07:53:37.534 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/powr1220.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5627.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch5636.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sch56xx-common.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht15.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sht21.ko 2023-11-24T07:53:37.535 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/shtc1.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/sis5595.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47b397.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m1.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/smsc47m192.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/thmc50.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp102.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp103.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp401.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/tmp421.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via-cputemp.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/via686a.ko 2023-11-24T07:53:37.536 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt1211.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/vt8231.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627ehf.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83627hf.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83781d.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83791d.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83792d.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83793.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83795.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l785ts.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/hwmon/w83l786ng.ko 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c 2023-11-24T07:53:37.537 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-11-24T07:53:37.538 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-i801.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-isch.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-parport.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-11-24T07:53:37.539 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-via.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-dev.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-mux.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-smbus.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/i2c/i2c-stub.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_cm.ko 2023-11-24T07:53:37.540 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_core.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_umad.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/iw_cm.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_cm.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-11-24T07:53:37.541 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp 2023-11-24T07:53:37.542 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/fm801-gp.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/gameport/gameport.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/joydev.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/keyboard/gpio_keys.ko 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse 2023-11-24T07:53:37.543 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/appletouch.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/bcm5974.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/cyapatp.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/elan_i2c.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/sermouse.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/synaptics_usb.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/mouse/vsxxxaa.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/rmi4/rmi_core.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/altera_ps2.ko 2023-11-24T07:53:37.544 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/arc_ps2.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/serio/serio_raw.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/input/sparse-keymap.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/iommu/amd/iommu_v2.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-blinkm.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lm3530.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-lp3944.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/leds-ss4200.ko 2023-11-24T07:53:37.545 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/bcache/bcache.ko 2023-11-24T07:53:37.546 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-bio-prison.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache-smq.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-cache.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-crypt.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-delay.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-flakey.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-log-userspace.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-multipath.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-queue-length.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-raid.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-round-robin.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-service-time.ko 2023-11-24T07:53:37.547 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-switch.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-thin-pool.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/dm-verity.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/faulty.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/linear.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/multipath.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid0.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid1.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid10.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/md/raid456.ko 2023-11-24T07:53:37.548 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptbase.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptctl.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptfc.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptlan.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptsas.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptscsih.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/message/fusion/mptspi.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_ich.ko 2023-11-24T07:53:37.549 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/lpc_sch.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/mfd-core.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/sm501.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/viperboard.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/vx855.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/mfd/wl1273-core.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds9802als.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/apds990x.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/bh1770glc.ko 2023-11-24T07:53:37.550 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/cb710/cb710.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/at24.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/eeprom/max6875.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/enclosure.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/hpilo.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29003.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/isl29020.ko 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d 2023-11-24T07:53:37.551 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-me.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei-txe.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/mei/mei.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-gru/gru.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xp.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpc.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-11-24T07:53:37.552 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_7xx1.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tifm_core.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/tsl2550.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_balloon.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/bonding/bonding.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/b53/b53_common.ko 2023-11-24T07:53:37.553 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/bcm-sf2.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6060.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/dummy.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/eql.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-11-24T07:53:37.554 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/8390.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/agere/et131x.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera 2023-11-24T07:53:37.555 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-11-24T07:53:37.556 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade 2023-11-24T07:53:37.557 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-11-24T07:53:37.558 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-11-24T07:53:37.559 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/dnet.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ethoc.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/fealnx.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e100.ko 2023-11-24T07:53:37.560 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-11-24T07:53:37.561 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/jme.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/skge.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4 2023-11-24T07:53:37.562 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi 2023-11-24T07:53:37.563 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-11-24T07:53:37.564 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/atp.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-11-24T07:53:37.565 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis190.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sis/sis900.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-11-24T07:53:37.566 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/cassini.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/niu.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sungem.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti 2023-11-24T07:53:37.567 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/ti/tlan.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/geneve.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154 2023-11-24T07:53:37.568 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ieee802154/fakelb.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ifb.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ipvlan/ipvlan.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvlan.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/macvtap.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mdio/mdio-regmap.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/mii.ko 2023-11-24T07:53:37.569 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/net_failover.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/netconsole.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/nlmon.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs-lynx.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/amd.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm7xxx.ko 2023-11-24T07:53:37.570 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/bcm87xx.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/broadcom.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/cicada.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/davicom.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/dp83640.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/et1011c.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/icplus.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/lxt.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/marvell.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/micrel.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/national.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/phylink.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/qsemi.ko 2023-11-24T07:53:37.571 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/realtek.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/smsc.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/ste10Xp.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/phy/vitesse.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/bsd_comp.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_async.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_deflate.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_generic.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_mppe.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/ppp_synctty.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppoe.ko 2023-11-24T07:53:37.572 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pppox.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/ppp/pptp.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slhc.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/slip/slip.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/sungem_phy.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tap.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_activebackup.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_broadcast.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-11-24T07:53:37.573 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_random.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/tun.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/veth.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/virtio_net.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/vxlan/vxlan.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netback/xen-netback.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/net/xen-netfront.ko 2023-11-24T07:53:37.574 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ntb/ntb.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme-core.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/nvme/host/nvme.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_cs.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_pc.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/parport/parport_serial.ko 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci 2023-11-24T07:53:37.575 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/pcie/aer_inject.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pci/xen-pcifront.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/i82092.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/pd6729.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pcmcia/yenta_socket.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-11-24T07:53:37.576 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acer-wmi.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/acerhdf.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/apple-gmux.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-laptop.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/asus-wmi.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/classmate-laptop.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-11-24T07:53:37.577 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/hdaps.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/intel_ips.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/msi-wmi.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/mxm-wmi.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-laptop.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/samsung-q10.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/topstar-laptop.ko 2023-11-24T07:53:37.578 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/toshiba_haps.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi-bmof.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/platform/x86/wmi.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_common.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/powercap/intel_rapl_msr.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-gpio.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps-ldisc.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/clients/pps_parport.ko 2023-11-24T07:53:37.579 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/pps/pps_core.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ptp/ptp_kvm.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-bq32k.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1286.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1307.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1374.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1511.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1553.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1672.ko 2023-11-24T07:53:37.580 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds1742.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds2404.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-ds3232.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-em3027.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-fm3130.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl12022.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-isl1208.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m41t80.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t35.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-m48t59.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-max6900.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-msm6242.ko 2023-11-24T07:53:37.581 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf2127.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf85063.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8523.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8563.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-pcf8583.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rp5c01.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rs5c372.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8025.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-rx8581.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/rtc/rtc-x1205.ko 2023-11-24T07:53:37.582 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-9xxx.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-sas.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/3w-xxxx.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/BusLogic.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/a100u2w.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aacraid/aacraid.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/advansys.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/am53c974.ko 2023-11-24T07:53:37.583 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/atp870u.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bfa/bfa.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ch.ko 2023-11-24T07:53:37.584 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/csiostor/csiostor.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dc395x.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-11-24T07:53:37.585 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/dmx3191d.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esas2r/esas2r.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/esp_scsi.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/fcoe.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/fnic/fnic.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hpsa.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hptiop.ko 2023-11-24T07:53:37.586 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/hv_storvsc.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/initio.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ipr.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ips.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/isci/isci.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/iscsi_tcp.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libfc/libfc.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libiscsi_tcp.ko 2023-11-24T07:53:37.587 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/libsas/libsas.ko 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/lpfc/lpfc.ko 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid.ko 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-11-24T07:53:37.588 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvsas/mvsas.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/mvumi.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/pmcraid.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla1280.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-11-24T07:53:37.589 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/raid_class.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_debug.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_fc.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_sas.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_spi.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/scsi_transport_srp.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/ses.ko 2023-11-24T07:53:37.590 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/st.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/stex.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/virtio_scsi.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/vmw_pvscsi.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/scsi/wd719x.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ssb/ssb.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-11-24T07:53:37.591 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/loopback/tcm_loop.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_file.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_iblock.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_mod.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_pscsi.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/target_core_user.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal 2023-11-24T07:53:37.592 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-11-24T07:53:37.593 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_gsm.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/n_hdlc.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/nozomi.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/arc_uart.ko 2023-11-24T07:53:37.594 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/tty/serial/jsm/jsm.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/core/ufshcd-core.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_aec.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_cif.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_pci_generic.ko 2023-11-24T07:53:37.595 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/uio/uio_sercos3.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-acm.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/cdc-wdm.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usblp.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/class/usbtmc.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/host/sl811-hcd.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/mdc800.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/image/microtek.ko 2023-11-24T07:53:37.596 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/adutux.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/appledisplay.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi26.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/emi62.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ezusb.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/idmouse.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/iowarrior.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/isight_firmware.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/ldusb.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/legousbtower.ko 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga 2023-11-24T07:53:37.597 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/trancevibrator.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usb3503.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usblcd.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/usbsevseg.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/uss720.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/misc/yurex.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/phy/phy-generic.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/aircable.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ark3116.ko 2023-11-24T07:53:37.598 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/belkin_sa.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ch341.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cp210x.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cyberjack.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/cypress_m8.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/digi_acceleport.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/empeg.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ftdi_sio.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/garmin_gps.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_edgeport.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/io_ti.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipaq.ko 2023-11-24T07:53:37.599 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ipw.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ir-usb.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/keyspan_pda.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kl5kusb105.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/kobil_sct.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mct_u232.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7720.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/mos7840.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/navman.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/omninet.ko 2023-11-24T07:53:37.600 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/opticon.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/option.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/oti6858.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/pl2303.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcaux.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/qcserial.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/quatech2.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/safe_serial.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/sierra.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/spcp8x5.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ssu100.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/symbolserial.ko 2023-11-24T07:53:37.601 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_debug.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/usb_wwan.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/visor.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/whiteheat.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/serial/xsens_mt.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/uas.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-alauda.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-cypress.ko 2023-11-24T07:53:37.602 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-datafab.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-freecom.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-isd200.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-karma.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-onetouch.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-realtek.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr09.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-sddr55.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/ums-usbat.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/storage/usb-storage.ko 2023-11-24T07:53:37.603 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-core.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/usbip-host.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/pci/vfio-pci.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_iotlb.ko 2023-11-24T07:53:37.604 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_net.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/vhost/vhost_scsi.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/apple_bl.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/lcd.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/video/backlight/platform_lcd.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_balloon.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_mmio.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci.ko 2023-11-24T07:53:37.605 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/virtio/virtio_ring.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2482.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/masters/ds2490.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-11-24T07:53:37.606 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_smem.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/slaves/w1_therm.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/w1/wire.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim1535_wdt.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/alim7101_wdt.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/f71808e_wdt.ko 2023-11-24T07:53:37.607 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/hpwdt.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/i6300esb.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/iTCO_wdt.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ib700wdt.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ibmasr.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it8712f_wdt.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/it87_wdt.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/machzwd.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/nv_tco.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_pci.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/pcwd_usb.ko 2023-11-24T07:53:37.608 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sch311x_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/softdog.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/sp5100_tco.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/via_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83877f_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/w83977f_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/wdt_pci.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/watchdog/xen_wdt.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-acpi-processor.ko 2023-11-24T07:53:37.609 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-evtchn.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntalloc.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-gntdev.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xen-privcmd.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/drivers/xen/xenfs/xenfs.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/9p/9p.ko 2023-11-24T07:53:37.610 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/affs/affs.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/befs/befs.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/binfmt_misc.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/btrfs/btrfs.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cachefiles/cachefiles.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ceph/ceph.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/coda/coda.ko 2023-11-24T07:53:37.611 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/cramfs/cramfs.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/dlm/dlm.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ecryptfs/ecryptfs.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/f2fs/f2fs.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/fat.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/msdos.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fat/vfat.ko 2023-11-24T07:53:37.612 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fscache/fscache.ko 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/cuse.ko 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/fuse/fuse.ko 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/gfs2/gfs2.ko 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfs/hfs.ko 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/hfsplus/hfsplus.ko 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs 2023-11-24T07:53:37.613 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/isofs/isofs.ko 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/jfs/jfs.ko 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/lockd/lockd.ko 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/minix/minix.ko 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/netfs/netfs.ko 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-11-24T07:53:37.614 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfs.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv3.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs/nfsv4.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/grace.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfs_common/nfs_acl.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nfsd/nfsd.ko 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2 2023-11-24T07:53:37.615 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nilfs2/nilfs2.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-celtic.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-centeuro.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-croatian.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-cyrillic.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-gaelic.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-greek.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-iceland.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-inuit.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-roman.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-romanian.ko 2023-11-24T07:53:37.616 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/mac-turkish.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1250.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1251.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp1255.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp737.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp775.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp850.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp852.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp855.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp857.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp860.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp861.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp862.ko 2023-11-24T07:53:37.617 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp863.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp864.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp865.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp866.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp869.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp874.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp932.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp936.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp949.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_cp950.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_euc-jp.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-1.ko 2023-11-24T07:53:37.618 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-13.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-14.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-15.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-2.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-3.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-4.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-5.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-6.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-7.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_iso8859-9.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-r.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-ru.ko 2023-11-24T07:53:37.619 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_koi8-u.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_ucs2_utils.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/nls/nls_utf8.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-11-24T07:53:37.620 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/overlayfs/overlay.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/pstore/ramoops.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/reiserfs/reiserfs.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/romfs/romfs.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/client/cifs.ko 2023-11-24T07:53:37.621 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_arc4.ko 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/smb/common/cifs_md4.ko 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/squashfs/squashfs.ko 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/sysv/sysv.ko 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/udf/udf.ko 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/ufs/ufs.ko 2023-11-24T07:53:37.622 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/fs/xfs/xfs.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/rcu/rcutorture.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/torture.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/asn1_encoder.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc-itu-t.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crc8.ko 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto 2023-11-24T07:53:37.623 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libarc4.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/crypto/libdes.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/libcrc32c.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lru_cache.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4_compress.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/lz4/lz4hc_compress.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/math/cordic.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/raid6/raid6_pq.ko 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon 2023-11-24T07:53:37.624 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/reed_solomon/reed_solomon.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_bm.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_fsm.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/ts_kmp.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/lib/zstd/zstd_compress.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/mm/hwpoison-inject.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/6lowpan.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_dest.ko 2023-11-24T07:53:37.625 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_fragment.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_hop.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_ipv6.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_mobility.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_routing.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/6lowpan/nhc_udp.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/garp.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/mrp.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/p8022.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/psnap.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/802/stp.ko 2023-11-24T07:53:37.626 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/8021q/8021q.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_fd.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_rdma.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/9p/9pnet_virtio.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/batman-adv/batman-adv.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/br_netfilter.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/bridge.ko 2023-11-24T07:53:37.627 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_among.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arp.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_limit.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_log.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-11-24T07:53:37.628 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_snat.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_stp.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/ebtables.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-11-24T07:53:37.629 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ceph/libceph.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/failover.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/core/pktgen.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_diag.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv4.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dccp/dccp_ipv6.ko 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver 2023-11-24T07:53:37.630 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dns_resolver/dns_resolver.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/dsa_core.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_brcm.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_dsa.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_none.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/dsa/tag_trailer.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ieee802154/ieee802154_socket.ko 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4 2023-11-24T07:53:37.631 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ah4.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/esp4.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/fou.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/gre.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/inet_diag.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_gre.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_tunnel.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ip_vti.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipcomp.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/ipip.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arp_tables.ko 2023-11-24T07:53:37.632 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/iptable_security.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-11-24T07:53:37.633 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_bic.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_dctcp.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_diag.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_highspeed.ko 2023-11-24T07:53:37.634 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_htcp.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_hybla.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_illinois.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_lp.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_scalable.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_vegas.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_veno.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_westwood.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tcp_yeah.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/tunnel4.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_diag.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/udp_tunnel.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv4/xfrm4_tunnel.ko 2023-11-24T07:53:37.635 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ah6.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/esp6.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/fou6.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_tunnel.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ip6_vti.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/ipcomp6.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-11-24T07:53:37.636 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-11-24T07:53:37.637 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/sit.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/tunnel6.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/ipv6/xfrm6_tunnel.ko 2023-11-24T07:53:37.638 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/key/af_key.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_core.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_debugfs.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_eth.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ip6.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_netlink.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/l2tp/l2tp_ppp.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/llc/llc.ko 2023-11-24T07:53:37.639 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mac802154/mac802154.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/mpls/mpls_gso.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-11-24T07:53:37.640 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs.ko 2023-11-24T07:53:37.641 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-11-24T07:53:37.642 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conncount.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_h323.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_irc.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sane.ko 2023-11-24T07:53:37.643 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_sip.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_dup_netdev.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_flow_table_inet.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_log_syslog.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_amanda.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_ftp.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_irc.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_sip.ko 2023-11-24T07:53:37.644 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_nat_tftp.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_synproxy_core.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nf_tables.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_acct.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_log.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_osf.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nfnetlink_queue.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_chain_nat.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_compat.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_ct.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_dup_netdev.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fib.ko 2023-11-24T07:53:37.645 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_fwd_netdev.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_hash.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_limit.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_log.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_masq.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_nat.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_numgen.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_queue.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_quota.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_redir.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject.ko 2023-11-24T07:53:37.646 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_reject_inet.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_socket.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/nft_tunnel.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_AUDIT.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CHECKSUM.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CLASSIFY.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_CT.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_DSCP.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HL.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_HMARK.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_IDLETIMER.ko 2023-11-24T07:53:37.647 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LED.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_LOG.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_MASQUERADE.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NETMAP.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFLOG.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_NFQUEUE.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_RATEEST.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_REDIRECT.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_SECMARK.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPMSS.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TEE.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TPROXY.ko 2023-11-24T07:53:37.648 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_TRACE.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_addrtype.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_bpf.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cgroup.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cluster.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_comment.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connbytes.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlabel.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connlimit.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_connmark.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_conntrack.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_cpu.ko 2023-11-24T07:53:37.649 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dccp.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_devgroup.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_dscp.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ecn.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_esp.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hashlimit.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_helper.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_hl.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipcomp.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_iprange.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_ipvs.ko 2023-11-24T07:53:37.650 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_l2tp.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_length.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_limit.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mac.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_mark.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_multiport.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nat.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_nfacct.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_osf.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_owner.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_physdev.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_pkttype.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_policy.ko 2023-11-24T07:53:37.651 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_quota.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_rateest.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_realm.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_recent.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_sctp.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_set.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_socket.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_state.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_statistic.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_string.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_tcpmss.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_time.ko 2023-11-24T07:53:37.652 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netfilter/xt_u32.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/netlink/netlink_diag.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/nsh/nsh.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/openvswitch.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-geneve.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-gre.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/openvswitch/vport-vxlan.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/packet/af_packet_diag.ko 2023-11-24T07:53:37.653 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_rdma.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/rds/rds_tcp.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_csum.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_gact.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_ipt.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_mirred.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_nat.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_pedit.ko 2023-11-24T07:53:37.654 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_police.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_simple.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_skbedit.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/act_vlan.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_basic.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_bpf.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_flow.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_fw.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_route.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/cls_u32.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_cmp.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_ipset.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_meta.ko 2023-11-24T07:53:37.655 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_nbyte.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_text.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/em_u32.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_choke.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_codel.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_drr.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_fq.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_gred.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hfsc.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_hhf.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_htb.ko 2023-11-24T07:53:37.656 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_ingress.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_mqprio_lib.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_multiq.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_netem.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_pie.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_plug.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_prio.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_qfq.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_red.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfb.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_sfq.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_tbf.ko 2023-11-24T07:53:37.657 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sched/sch_teql.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sctp/sctp_diag.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/sunrpc.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc 2023-11-24T07:53:37.658 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/diag.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/tipc/tipc.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/unix/unix_diag.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_diag.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/vmw_vsock/vsock_loopback.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/net/xfrm/xfrm_ipcomp.ko 2023-11-24T07:53:37.659 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/security/keys/trusted-keys/trusted.ko 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/kernel/virt/lib/irqbypass.ko 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.alias.bin 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.alias.bin 2023-11-24T07:53:37.660 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.bin 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.builtin.modinfo 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.dep.bin 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.devname 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.order 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.softdep 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols 2023-11-24T07:53:37.661 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.6.0-g8247bbb5b470/modules.symbols.bin 2023-11-24T07:53:37.662 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g8247bbb5b470 2023-11-24T07:53:37.662 DEBUG:teuthology.orchestra.run.smithi156:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-11-24T07:53:37.683 INFO:teuthology.orchestra.run.smithi156.stdout:postinstall scriptlet (using /bin/sh): 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-g8247bbb5b470 -a -r /boot/System.map-6.6.0-g8247bbb5b470 ]; then 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:cp /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:cp /boot/System.map-6.6.0-g8247bbb5b470 /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:rm -f /boot/vmlinuz-6.6.0-g8247bbb5b470 /boot/System.map-6.6.0-g8247bbb5b470 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/installkernel 6.6.0-g8247bbb5b470 /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:rm -f /boot/.vmlinuz-6.6.0-g8247bbb5b470-rpm /boot/.System.map-6.6.0-g8247bbb5b470-rpm 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:preuninstall scriptlet (using /bin/sh): 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:new-kernel-pkg --remove 6.6.0-g8247bbb5b470 --rminitrd --initrdfile=/boot/initramfs-6.6.0-g8247bbb5b470.img 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-11-24T07:53:37.684 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-install remove 6.6.0-g8247bbb5b470 2023-11-24T07:53:37.685 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-11-24T07:53:37.685 INFO:teuthology.orchestra.run.smithi156.stdout:postuninstall scriptlet (using /bin/sh): 2023-11-24T07:53:37.685 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/update-bootloader ]; then 2023-11-24T07:53:37.685 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/update-bootloader --remove 6.6.0-g8247bbb5b470 2023-11-24T07:53:37.685 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-11-24T07:53:37.686 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -qi grub2-tools 2023-11-24T07:53:37.759 INFO:teuthology.orchestra.run.smithi156.stdout:Name : grub2-tools 2023-11-24T07:53:37.759 INFO:teuthology.orchestra.run.smithi156.stdout:Epoch : 1 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Version : 2.06 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Release : 68.el9 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Architecture: x86_64 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Group : Unspecified 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Size : 8264254 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:License : GPLv3+ 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-11-24T07:53:37.760 INFO:teuthology.orchestra.run.smithi156.stdout:Packager : builder@centos.org 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:Vendor : CentOS 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:Summary : Support tools for GRUB. 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:Description : 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:hardware devices. 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T07:53:37.761 INFO:teuthology.orchestra.run.smithi156.stdout:This subpackage provides tools for support of all platforms. 2023-11-24T07:53:37.762 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-24T07:53:37.762 INFO:teuthology.task.kernel:Updating grub on smithi156 to boot 6.6.0-g8247bbb5b470 2023-11-24T07:53:37.762 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-24T07:53:38.427 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2023-11-24T07:53:39.882 INFO:teuthology.orchestra.run.smithi156.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-11-24T07:53:39.901 INFO:teuthology.orchestra.run.smithi156.stderr:done 2023-11-24T07:53:39.904 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-11-24T07:53:39.919 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.26zI98zDn2 2023-11-24T07:53:39.919 DEBUG:teuthology.orchestra.run.smithi156:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.26zI98zDn2 2023-11-24T07:53:39.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.26zI98zDn2 2023-11-24T07:53:40.133 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.26zI98zDn2 is 6KB 2023-11-24T07:53:40.181 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.26zI98zDn2 2023-11-24T07:53:40.195 DEBUG:teuthology.orchestra.run.smithi156:> sudo /bin/ls /boot/loader/entries || true 2023-11-24T07:53:40.260 INFO:teuthology.orchestra.run.smithi156.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-11-24T07:53:40.261 INFO:teuthology.orchestra.run.smithi156.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-11-24T07:53:40.261 INFO:teuthology.orchestra.run.smithi156.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-11-24T07:53:40.261 INFO:teuthology.orchestra.run.smithi156.stdout:15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470.conf 2023-11-24T07:53:40.262 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-g8247bbb5b470 2023-11-24T07:53:40.450 DEBUG:teuthology.orchestra.run.smithi156:> sudo shutdown -r now 2023-11-24T07:53:58.539 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-24T07:53:58.539 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T07:53:58.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:54:10.465 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-24T07:54:10.465 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T07:54:10.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:54:17.058 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-11-24T07:54:26.066 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T07:54:26.067 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:54:28.835 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-11-24T07:54:29.122 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-11-24T07:54:37.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T07:54:37.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:54:40.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-11-24T07:54:41.123 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T07:54:41.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:54:52.914 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T07:54:52.915 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:54:59.554 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.106 2023-11-24T07:55:11.330 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-11-24T07:55:14.558 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T07:55:14.559 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:55:16.048 DEBUG:teuthology.orchestra.run.smithi106:> true 2023-11-24T07:55:16.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T07:55:16.730 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g8247bbb5b470"... 2023-11-24T07:55:16.730 DEBUG:teuthology.orchestra.run.smithi106:> uname -r 2023-11-24T07:55:16.772 INFO:teuthology.orchestra.run.smithi106.stdout:6.6.0-g8247bbb5b470 2023-11-24T07:55:16.773 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T07:55:16.773 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T07:55:16.773 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T07:55:17.774 DEBUG:teuthology.parallel:result is None 2023-11-24T07:55:26.332 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T07:55:26.332 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:55:29.410 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-11-24T07:55:47.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T07:55:47.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T07:55:47.840 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-11-24T07:55:48.730 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T07:55:48.730 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g8247bbb5b470"... 2023-11-24T07:55:48.730 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-11-24T07:55:48.807 INFO:teuthology.orchestra.run.smithi156.stdout:6.6.0-g8247bbb5b470 2023-11-24T07:55:48.807 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g8247bbb5b470 vs 6.6.0-g8247bbb5b470 2023-11-24T07:55:48.807 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-24T07:55:48.807 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-11-24T07:55:49.808 DEBUG:teuthology.parallel:result is None 2023-11-24T07:55:49.809 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-24T07:55:49.818 INFO:teuthology.task.internal:Creating test directory... 2023-11-24T07:55:49.818 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:55:49.829 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-24T07:55:49.887 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-24T07:55:49.893 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-24T07:55:49.963 INFO:teuthology.task.internal:Creating archive directory... 2023-11-24T07:55:49.964 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:55:49.973 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-24T07:55:50.052 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-24T07:55:50.059 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-24T07:55:50.060 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:55:50.068 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-24T07:55:50.215 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:55:50.230 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:55:50.276 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:55:50.305 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-24T07:55:50.309 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-24T07:55:50.316 INFO:teuthology.task.internal:Configuring sudo... 2023-11-24T07:55:50.316 DEBUG:teuthology.orchestra.run.smithi106:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:55:50.324 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-24T07:55:50.405 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-24T07:55:50.417 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-24T07:55:50.417 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:55:50.425 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-24T07:55:50.471 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:55:50.551 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:55:50.660 DEBUG:teuthology.orchestra.run.smithi106:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:55:50.749 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:55:50.844 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T07:55:50.845 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:55:50.938 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:55:51.018 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-24T07:55:51.106 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:55:51.174 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-24T07:55:51.272 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T07:55:51.272 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-24T07:55:51.344 DEBUG:teuthology.orchestra.run.smithi106:> sudo service rsyslog restart 2023-11-24T07:55:51.353 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-11-24T07:55:51.484 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:55:51.682 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T07:55:51.794 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-24T07:55:51.805 INFO:teuthology.task.internal:Starting timer... 2023-11-24T07:55:51.805 INFO:teuthology.run_tasks:Running task pcp... 2023-11-24T07:55:51.842 INFO:teuthology.run_tasks:Running task selinux... 2023-11-24T07:55:51.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo service auditd rotate 2023-11-24T07:55:51.998 INFO:teuthology.orchestra.run.smithi106.stdout:Rotating logs: 2023-11-24T07:55:52.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-11-24T07:55:52.163 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-11-24T07:55:52.169 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-24T07:55:52.169 DEBUG:teuthology.orchestra.run.smithi106:> /usr/sbin/getenforce 2023-11-24T07:55:52.236 INFO:teuthology.orchestra.run.smithi106.stdout:Permissive 2023-11-24T07:55:52.236 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-11-24T07:55:52.303 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-11-24T07:55:52.304 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi106.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive'} 2023-11-24T07:55:52.304 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:55:52.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:55:52.410 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T07:55:52.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T07:55:52.507 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-24T07:55:52.508 DEBUG:teuthology.orchestra.run.smithi106:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:55:52.624 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-11-24T07:55:52.724 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-24T07:55:52.733 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-24T07:55:52.739 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-24T07:55:52.871 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-24T07:55:52.890 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-24T07:55:52.892 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi106.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-24T08:05:51.565 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi106.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-11-24T08:05:51.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T08:05:51.567 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi106.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T08:05:51.704 DEBUG:teuthology.orchestra.run.smithi106:> true 2023-11-24T08:05:51.850 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi106.front.sepia.ceph.com' 2023-11-24T08:05:51.850 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T08:05:51.851 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-24T08:05:51.989 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-11-24T08:05:52.144 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-11-24T08:05:52.144 INFO:teuthology.run_tasks:Running task clock... 2023-11-24T08:05:52.154 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-24T08:05:52.154 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T08:05:52.155 DEBUG:teuthology.orchestra.run.smithi106:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:05:52.164 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-24T08:05:52.164 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:05:52.263 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T08:05:52.270 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-24T08:05:52.315 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T08:05:52.322 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-24T08:05:52.405 INFO:teuthology.orchestra.run.smithi106.stderr:sudo: ntpd: command not found 2023-11-24T08:05:52.416 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-11-24T08:05:52.446 INFO:teuthology.orchestra.run.smithi106.stdout:506 Cannot talk to daemon 2023-11-24T08:05:52.459 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-11-24T08:05:52.494 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T08:05:52.510 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-24T08:05:52.546 INFO:teuthology.orchestra.run.smithi106.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T08:05:52.561 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-24T08:05:52.672 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:05:52.681 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:05:52.681 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2023-11-24T08:05:52.681 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.681 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.681 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.681 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.682 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:05:52.691 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:05:52.691 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-11-24T08:05:52.691 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.691 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.691 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.691 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:05:52.693 INFO:teuthology.run_tasks:Running task install... 2023-11-24T08:05:52.701 DEBUG:teuthology.task.install:project ceph 2023-11-24T08:05:52.701 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}} 2023-11-24T08:05:52.701 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'} 2023-11-24T08:05:52.702 INFO:teuthology.task.install:Using flavor: default 2023-11-24T08:05:52.708 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-24T08:05:52.708 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-24T08:05:52.709 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T08:05:52.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:05:52.711 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'tag': None, 'wait_for_package': False} 2023-11-24T08:05:52.711 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:05:52.857 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T08:05:52.857 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T08:05:52.927 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/ 2023-11-24T08:05:52.927 INFO:teuthology.task.install.rpm:Package version is 18.0.0-7484.ga13ea5ff 2023-11-24T08:05:52.969 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T08:05:52.969 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:05:52.969 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T08:05:53.080 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T08:05:53.081 DEBUG:teuthology.orchestra.run.smithi106:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T08:05:53.103 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/rishabh-mgr-vol-misc/a13ea5ff00cb4df7a9cc941166ddeb358881144c/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-24T08:05:53.103 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:05:53.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-24T08:05:53.205 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-24T08:05:53.205 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a13ea5ff00cb4df7a9cc941166ddeb358881144c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-24T08:05:53.218 DEBUG:teuthology.orchestra.run.smithi106:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T08:05:53.345 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-24T08:05:53.347 DEBUG:teuthology.orchestra.run.smithi106:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T08:05:53.443 INFO:teuthology.orchestra.run.smithi106.stdout:check_obsoletes = 1 2023-11-24T08:05:53.461 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum clean all 2023-11-24T08:05:53.472 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-24T08:05:53.565 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-11-24T08:05:53.569 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-11-24T08:05:53.822 INFO:teuthology.orchestra.run.smithi106.stdout:73 files removed 2023-11-24T08:05:53.859 DEBUG:teuthology.orchestra.run.smithi106:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T08:05:53.938 INFO:teuthology.orchestra.run.smithi156.stdout:73 files removed 2023-11-24T08:05:53.976 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-24T08:05:54.842 INFO:teuthology.orchestra.run.smithi106.stdout:ceph packages for x86_64 162 kB/s | 78 kB 00:00 2023-11-24T08:05:54.941 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 173 kB/s | 78 kB 00:00 2023-11-24T08:05:55.343 INFO:teuthology.orchestra.run.smithi106.stdout:ceph noarch packages 23 kB/s | 9.5 kB 00:00 2023-11-24T08:05:55.399 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 26 kB/s | 9.5 kB 00:00 2023-11-24T08:05:55.877 INFO:teuthology.orchestra.run.smithi106.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2023-11-24T08:05:55.933 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 4.1 kB/s | 1.9 kB 00:00 2023-11-24T08:05:56.614 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - AppStream 9.6 MB/s | 6.4 MB 00:00 2023-11-24T08:05:56.654 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - AppStream 9.9 MB/s | 6.4 MB 00:00 2023-11-24T08:05:58.296 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS-9 - BaseOS 4.9 MB/s | 2.0 MB 00:00 2023-11-24T08:05:58.297 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS-9 - BaseOS 4.4 MB/s | 2.0 MB 00:00 2023-11-24T08:05:59.792 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - BaseOS 7.2 MB/s | 7.9 MB 00:01 2023-11-24T08:05:59.877 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 7.9 MB 00:01 2023-11-24T08:06:03.265 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - AppStream 8.2 MB/s | 18 MB 00:02 2023-11-24T08:06:03.686 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 18 MB 00:02 2023-11-24T08:06:07.802 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 5.8 MB 00:00 2023-11-24T08:06:08.862 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 5.8 MB 00:01 2023-11-24T08:06:09.461 INFO:teuthology.orchestra.run.smithi156.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 15 kB 00:00 2023-11-24T08:06:10.479 INFO:teuthology.orchestra.run.smithi106.stdout:CentOS Stream 9 - Extras packages 47 kB/s | 15 kB 00:00 2023-11-24T08:06:11.897 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 20 MB 00:02 2023-11-24T08:06:12.114 INFO:teuthology.orchestra.run.smithi106.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2023-11-24T08:06:17.621 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 18 kB/s | 1.7 kB 00:00 2023-11-24T08:06:19.885 INFO:teuthology.orchestra.run.smithi106.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2023-11-24T08:06:20.501 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:06:20.503 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:06:20.521 INFO:teuthology.orchestra.run.smithi106.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:06:20.523 INFO:teuthology.orchestra.run.smithi106.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-11-24T08:06:20.524 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T08:06:20.525 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T08:06:20.527 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T08:06:20.528 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T08:06:20.531 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T08:06:20.534 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T08:06:20.536 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T08:06:20.543 INFO:teuthology.orchestra.run.smithi106.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-11-24T08:06:20.545 INFO:teuthology.orchestra.run.smithi106.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-11-24T08:06:20.546 INFO:teuthology.orchestra.run.smithi106.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-11-24T08:06:20.548 INFO:teuthology.orchestra.run.smithi106.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-11-24T08:06:20.550 INFO:teuthology.orchestra.run.smithi106.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-11-24T08:06:20.554 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-11-24T08:06:20.555 INFO:teuthology.orchestra.run.smithi106.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-11-24T08:06:20.617 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-11-24T08:06:20.633 INFO:teuthology.orchestra.run.smithi106.stdout:Dependencies resolved. 2023-11-24T08:06:20.642 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2023-11-24T08:06:20.642 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-11-24T08:06:20.643 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2023-11-24T08:06:20.643 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-11-24T08:06:20.643 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T08:06:20.643 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T08:06:20.643 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T08:06:20.643 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T08:06:20.644 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T08:06:20.644 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T08:06:20.644 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T08:06:20.644 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T08:06:20.644 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T08:06:20.644 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T08:06:20.645 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T08:06:20.645 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T08:06:20.648 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T08:06:20.648 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T08:06:20.648 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T08:06:20.649 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T08:06:20.650 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T08:06:20.650 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T08:06:20.650 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T08:06:20.650 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T08:06:20.650 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T08:06:20.650 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T08:06:20.651 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T08:06:20.651 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T08:06:20.651 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T08:06:20.651 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T08:06:20.651 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T08:06:20.652 INFO:teuthology.orchestra.run.smithi156.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T08:06:20.653 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T08:06:20.654 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T08:06:20.655 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T08:06:20.656 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T08:06:20.657 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T08:06:20.658 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T08:06:20.659 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T08:06:20.660 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout:Installing weak dependencies: 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:06:20.661 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-11-24T08:06:20.662 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================ 2023-11-24T08:06:20.662 INFO:teuthology.orchestra.run.smithi156.stdout:Install 132 Packages 2023-11-24T08:06:20.662 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 15 Packages 2023-11-24T08:06:20.662 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: Package Arch Version Repository Size 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout:Installing: 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: ceph x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 6.5 k 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 5.2 M 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 859 k 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 141 k 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 1.5 M 2023-11-24T08:06:20.663 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 133 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 1.8 M 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 7.4 M 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 50 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 8.6 M 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 41 M 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 262 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 772 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 217 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 18 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-11-24T08:06:20.664 INFO:teuthology.orchestra.run.smithi106.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 31 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 719 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 126 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 162 k 2023-11-24T08:06:20.665 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 319 k 2023-11-24T08:06:20.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 298 k 2023-11-24T08:06:20.666 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 99 k 2023-11-24T08:06:20.666 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 85 k 2023-11-24T08:06:20.666 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:06:20.666 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 170 k 2023-11-24T08:06:20.666 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-11-24T08:06:20.667 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrading: 2023-11-24T08:06:20.667 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager x86_64 1:1.45.6-1.el9 CentOS-BaseOS 2.2 M 2023-11-24T08:06:20.667 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm x86_64 1:1.45.6-1.el9 CentOS-BaseOS 1.8 M 2023-11-24T08:06:20.667 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team x86_64 1:1.45.6-1.el9 CentOS-BaseOS 42 k 2023-11-24T08:06:20.667 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui x86_64 1:1.45.6-1.el9 CentOS-BaseOS 245 k 2023-11-24T08:06:20.667 INFO:teuthology.orchestra.run.smithi106.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: librados2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.3 M 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 3.0 M 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-11-24T08:06:20.668 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout:Installing dependencies: 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 19 M 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 28 k 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 2.1 M 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 243 k 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.4 M 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 18 M 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts noarch 2:18.0.0-7484.ga13ea5ff.el9 ceph-noarch 15 k 2023-11-24T08:06:20.669 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 24 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow x86_64 9.0.0-9.el9 epel 4.4 M 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc noarch 9.0.0-9.el9 epel 24 k 2023-11-24T08:06:20.670 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 163 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 476 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2 x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 4.8 M 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-11-24T08:06:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs x86_64 9.0.0-9.el9 epel 837 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 45 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common x86_64 2:18.0.0-7484.ga13ea5ff.el9 ceph 129 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-11-24T08:06:20.672 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-11-24T08:06:20.673 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-11-24T08:06:20.674 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-11-24T08:06:20.675 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-11-24T08:06:20.676 INFO:teuthology.orchestra.run.smithi106.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout:Installing weak dependencies: 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction Summary 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout:============================================================================================ 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout:Install 132 Packages 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout:Upgrade 15 Packages 2023-11-24T08:06:20.677 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:06:20.678 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 204 M 2023-11-24T08:06:20.678 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-11-24T08:06:20.678 INFO:teuthology.orchestra.run.smithi106.stdout:Total download size: 204 M 2023-11-24T08:06:20.678 INFO:teuthology.orchestra.run.smithi106.stdout:Downloading Packages: 2023-11-24T08:06:21.606 INFO:teuthology.orchestra.run.smithi156.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 22 kB/s | 6.5 kB 00:00 2023-11-24T08:06:22.032 INFO:teuthology.orchestra.run.smithi106.stdout:(1/147): ceph-18.0.0-7484.ga13ea5ff.el9.x86_64. 17 kB/s | 6.5 kB 00:00 2023-11-24T08:06:22.606 INFO:teuthology.orchestra.run.smithi156.stdout:(2/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 15 MB/s | 19 MB 00:01 2023-11-24T08:06:22.648 INFO:teuthology.orchestra.run.smithi156.stdout:(3/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 826 kB/s | 859 kB 00:01 2023-11-24T08:06:22.706 INFO:teuthology.orchestra.run.smithi156.stdout:(4/147): ceph-immutable-object-cache-18.0.0-748 1.4 MB/s | 141 kB 00:00 2023-11-24T08:06:22.848 INFO:teuthology.orchestra.run.smithi156.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 10 MB/s | 1.5 MB 00:00 2023-11-24T08:06:22.874 INFO:teuthology.orchestra.run.smithi106.stdout:(2/147): ceph-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.0 MB/s | 859 kB 00:00 2023-11-24T08:06:23.083 INFO:teuthology.orchestra.run.smithi106.stdout:(3/147): ceph-immutable-object-cache-18.0.0-748 681 kB/s | 141 kB 00:00 2023-11-24T08:06:23.215 INFO:teuthology.orchestra.run.smithi156.stdout:(6/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 12 MB/s | 4.4 MB 00:00 2023-11-24T08:06:23.342 INFO:teuthology.orchestra.run.smithi106.stdout:(4/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 3.1 MB/s | 5.2 MB 00:01 2023-11-24T08:06:23.707 INFO:teuthology.orchestra.run.smithi156.stdout:(7/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 2.0 MB/s | 2.1 MB 00:01 2023-11-24T08:06:23.824 INFO:teuthology.orchestra.run.smithi156.stdout:(8/147): ceph-base-18.0.0-7484.ga13ea5ff.el9.x8 2.1 MB/s | 5.2 MB 00:02 2023-11-24T08:06:23.859 INFO:teuthology.orchestra.run.smithi106.stdout:(5/147): ceph-mgr-18.0.0-7484.ga13ea5ff.el9.x86 2.8 MB/s | 1.5 MB 00:00 2023-11-24T08:06:23.924 INFO:teuthology.orchestra.run.smithi156.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 243 kB/s | 24 kB 00:00 2023-11-24T08:06:24.268 INFO:teuthology.orchestra.run.smithi106.stdout:(6/147): ceph-mds-18.0.0-7484.ga13ea5ff.el9.x86 1.8 MB/s | 2.1 MB 00:01 2023-11-24T08:06:24.850 INFO:teuthology.orchestra.run.smithi156.stdout:(10/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 11 MB/s | 18 MB 00:01 2023-11-24T08:06:24.925 INFO:teuthology.orchestra.run.smithi156.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.9 MB/s | 217 kB 00:00 2023-11-24T08:06:25.000 INFO:teuthology.orchestra.run.smithi156.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 421 kB/s | 31 kB 00:00 2023-11-24T08:06:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 4.5 MB/s | 719 kB 00:00 2023-11-24T08:06:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.9 MB/s | 163 kB 00:00 2023-11-24T08:06:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.5 MB/s | 126 kB 00:00 2023-11-24T08:06:25.418 INFO:teuthology.orchestra.run.smithi106.stdout:(7/147): ceph-mon-18.0.0-7484.ga13ea5ff.el9.x86 2.8 MB/s | 4.4 MB 00:01 2023-11-24T08:06:25.451 INFO:teuthology.orchestra.run.smithi156.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 3.8 MB/s | 476 kB 00:00 2023-11-24T08:06:25.660 INFO:teuthology.orchestra.run.smithi106.stdout:(8/147): ceph-common-18.0.0-7484.ga13ea5ff.el9. 4.7 MB/s | 19 MB 00:04 2023-11-24T08:06:25.745 INFO:teuthology.orchestra.run.smithi106.stdout:(9/147): ceph-selinux-18.0.0-7484.ga13ea5ff.el9 293 kB/s | 24 kB 00:00 2023-11-24T08:06:26.185 INFO:teuthology.orchestra.run.smithi156.stdout:(17/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 6.5 MB/s | 4.8 MB 00:00 2023-11-24T08:06:26.268 INFO:teuthology.orchestra.run.smithi156.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 549 kB/s | 45 kB 00:00 2023-11-24T08:06:26.351 INFO:teuthology.orchestra.run.smithi156.stdout:(19/147): python3-ceph-common-18.0.0-7484.ga13e 1.5 MB/s | 129 kB 00:00 2023-11-24T08:06:26.444 INFO:teuthology.orchestra.run.smithi156.stdout:(20/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.7 MB/s | 162 kB 00:00 2023-11-24T08:06:26.544 INFO:teuthology.orchestra.run.smithi156.stdout:(21/147): python3-rados-18.0.0-7484.ga13ea5ff.e 3.1 MB/s | 319 kB 00:00 2023-11-24T08:06:26.653 INFO:teuthology.orchestra.run.smithi156.stdout:(22/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 2.7 MB/s | 298 kB 00:00 2023-11-24T08:06:26.745 INFO:teuthology.orchestra.run.smithi156.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.1 MB/s | 99 kB 00:00 2023-11-24T08:06:26.837 INFO:teuthology.orchestra.run.smithi156.stdout:(24/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 941 kB/s | 85 kB 00:00 2023-11-24T08:06:27.087 INFO:teuthology.orchestra.run.smithi156.stdout:(25/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 2.6 MB/s | 8.6 MB 00:03 2023-11-24T08:06:27.127 INFO:teuthology.orchestra.run.smithi106.stdout:(10/147): ceph-osd-18.0.0-7484.ga13ea5ff.el9.x8 6.2 MB/s | 18 MB 00:02 2023-11-24T08:06:27.187 INFO:teuthology.orchestra.run.smithi156.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 1.7 MB/s | 170 kB 00:00 2023-11-24T08:06:27.235 INFO:teuthology.orchestra.run.smithi106.stdout:(11/147): cephfs-mirror-18.0.0-7484.ga13ea5ff.e 2.0 MB/s | 217 kB 00:00 2023-11-24T08:06:27.271 INFO:teuthology.orchestra.run.smithi156.stdout:(27/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 7.0 MB/s | 3.0 MB 00:00 2023-11-24T08:06:27.296 INFO:teuthology.orchestra.run.smithi156.stdout:(28/147): ceph-grafana-dashboards-18.0.0-7484.g 261 kB/s | 28 kB 00:00 2023-11-24T08:06:27.319 INFO:teuthology.orchestra.run.smithi106.stdout:(12/147): libcephfs-devel-18.0.0-7484.ga13ea5ff 378 kB/s | 31 kB 00:00 2023-11-24T08:06:27.329 INFO:teuthology.orchestra.run.smithi156.stdout:(29/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.3 MB/s | 133 kB 00:00 2023-11-24T08:06:27.480 INFO:teuthology.orchestra.run.smithi106.stdout:(13/147): libcephfs2-18.0.0-7484.ga13ea5ff.el9. 4.4 MB/s | 719 kB 00:00 2023-11-24T08:06:27.580 INFO:teuthology.orchestra.run.smithi106.stdout:(14/147): libcephsqlite-18.0.0-7484.ga13ea5ff.e 1.6 MB/s | 163 kB 00:00 2023-11-24T08:06:27.680 INFO:teuthology.orchestra.run.smithi106.stdout:(15/147): librados-devel-18.0.0-7484.ga13ea5ff. 1.2 MB/s | 126 kB 00:00 2023-11-24T08:06:27.806 INFO:teuthology.orchestra.run.smithi106.stdout:(16/147): libradosstriper1-18.0.0-7484.ga13ea5f 3.7 MB/s | 476 kB 00:00 2023-11-24T08:06:27.922 INFO:teuthology.orchestra.run.smithi156.stdout:(30/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 2.8 MB/s | 1.8 MB 00:00 2023-11-24T08:06:28.406 INFO:teuthology.orchestra.run.smithi106.stdout:(17/147): ceph-radosgw-18.0.0-7484.ga13ea5ff.el 2.9 MB/s | 8.6 MB 00:02 2023-11-24T08:06:28.413 INFO:teuthology.orchestra.run.smithi156.stdout:(31/147): ceph-mgr-diskprediction-local-18.0.0- 6.8 MB/s | 7.4 MB 00:01 2023-11-24T08:06:28.447 INFO:teuthology.orchestra.run.smithi156.stdout:(32/147): ceph-mgr-modules-core-18.0.0-7484.ga1 463 kB/s | 243 kB 00:00 2023-11-24T08:06:28.456 INFO:teuthology.orchestra.run.smithi106.stdout:(18/147): python3-ceph-argparse-18.0.0-7484.ga1 923 kB/s | 45 kB 00:00 2023-11-24T08:06:28.472 INFO:teuthology.orchestra.run.smithi156.stdout:(33/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 873 kB/s | 50 kB 00:00 2023-11-24T08:06:28.531 INFO:teuthology.orchestra.run.smithi156.stdout:(34/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 4.4 MB/s | 262 kB 00:00 2023-11-24T08:06:28.556 INFO:teuthology.orchestra.run.smithi156.stdout:(35/147): ceph-prometheus-alerts-18.0.0-7484.ga 140 kB/s | 15 kB 00:00 2023-11-24T08:06:28.590 INFO:teuthology.orchestra.run.smithi106.stdout:(19/147): librgw2-18.0.0-7484.ga13ea5ff.el9.x86 6.1 MB/s | 4.8 MB 00:00 2023-11-24T08:06:28.615 INFO:teuthology.orchestra.run.smithi156.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 316 kB/s | 18 kB 00:00 2023-11-24T08:06:28.615 INFO:teuthology.orchestra.run.smithi106.stdout:(20/147): python3-ceph-common-18.0.0-7484.ga13e 817 kB/s | 129 kB 00:00 2023-11-24T08:06:28.690 INFO:teuthology.orchestra.run.smithi106.stdout:(21/147): python3-cephfs-18.0.0-7484.ga13ea5ff. 1.6 MB/s | 162 kB 00:00 2023-11-24T08:06:28.724 INFO:teuthology.orchestra.run.smithi106.stdout:(22/147): python3-rados-18.0.0-7484.ga13ea5ff.e 2.9 MB/s | 319 kB 00:00 2023-11-24T08:06:28.783 INFO:teuthology.orchestra.run.smithi106.stdout:(23/147): python3-rgw-18.0.0-7484.ga13ea5ff.el9 1.7 MB/s | 99 kB 00:00 2023-11-24T08:06:28.816 INFO:teuthology.orchestra.run.smithi106.stdout:(24/147): python3-rbd-18.0.0-7484.ga13ea5ff.el9 2.3 MB/s | 298 kB 00:00 2023-11-24T08:06:28.841 INFO:teuthology.orchestra.run.smithi106.stdout:(25/147): rbd-fuse-18.0.0-7484.ga13ea5ff.el9.x8 1.4 MB/s | 85 kB 00:00 2023-11-24T08:06:28.873 INFO:teuthology.orchestra.run.smithi156.stdout:(37/147): automake-1.16.2-8.el9.noarch.rpm 2.5 MB/s | 662 kB 00:00 2023-11-24T08:06:28.917 INFO:teuthology.orchestra.run.smithi106.stdout:(26/147): rbd-nbd-18.0.0-7484.ga13ea5ff.el9.x86 2.2 MB/s | 170 kB 00:00 2023-11-24T08:06:28.975 INFO:teuthology.orchestra.run.smithi106.stdout:(27/147): ceph-grafana-dashboards-18.0.0-7484.g 488 kB/s | 28 kB 00:00 2023-11-24T08:06:28.976 INFO:teuthology.orchestra.run.smithi156.stdout:(38/147): bison-3.7.4-5.el9.x86_64.rpm 8.8 MB/s | 920 kB 00:00 2023-11-24T08:06:29.024 INFO:teuthology.orchestra.run.smithi156.stdout:(39/147): boost-program-options-1.75.0-8.el9.x8 2.2 MB/s | 106 kB 00:00 2023-11-24T08:06:29.042 INFO:teuthology.orchestra.run.smithi106.stdout:(28/147): ceph-mgr-cephadm-18.0.0-7484.ga13ea5f 2.0 MB/s | 133 kB 00:00 2023-11-24T08:06:29.057 INFO:teuthology.orchestra.run.smithi156.stdout:(40/147): elfutils-libelf-devel-0.190-1.el9.x86 660 kB/s | 21 kB 00:00 2023-11-24T08:06:29.091 INFO:teuthology.orchestra.run.smithi156.stdout:(41/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 1.3 MB/s | 772 kB 00:00 2023-11-24T08:06:29.125 INFO:teuthology.orchestra.run.smithi156.stdout:(42/147): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 307 kB 00:00 2023-11-24T08:06:29.142 INFO:teuthology.orchestra.run.smithi156.stdout:(43/147): flexiblas-3.0.4-8.el9.x86_64.rpm 676 kB/s | 32 kB 00:00 2023-11-24T08:06:29.205 INFO:teuthology.orchestra.run.smithi106.stdout:(29/147): rbd-mirror-18.0.0-7484.ga13ea5ff.el9. 7.8 MB/s | 3.0 MB 00:00 2023-11-24T08:06:29.259 INFO:teuthology.orchestra.run.smithi156.stdout:(44/147): flexiblas-openblas-openmp-3.0.4-8.el9 147 kB/s | 17 kB 00:00 2023-11-24T08:06:29.342 INFO:teuthology.orchestra.run.smithi156.stdout:(45/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2023-11-24T08:06:29.393 INFO:teuthology.orchestra.run.smithi156.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 1.4 MB/s | 186 kB 00:00 2023-11-24T08:06:29.418 INFO:teuthology.orchestra.run.smithi156.stdout:(47/147): libaio-devel-0.3.111-13.el9.x86_64.rp 138 kB/s | 10 kB 00:00 2023-11-24T08:06:29.443 INFO:teuthology.orchestra.run.smithi156.stdout:(48/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.2 MB/s | 159 kB 00:00 2023-11-24T08:06:29.471 INFO:teuthology.orchestra.run.smithi156.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 879 kB/s | 44 kB 00:00 2023-11-24T08:06:29.513 INFO:teuthology.orchestra.run.smithi156.stdout:(50/147): librdkafka-1.6.1-102.el9.x86_64.rpm 9.5 MB/s | 661 kB 00:00 2023-11-24T08:06:29.526 INFO:teuthology.orchestra.run.smithi106.stdout:(30/147): ceph-mgr-dashboard-18.0.0-7484.ga13ea 3.6 MB/s | 1.8 MB 00:00 2023-11-24T08:06:29.555 INFO:teuthology.orchestra.run.smithi156.stdout:(51/147): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2023-11-24T08:06:29.580 INFO:teuthology.orchestra.run.smithi156.stdout:(52/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 399 kB/s | 24 kB 00:00 2023-11-24T08:06:29.605 INFO:teuthology.orchestra.run.smithi156.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2023-11-24T08:06:29.610 INFO:teuthology.orchestra.run.smithi106.stdout:(31/147): ceph-mgr-modules-core-18.0.0-7484.ga1 2.9 MB/s | 243 kB 00:00 2023-11-24T08:06:29.647 INFO:teuthology.orchestra.run.smithi156.stdout:(54/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.2 MB/s | 282 kB 00:00 2023-11-24T08:06:29.668 INFO:teuthology.orchestra.run.smithi106.stdout:(32/147): ceph-mgr-rook-18.0.0-7484.ga13ea5ff.e 877 kB/s | 50 kB 00:00 2023-11-24T08:06:29.713 INFO:teuthology.orchestra.run.smithi156.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.0 MB/s | 217 kB 00:00 2023-11-24T08:06:29.735 INFO:teuthology.orchestra.run.smithi106.stdout:(33/147): ceph-prometheus-alerts-18.0.0-7484.ga 230 kB/s | 15 kB 00:00 2023-11-24T08:06:29.873 INFO:teuthology.orchestra.run.smithi156.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 153 kB/s | 34 kB 00:00 2023-11-24T08:06:29.886 INFO:teuthology.orchestra.run.smithi106.stdout:(34/147): ceph-mgr-diskprediction-local-18.0.0- 11 MB/s | 7.4 MB 00:00 2023-11-24T08:06:29.919 INFO:teuthology.orchestra.run.smithi106.stdout:(35/147): ceph-volume-18.0.0-7484.ga13ea5ff.el9 1.4 MB/s | 262 kB 00:00 2023-11-24T08:06:29.970 INFO:teuthology.orchestra.run.smithi106.stdout:(36/147): cephfs-top-18.0.0-7484.ga13ea5ff.el9. 368 kB/s | 18 kB 00:00 2023-11-24T08:06:30.007 INFO:teuthology.orchestra.run.smithi156.stdout:(57/147): openblas-openmp-0.3.21-2.el9.x86_64.r 16 MB/s | 4.7 MB 00:00 2023-11-24T08:06:30.011 INFO:teuthology.orchestra.run.smithi106.stdout:(37/147): cephadm-18.0.0-7484.ga13ea5ff.el9.noa 6.1 MB/s | 772 kB 00:00 2023-11-24T08:06:30.082 INFO:teuthology.orchestra.run.smithi156.stdout:(58/147): postgresql-13.11-1.el9.x86_64.rpm 7.8 MB/s | 1.5 MB 00:00 2023-11-24T08:06:30.156 INFO:teuthology.orchestra.run.smithi156.stdout:(59/147): postgresql-contrib-13.11-1.el9.x86_64 5.4 MB/s | 811 kB 00:00 2023-11-24T08:06:30.191 INFO:teuthology.orchestra.run.smithi156.stdout:(60/147): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2023-11-24T08:06:30.295 INFO:teuthology.orchestra.run.smithi106.stdout:(38/147): automake-1.16.2-8.el9.noarch.rpm 2.0 MB/s | 662 kB 00:00 2023-11-24T08:06:30.333 INFO:teuthology.orchestra.run.smithi156.stdout:(61/147): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-11-24T08:06:30.353 INFO:teuthology.orchestra.run.smithi106.stdout:(39/147): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 920 kB 00:00 2023-11-24T08:06:30.370 INFO:teuthology.orchestra.run.smithi106.stdout:(40/147): boost-program-options-1.75.0-8.el9.x8 1.5 MB/s | 106 kB 00:00 2023-11-24T08:06:30.404 INFO:teuthology.orchestra.run.smithi106.stdout:(41/147): elfutils-libelf-devel-0.190-1.el9.x86 443 kB/s | 21 kB 00:00 2023-11-24T08:06:30.438 INFO:teuthology.orchestra.run.smithi106.stdout:(42/147): flex-2.6.4-9.el9.x86_64.rpm 4.5 MB/s | 307 kB 00:00 2023-11-24T08:06:30.466 INFO:teuthology.orchestra.run.smithi156.stdout:(62/147): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2023-11-24T08:06:30.491 INFO:teuthology.orchestra.run.smithi156.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.6 MB/s | 228 kB 00:00 2023-11-24T08:06:30.501 INFO:teuthology.orchestra.run.smithi106.stdout:(43/147): flexiblas-3.0.4-8.el9.x86_64.rpm 339 kB/s | 32 kB 00:00 2023-11-24T08:06:30.558 INFO:teuthology.orchestra.run.smithi156.stdout:(64/147): python3-jsonpointer-2.0-4.el9.noarch. 276 kB/s | 18 kB 00:00 2023-11-24T08:06:30.584 INFO:teuthology.orchestra.run.smithi156.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 268 kB/s | 24 kB 00:00 2023-11-24T08:06:30.605 INFO:teuthology.orchestra.run.smithi106.stdout:(44/147): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2023-11-24T08:06:30.609 INFO:teuthology.orchestra.run.smithi156.stdout:(66/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T08:06:30.630 INFO:teuthology.orchestra.run.smithi106.stdout:(45/147): flexiblas-openblas-openmp-3.0.4-8.el9 143 kB/s | 17 kB 00:00 2023-11-24T08:06:30.634 INFO:teuthology.orchestra.run.smithi156.stdout:(67/147): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2023-11-24T08:06:30.664 INFO:teuthology.orchestra.run.smithi106.stdout:(46/147): jq-1.6-16.el9.x86_64.rpm 3.2 MB/s | 186 kB 00:00 2023-11-24T08:06:30.689 INFO:teuthology.orchestra.run.smithi106.stdout:(47/147): libaio-devel-0.3.111-13.el9.x86_64.rp 179 kB/s | 10 kB 00:00 2023-11-24T08:06:30.714 INFO:teuthology.orchestra.run.smithi106.stdout:(48/147): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2023-11-24T08:06:30.740 INFO:teuthology.orchestra.run.smithi106.stdout:(49/147): librabbitmq-0.11.0-7.el9.x86_64.rpm 921 kB/s | 44 kB 00:00 2023-11-24T08:06:30.768 INFO:teuthology.orchestra.run.smithi156.stdout:(68/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 4.8 MB 00:00 2023-11-24T08:06:30.790 INFO:teuthology.orchestra.run.smithi106.stdout:(50/147): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 661 kB 00:00 2023-11-24T08:06:30.793 INFO:teuthology.orchestra.run.smithi156.stdout:(69/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-11-24T08:06:30.824 INFO:teuthology.orchestra.run.smithi106.stdout:(51/147): libtool-2.4.6-45.el9.x86_64.rpm 7.3 MB/s | 578 kB 00:00 2023-11-24T08:06:30.849 INFO:teuthology.orchestra.run.smithi106.stdout:(52/147): libuuid-devel-2.37.4-15.el9.x86_64.rp 460 kB/s | 24 kB 00:00 2023-11-24T08:06:30.874 INFO:teuthology.orchestra.run.smithi106.stdout:(53/147): libzstd-devel-1.5.1-2.el9.x86_64.rpm 942 kB/s | 46 kB 00:00 2023-11-24T08:06:30.908 INFO:teuthology.orchestra.run.smithi106.stdout:(54/147): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 282 kB 00:00 2023-11-24T08:06:30.941 INFO:teuthology.orchestra.run.smithi106.stdout:(55/147): oniguruma-6.9.6-1.el9.5.x86_64.rpm 3.3 MB/s | 217 kB 00:00 2023-11-24T08:06:30.967 INFO:teuthology.orchestra.run.smithi106.stdout:(56/147): openblas-0.3.21-2.el9.x86_64.rpm 608 kB/s | 34 kB 00:00 2023-11-24T08:06:31.121 INFO:teuthology.orchestra.run.smithi106.stdout:(57/147): openblas-openmp-0.3.21-2.el9.x86_64.r 27 MB/s | 4.7 MB 00:00 2023-11-24T08:06:31.175 INFO:teuthology.orchestra.run.smithi106.stdout:(58/147): postgresql-13.11-1.el9.x86_64.rpm 7.9 MB/s | 1.5 MB 00:00 2023-11-24T08:06:31.226 INFO:teuthology.orchestra.run.smithi106.stdout:(59/147): postgresql-contrib-13.11-1.el9.x86_64 7.6 MB/s | 811 kB 00:00 2023-11-24T08:06:31.251 INFO:teuthology.orchestra.run.smithi106.stdout:(60/147): postgresql-private-libs-13.11-1.el9.x 1.9 MB/s | 134 kB 00:00 2023-11-24T08:06:31.333 INFO:teuthology.orchestra.run.smithi156.stdout:(70/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 5.6 MB/s | 41 MB 00:07 2023-11-24T08:06:31.369 INFO:teuthology.orchestra.run.smithi156.stdout:(71/147): python3-oauthlib-3.1.1-5.el9.noarch.r 304 kB/s | 174 kB 00:00 2023-11-24T08:06:31.385 INFO:teuthology.orchestra.run.smithi156.stdout:(72/147): python3-prettytable-0.7.2-27.el9.noar 70 kB/s | 41 kB 00:00 2023-11-24T08:06:31.401 INFO:teuthology.orchestra.run.smithi106.stdout:(61/147): postgresql-server-13.11-1.el9.x86_64. 33 MB/s | 5.7 MB 00:00 2023-11-24T08:06:31.402 INFO:teuthology.orchestra.run.smithi156.stdout:(73/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2023-11-24T08:06:31.428 INFO:teuthology.orchestra.run.smithi156.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 1.2 MB/s | 47 kB 00:00 2023-11-24T08:06:31.461 INFO:teuthology.orchestra.run.smithi156.stdout:(75/147): python3-pyasn1-modules-0.4.8-6.el9.no 2.8 MB/s | 211 kB 00:00 2023-11-24T08:06:31.478 INFO:teuthology.orchestra.run.smithi156.stdout:(76/147): python3-requests-oauthlib-1.3.0-12.el 571 kB/s | 43 kB 00:00 2023-11-24T08:06:31.527 INFO:teuthology.orchestra.run.smithi106.stdout:(62/147): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 5.8 MB 00:00 2023-11-24T08:06:31.552 INFO:teuthology.orchestra.run.smithi106.stdout:(63/147): python3-jinja2-2.11.3-4.el9.noarch.rp 1.6 MB/s | 228 kB 00:00 2023-11-24T08:06:31.602 INFO:teuthology.orchestra.run.smithi106.stdout:(64/147): python3-jsonpointer-2.0-4.el9.noarch. 371 kB/s | 18 kB 00:00 2023-11-24T08:06:31.627 INFO:teuthology.orchestra.run.smithi106.stdout:(65/147): python3-jsonpatch-1.21-16.el9.noarch. 329 kB/s | 24 kB 00:00 2023-11-24T08:06:31.653 INFO:teuthology.orchestra.run.smithi106.stdout:(66/147): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 144 kB 00:00 2023-11-24T08:06:31.678 INFO:teuthology.orchestra.run.smithi106.stdout:(67/147): python3-markupsafe-1.1.1-12.el9.x86_6 655 kB/s | 32 kB 00:00 2023-11-24T08:06:31.804 INFO:teuthology.orchestra.run.smithi156.stdout:(77/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2023-11-24T08:06:31.820 INFO:teuthology.orchestra.run.smithi156.stdout:(78/147): python3-toml-0.10.2-6.el9.noarch.rpm 118 kB/s | 39 kB 00:00 2023-11-24T08:06:31.845 INFO:teuthology.orchestra.run.smithi106.stdout:(68/147): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2023-11-24T08:06:31.854 INFO:teuthology.orchestra.run.smithi156.stdout:(79/147): socat-1.7.4.1-5.el9.x86_64.rpm 831 kB/s | 300 kB 00:00 2023-11-24T08:06:31.890 INFO:teuthology.orchestra.run.smithi106.stdout:(69/147): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2023-11-24T08:06:31.896 INFO:teuthology.orchestra.run.smithi156.stdout:(80/147): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 56 kB 00:00 2023-11-24T08:06:31.921 INFO:teuthology.orchestra.run.smithi156.stdout:(81/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.3 MB/s | 85 kB 00:00 2023-11-24T08:06:31.937 INFO:teuthology.orchestra.run.smithi156.stdout:(82/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 768 kB/s | 63 kB 00:00 2023-11-24T08:06:31.954 INFO:teuthology.orchestra.run.smithi106.stdout:(70/147): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 174 kB 00:00 2023-11-24T08:06:31.979 INFO:teuthology.orchestra.run.smithi106.stdout:(71/147): python3-prettytable-0.7.2-27.el9.noar 456 kB/s | 41 kB 00:00 2023-11-24T08:06:31.980 INFO:teuthology.orchestra.run.smithi156.stdout:(83/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 1.1 MB/s | 44 kB 00:00 2023-11-24T08:06:32.005 INFO:teuthology.orchestra.run.smithi156.stdout:(84/147): fuse-2.9.9-15.el9.x86_64.rpm 1.2 MB/s | 79 kB 00:00 2023-11-24T08:06:32.046 INFO:teuthology.orchestra.run.smithi106.stdout:(72/147): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.0 MB/s | 132 kB 00:00 2023-11-24T08:06:32.047 INFO:teuthology.orchestra.run.smithi156.stdout:(85/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-11-24T08:06:32.096 INFO:teuthology.orchestra.run.smithi106.stdout:(73/147): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 211 kB 00:00 2023-11-24T08:06:32.140 INFO:teuthology.orchestra.run.smithi156.stdout:(86/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-11-24T08:06:32.146 INFO:teuthology.orchestra.run.smithi106.stdout:(74/147): python3-pytz-2021.1-5.el9.noarch.rpm 964 kB/s | 47 kB 00:00 2023-11-24T08:06:32.172 INFO:teuthology.orchestra.run.smithi106.stdout:(75/147): python3-requests-oauthlib-1.3.0-12.el 578 kB/s | 43 kB 00:00 2023-11-24T08:06:32.172 INFO:teuthology.orchestra.run.smithi156.stdout:(87/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.9 MB/s | 241 kB 00:00 2023-11-24T08:06:32.215 INFO:teuthology.orchestra.run.smithi156.stdout:(88/147): python3-cryptography-36.0.1-4.el9.x86 6.9 MB/s | 1.1 MB 00:00 2023-11-24T08:06:32.222 INFO:teuthology.orchestra.run.smithi106.stdout:(76/147): python3-toml-0.10.2-6.el9.noarch.rpm 794 kB/s | 39 kB 00:00 2023-11-24T08:06:32.265 INFO:teuthology.orchestra.run.smithi156.stdout:(89/147): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 103 kB 00:00 2023-11-24T08:06:32.290 INFO:teuthology.orchestra.run.smithi156.stdout:(90/147): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-11-24T08:06:32.315 INFO:teuthology.orchestra.run.smithi156.stdout:(91/147): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 124 kB 00:00 2023-11-24T08:06:32.349 INFO:teuthology.orchestra.run.smithi156.stdout:(92/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T08:06:32.514 INFO:teuthology.orchestra.run.smithi106.stdout:(77/147): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2023-11-24T08:06:32.547 INFO:teuthology.orchestra.run.smithi106.stdout:(78/147): socat-1.7.4.1-5.el9.x86_64.rpm 962 kB/s | 300 kB 00:00 2023-11-24T08:06:32.549 INFO:teuthology.orchestra.run.smithi156.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 1.5 MB/s | 309 kB 00:00 2023-11-24T08:06:32.573 INFO:teuthology.orchestra.run.smithi106.stdout:(79/147): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 56 kB 00:00 2023-11-24T08:06:32.598 INFO:teuthology.orchestra.run.smithi106.stdout:(80/147): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.7 MB/s | 85 kB 00:00 2023-11-24T08:06:32.623 INFO:teuthology.orchestra.run.smithi106.stdout:(81/147): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 63 kB 00:00 2023-11-24T08:06:32.649 INFO:teuthology.orchestra.run.smithi106.stdout:(82/147): zlib-devel-1.2.11-40.el9.x86_64.rpm 897 kB/s | 44 kB 00:00 2023-11-24T08:06:32.674 INFO:teuthology.orchestra.run.smithi106.stdout:(83/147): fuse-2.9.9-15.el9.x86_64.rpm 1.6 MB/s | 79 kB 00:00 2023-11-24T08:06:32.716 INFO:teuthology.orchestra.run.smithi106.stdout:(84/147): libgfortran-11.4.1-2.3.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2023-11-24T08:06:32.741 INFO:teuthology.orchestra.run.smithi156.stdout:(94/147): libarrow-9.0.0-9.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2023-11-24T08:06:32.776 INFO:teuthology.orchestra.run.smithi106.stdout:(85/147): libquadmath-11.4.1-2.3.el9.x86_64.rpm 3.2 MB/s | 195 kB 00:00 2023-11-24T08:06:32.776 INFO:teuthology.orchestra.run.smithi106.stdout:(86/147): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 241 kB 00:00 2023-11-24T08:06:32.783 INFO:teuthology.orchestra.run.smithi156.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 593 kB/s | 24 kB 00:00 2023-11-24T08:06:32.825 INFO:teuthology.orchestra.run.smithi106.stdout:(87/147): python3-cryptography-36.0.1-4.el9.x86 14 MB/s | 1.1 MB 00:00 2023-11-24T08:06:32.990 INFO:teuthology.orchestra.run.smithi156.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 988 kB/s | 49 kB 00:00 2023-11-24T08:06:32.991 INFO:teuthology.orchestra.run.smithi156.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2023-11-24T08:06:32.991 INFO:teuthology.orchestra.run.smithi156.stdout:(98/147): python3-devel-3.9.17-2.el9.x86_64.rpm 393 kB/s | 245 kB 00:00 2023-11-24T08:06:32.991 INFO:teuthology.orchestra.run.smithi106.stdout:(88/147): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2023-11-24T08:06:32.991 INFO:teuthology.orchestra.run.smithi106.stdout:(89/147): python3-pycparser-2.20-6.el9.noarch.r 2.5 MB/s | 124 kB 00:00 2023-11-24T08:06:32.991 INFO:teuthology.orchestra.run.smithi106.stdout:(90/147): python3-requests-2.25.1-7.el9.noarch. 1.9 MB/s | 113 kB 00:00 2023-11-24T08:06:32.991 INFO:teuthology.orchestra.run.smithi106.stdout:(91/147): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-11-24T08:06:32.992 INFO:teuthology.orchestra.run.smithi156.stdout:(99/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 7.6 MB/s | 837 kB 00:00 2023-11-24T08:06:33.043 INFO:teuthology.orchestra.run.smithi156.stdout:(100/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 886 kB/s | 43 kB 00:00 2023-11-24T08:06:33.084 INFO:teuthology.orchestra.run.smithi156.stdout:(101/147): python3-cachetools-4.2.4-1.el9.noarc 787 kB/s | 32 kB 00:00 2023-11-24T08:06:33.126 INFO:teuthology.orchestra.run.smithi156.stdout:(102/147): python3-certifi-2021.10.8-2.el9.noar 367 kB/s | 15 kB 00:00 2023-11-24T08:06:33.185 INFO:teuthology.orchestra.run.smithi156.stdout:(103/147): python3-asyncssh-2.13.2-3.el9.noarch 2.2 MB/s | 548 kB 00:00 2023-11-24T08:06:33.219 INFO:teuthology.orchestra.run.smithi156.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2023-11-24T08:06:33.269 INFO:teuthology.orchestra.run.smithi156.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2023-11-24T08:06:33.328 INFO:teuthology.orchestra.run.smithi156.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 1.8 MB/s | 197 kB 00:00 2023-11-24T08:06:33.353 INFO:teuthology.orchestra.run.smithi156.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 129 kB/s | 11 kB 00:00 2023-11-24T08:06:33.378 INFO:teuthology.orchestra.run.smithi156.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2023-11-24T08:06:33.404 INFO:teuthology.orchestra.run.smithi156.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2023-11-24T08:06:33.429 INFO:teuthology.orchestra.run.smithi156.stdout:(110/147): python3-jaraco-functools-3.5.0-2.el9 391 kB/s | 19 kB 00:00 2023-11-24T08:06:33.454 INFO:teuthology.orchestra.run.smithi156.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 397 kB/s | 20 kB 00:00 2023-11-24T08:06:33.480 INFO:teuthology.orchestra.run.smithi156.stdout:(112/147): python3-jwt+crypto-2.4.0-1.el9.noarc 183 kB/s | 9.0 kB 00:00 2023-11-24T08:06:33.505 INFO:teuthology.orchestra.run.smithi156.stdout:(113/147): python3-jwt-2.4.0-1.el9.noarch.rpm 828 kB/s | 41 kB 00:00 2023-11-24T08:06:33.555 INFO:teuthology.orchestra.run.smithi156.stdout:(114/147): python3-logutils-0.3.5-21.el9.noarch 940 kB/s | 46 kB 00:00 2023-11-24T08:06:33.605 INFO:teuthology.orchestra.run.smithi156.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2023-11-24T08:06:33.618 INFO:teuthology.orchestra.run.smithi106.stdout:(92/147): ceph-test-18.0.0-7484.ga13ea5ff.el9.x 5.2 MB/s | 41 MB 00:07 2023-11-24T08:06:33.631 INFO:teuthology.orchestra.run.smithi156.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2023-11-24T08:06:33.648 INFO:teuthology.orchestra.run.smithi156.stdout:(117/147): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2023-11-24T08:06:33.736 INFO:teuthology.orchestra.run.smithi106.stdout:(93/147): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.6 MB/s | 309 kB 00:00 2023-11-24T08:06:33.748 INFO:teuthology.orchestra.run.smithi156.stdout:(118/147): openssl-devel-3.0.7-25.el9.x86_64.rp 2.8 MB/s | 4.1 MB 00:01 2023-11-24T08:06:33.765 INFO:teuthology.orchestra.run.smithi156.stdout:(119/147): python3-portend-3.1.0-2.el9.noarch.r 141 kB/s | 16 kB 00:00 2023-11-24T08:06:33.798 INFO:teuthology.orchestra.run.smithi156.stdout:(120/147): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2023-11-24T08:06:33.816 INFO:teuthology.orchestra.run.smithi156.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 619 kB/s | 31 kB 00:00 2023-11-24T08:06:33.849 INFO:teuthology.orchestra.run.smithi156.stdout:(122/147): python3-routes-2.5.1-5.el9.noarch.rp 3.7 MB/s | 188 kB 00:00 2023-11-24T08:06:33.866 INFO:teuthology.orchestra.run.smithi156.stdout:(123/147): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2023-11-24T08:06:33.891 INFO:teuthology.orchestra.run.smithi156.stdout:(124/147): python3-tempora-5.0.0-2.el9.noarch.r 864 kB/s | 36 kB 00:00 2023-11-24T08:06:33.917 INFO:teuthology.orchestra.run.smithi156.stdout:(125/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 537 kB/s | 90 kB 00:00 2023-11-24T08:06:33.938 INFO:teuthology.orchestra.run.smithi106.stdout:(94/147): libarrow-9.0.0-9.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2023-11-24T08:06:33.942 INFO:teuthology.orchestra.run.smithi156.stdout:(126/147): python3-typing-extensions-4.4.0-2.el 683 kB/s | 51 kB 00:00 2023-11-24T08:06:33.969 INFO:teuthology.orchestra.run.smithi106.stdout:(95/147): libarrow-doc-9.0.0-9.el9.noarch.rpm 763 kB/s | 24 kB 00:00 2023-11-24T08:06:33.976 INFO:teuthology.orchestra.run.smithi156.stdout:(127/147): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2023-11-24T08:06:34.001 INFO:teuthology.orchestra.run.smithi156.stdout:(128/147): python3-websocket-client-1.2.3-2.el9 1.1 MB/s | 90 kB 00:00 2023-11-24T08:06:34.003 INFO:teuthology.orchestra.run.smithi106.stdout:(96/147): liboath-2.6.7-2.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2023-11-24T08:06:34.026 INFO:teuthology.orchestra.run.smithi156.stdout:(129/147): python3-zc-lockfile-2.0-10.el9.noarc 402 kB/s | 20 kB 00:00 2023-11-24T08:06:34.044 INFO:teuthology.orchestra.run.smithi106.stdout:(97/147): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2023-11-24T08:06:34.060 INFO:teuthology.orchestra.run.smithi156.stdout:(130/147): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2023-11-24T08:06:34.103 INFO:teuthology.orchestra.run.smithi106.stdout:(98/147): parquet-libs-9.0.0-9.el9.x86_64.rpm 14 MB/s | 837 kB 00:00 2023-11-24T08:06:34.118 INFO:teuthology.orchestra.run.smithi156.stdout:(131/147): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2023-11-24T08:06:34.128 INFO:teuthology.orchestra.run.smithi106.stdout:(99/147): python3-devel-3.9.17-2.el9.x86_64.rpm 206 kB/s | 245 kB 00:01 2023-11-24T08:06:34.170 INFO:teuthology.orchestra.run.smithi106.stdout:(100/147): python3-asyncssh-2.13.2-3.el9.noarch 8.2 MB/s | 548 kB 00:00 2023-11-24T08:06:34.193 INFO:teuthology.orchestra.run.smithi156.stdout:(132/147): thrift-0.15.0-2.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2023-11-24T08:06:34.244 INFO:teuthology.orchestra.run.smithi156.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.3 MB/s | 68 kB 00:00 2023-11-24T08:06:34.245 INFO:teuthology.orchestra.run.smithi106.stdout:(101/147): python3-cachetools-4.2.4-1.el9.noarc 432 kB/s | 32 kB 00:00 2023-11-24T08:06:34.271 INFO:teuthology.orchestra.run.smithi106.stdout:(102/147): python3-bcrypt-3.2.2-1.el9.x86_64.rp 308 kB/s | 43 kB 00:00 2023-11-24T08:06:34.296 INFO:teuthology.orchestra.run.smithi106.stdout:(103/147): python3-certifi-2021.10.8-2.el9.noar 304 kB/s | 15 kB 00:00 2023-11-24T08:06:34.346 INFO:teuthology.orchestra.run.smithi106.stdout:(104/147): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2023-11-24T08:06:34.389 INFO:teuthology.orchestra.run.smithi106.stdout:(105/147): python3-cherrypy-18.6.1-2.el9.noarch 3.8 MB/s | 358 kB 00:00 2023-11-24T08:06:34.411 INFO:teuthology.orchestra.run.smithi156.stdout:(134/147): NetworkManager-1.45.6-1.el9.x86_64.r 14 MB/s | 2.2 MB 00:00 2023-11-24T08:06:34.414 INFO:teuthology.orchestra.run.smithi106.stdout:(106/147): python3-google-auth-2.23.4-1.el9.noa 2.9 MB/s | 197 kB 00:00 2023-11-24T08:06:34.439 INFO:teuthology.orchestra.run.smithi106.stdout:(107/147): python3-jaraco-8.2.1-3.el9.noarch.rp 217 kB/s | 11 kB 00:00 2023-11-24T08:06:34.464 INFO:teuthology.orchestra.run.smithi106.stdout:(108/147): python3-jaraco-classes-3.2.1-5.el9.n 358 kB/s | 18 kB 00:00 2023-11-24T08:06:34.489 INFO:teuthology.orchestra.run.smithi106.stdout:(109/147): python3-jaraco-collections-3.0.0-8.e 471 kB/s | 23 kB 00:00 2023-11-24T08:06:34.515 INFO:teuthology.orchestra.run.smithi106.stdout:(110/147): python3-jaraco-functools-3.5.0-2.el9 394 kB/s | 19 kB 00:00 2023-11-24T08:06:34.520 INFO:teuthology.orchestra.run.smithi156.stdout:(135/147): NetworkManager-libnm-1.45.6-1.el9.x8 16 MB/s | 1.8 MB 00:00 2023-11-24T08:06:34.540 INFO:teuthology.orchestra.run.smithi106.stdout:(111/147): python3-jaraco-text-3.2.0-6.el9.noar 398 kB/s | 20 kB 00:00 2023-11-24T08:06:34.565 INFO:teuthology.orchestra.run.smithi106.stdout:(112/147): python3-jwt+crypto-2.4.0-1.el9.noarc 183 kB/s | 9.0 kB 00:00 2023-11-24T08:06:34.570 INFO:teuthology.orchestra.run.smithi156.stdout:(136/147): NetworkManager-team-1.45.6-1.el9.x86 857 kB/s | 42 kB 00:00 2023-11-24T08:06:34.590 INFO:teuthology.orchestra.run.smithi106.stdout:(113/147): python3-jwt-2.4.0-1.el9.noarch.rpm 832 kB/s | 41 kB 00:00 2023-11-24T08:06:34.621 INFO:teuthology.orchestra.run.smithi156.stdout:(137/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.8 MB/s | 245 kB 00:00 2023-11-24T08:06:34.624 INFO:teuthology.orchestra.run.smithi106.stdout:(114/147): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2023-11-24T08:06:34.671 INFO:teuthology.orchestra.run.smithi156.stdout:(138/147): acl-2.3.1-4.el9.x86_64.rpm 1.4 MB/s | 69 kB 00:00 2023-11-24T08:06:34.674 INFO:teuthology.orchestra.run.smithi106.stdout:(115/147): python3-kubernetes-26.1.0-2.el9.noar 9.5 MB/s | 1.0 MB 00:00 2023-11-24T08:06:34.700 INFO:teuthology.orchestra.run.smithi106.stdout:(116/147): python3-more-itertools-8.12.0-2.el9. 1.0 MB/s | 79 kB 00:00 2023-11-24T08:06:34.713 INFO:teuthology.orchestra.run.smithi156.stdout:(139/147): elfutils-debuginfod-client-0.190-1.e 896 kB/s | 36 kB 00:00 2023-11-24T08:06:34.716 INFO:teuthology.orchestra.run.smithi106.stdout:(117/147): python3-natsort-7.1.1-5.el9.noarch.r 1.4 MB/s | 58 kB 00:00 2023-11-24T08:06:34.750 INFO:teuthology.orchestra.run.smithi106.stdout:(118/147): python3-pecan-1.4.2-3.el9.noarch.rpm 5.4 MB/s | 272 kB 00:00 2023-11-24T08:06:34.767 INFO:teuthology.orchestra.run.smithi106.stdout:(119/147): python3-portend-3.1.0-2.el9.noarch.r 333 kB/s | 16 kB 00:00 2023-11-24T08:06:34.793 INFO:teuthology.orchestra.run.smithi106.stdout:(120/147): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2023-11-24T08:06:34.797 INFO:teuthology.orchestra.run.smithi156.stdout:(140/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 4.5 MB/s | 3.0 MB 00:00 2023-11-24T08:06:34.818 INFO:teuthology.orchestra.run.smithi106.stdout:(121/147): python3-repoze-lru-0.7-16.el9.noarch 625 kB/s | 31 kB 00:00 2023-11-24T08:06:34.830 INFO:teuthology.orchestra.run.smithi156.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 1.6 MB/s | 192 kB 00:00 2023-11-24T08:06:34.851 INFO:teuthology.orchestra.run.smithi106.stdout:(122/147): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2023-11-24T08:06:34.885 INFO:teuthology.orchestra.run.smithi106.stdout:(123/147): python3-tempora-5.0.0-2.el9.noarch.r 1.1 MB/s | 36 kB 00:00 2023-11-24T08:06:34.897 INFO:teuthology.orchestra.run.smithi156.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 2.6 MB/s | 255 kB 00:00 2023-11-24T08:06:34.910 INFO:teuthology.orchestra.run.smithi106.stdout:(124/147): python3-rsa-4.9-2.el9.noarch.rpm 650 kB/s | 59 kB 00:00 2023-11-24T08:06:34.931 INFO:teuthology.orchestra.run.smithi156.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 226 kB/s | 22 kB 00:00 2023-11-24T08:06:34.964 INFO:teuthology.orchestra.run.smithi156.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 4.8 MB/s | 321 kB 00:00 2023-11-24T08:06:34.966 INFO:teuthology.orchestra.run.smithi106.stdout:(125/147): python3-typing-extensions-4.4.0-2.el 636 kB/s | 51 kB 00:00 2023-11-24T08:06:35.011 INFO:teuthology.orchestra.run.smithi106.stdout:(126/147): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2023-11-24T08:06:35.028 INFO:teuthology.orchestra.run.smithi106.stdout:(127/147): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2023-11-24T08:06:35.070 INFO:teuthology.orchestra.run.smithi106.stdout:(128/147): python3-zc-lockfile-2.0-10.el9.noarc 502 kB/s | 20 kB 00:00 2023-11-24T08:06:35.103 INFO:teuthology.orchestra.run.smithi106.stdout:(129/147): python3-werkzeug-2.0.3-3.el9.1.noarc 4.6 MB/s | 427 kB 00:00 2023-11-24T08:06:35.137 INFO:teuthology.orchestra.run.smithi106.stdout:(130/147): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2023-11-24T08:06:35.304 INFO:teuthology.orchestra.run.smithi106.stdout:(131/147): thrift-0.15.0-2.el9.x86_64.rpm 8.0 MB/s | 1.6 MB 00:00 2023-11-24T08:06:35.314 INFO:teuthology.orchestra.run.smithi156.stdout:(145/147): openssl-3.0.7-25.el9.x86_64.rpm 3.2 MB/s | 1.2 MB 00:00 2023-11-24T08:06:35.381 INFO:teuthology.orchestra.run.smithi156.stdout:(146/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 5.2 MB/s | 2.2 MB 00:00 2023-11-24T08:06:35.615 INFO:teuthology.orchestra.run.smithi156.stdout:(147/147): librados2-18.0.0-7484.ga13ea5ff.el9. 2.1 MB/s | 3.3 MB 00:01 2023-11-24T08:06:35.623 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-11-24T08:06:35.623 INFO:teuthology.orchestra.run.smithi156.stdout:Total 14 MB/s | 204 MB 00:14 2023-11-24T08:06:35.687 INFO:teuthology.orchestra.run.smithi106.stdout:(132/147): librados2-18.0.0-7484.ga13ea5ff.el9. 6.0 MB/s | 3.3 MB 00:00 2023-11-24T08:06:35.846 INFO:teuthology.orchestra.run.smithi106.stdout:(133/147): libacl-devel-2.3.1-4.el9.x86_64.rpm 432 kB/s | 68 kB 00:00 2023-11-24T08:06:35.847 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-11-24T08:06:35.938 INFO:teuthology.orchestra.run.smithi106.stdout:(134/147): librbd1-18.0.0-7484.ga13ea5ff.el9.x8 4.8 MB/s | 3.0 MB 00:00 2023-11-24T08:06:36.022 INFO:teuthology.orchestra.run.smithi106.stdout:(135/147): NetworkManager-1.45.6-1.el9.x86_64.r 13 MB/s | 2.2 MB 00:00 2023-11-24T08:06:36.089 INFO:teuthology.orchestra.run.smithi106.stdout:(136/147): NetworkManager-libnm-1.45.6-1.el9.x8 13 MB/s | 1.8 MB 00:00 2023-11-24T08:06:36.114 INFO:teuthology.orchestra.run.smithi106.stdout:(137/147): NetworkManager-team-1.45.6-1.el9.x86 548 kB/s | 42 kB 00:00 2023-11-24T08:06:36.127 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-11-24T08:06:36.127 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-11-24T08:06:36.148 INFO:teuthology.orchestra.run.smithi106.stdout:(138/147): NetworkManager-tui-1.45.6-1.el9.x86_ 4.3 MB/s | 245 kB 00:00 2023-11-24T08:06:36.173 INFO:teuthology.orchestra.run.smithi106.stdout:(139/147): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 69 kB 00:00 2023-11-24T08:06:36.198 INFO:teuthology.orchestra.run.smithi106.stdout:(140/147): elfutils-debuginfod-client-0.190-1.e 761 kB/s | 36 kB 00:00 2023-11-24T08:06:36.255 INFO:teuthology.orchestra.run.smithi106.stdout:(141/147): elfutils-libelf-0.190-1.el9.x86_64.r 2.4 MB/s | 192 kB 00:00 2023-11-24T08:06:36.290 INFO:teuthology.orchestra.run.smithi106.stdout:(142/147): elfutils-libs-0.190-1.el9.x86_64.rpm 2.8 MB/s | 255 kB 00:00 2023-11-24T08:06:36.316 INFO:teuthology.orchestra.run.smithi106.stdout:(143/147): libacl-2.3.1-4.el9.x86_64.rpm 387 kB/s | 22 kB 00:00 2023-11-24T08:06:36.350 INFO:teuthology.orchestra.run.smithi106.stdout:(144/147): xfsdump-3.1.12-4.el9.x86_64.rpm 5.5 MB/s | 321 kB 00:00 2023-11-24T08:06:36.633 INFO:teuthology.orchestra.run.smithi106.stdout:(145/147): openssl-devel-3.0.7-25.el9.x86_64.rp 1.1 MB/s | 4.1 MB 00:03 2023-11-24T08:06:37.000 INFO:teuthology.orchestra.run.smithi106.stdout:(146/147): openssl-3.0.7-25.el9.x86_64.rpm 1.8 MB/s | 1.2 MB 00:00 2023-11-24T08:06:37.491 INFO:teuthology.orchestra.run.smithi106.stdout:(147/147): openssl-libs-3.0.7-25.el9.x86_64.rpm 1.9 MB/s | 2.2 MB 00:01 2023-11-24T08:06:37.506 INFO:teuthology.orchestra.run.smithi106.stdout:-------------------------------------------------------------------------------- 2023-11-24T08:06:37.506 INFO:teuthology.orchestra.run.smithi106.stdout:Total 12 MB/s | 204 MB 00:16 2023-11-24T08:06:37.725 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction check 2023-11-24T08:06:38.001 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction check succeeded. 2023-11-24T08:06:38.001 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction test 2023-11-24T08:06:38.296 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-11-24T08:06:38.300 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-11-24T08:06:40.298 INFO:teuthology.orchestra.run.smithi106.stdout:Transaction test succeeded. 2023-11-24T08:06:40.298 INFO:teuthology.orchestra.run.smithi106.stdout:Running transaction 2023-11-24T08:06:44.804 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-11-24T08:06:44.934 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T08:06:44.972 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T08:06:45.044 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T08:06:45.132 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T08:06:45.165 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T08:06:45.263 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T08:06:45.684 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T08:06:45.738 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:06:45.894 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:06:45.930 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:06:46.051 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:06:46.092 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:06:46.168 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T08:06:46.471 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T08:06:46.536 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T08:06:46.561 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:06:46.589 INFO:teuthology.orchestra.run.smithi106.stdout: Preparing : 1/1 2023-11-24T08:06:46.734 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/162 2023-11-24T08:06:46.761 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:06:46.775 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/162 2023-11-24T08:06:46.787 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:06:46.847 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/162 2023-11-24T08:06:46.878 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:06:46.902 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/162 2023-11-24T08:06:46.941 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/162 2023-11-24T08:06:47.041 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/162 2023-11-24T08:06:47.190 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T08:06:47.400 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:06:47.460 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/162 2023-11-24T08:06:47.496 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:06:47.676 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:06:47.692 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:06:47.753 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:06:47.896 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:06:47.940 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:06:48.100 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:06:48.138 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:06:48.178 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:06:48.215 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/162 2023-11-24T08:06:48.234 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T08:06:48.336 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T08:06:48.409 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T08:06:48.447 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T08:06:48.501 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T08:06:48.504 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/162 2023-11-24T08:06:48.565 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T08:06:48.570 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/162 2023-11-24T08:06:48.604 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:06:48.706 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T08:06:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T08:06:48.783 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T08:06:48.804 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 14/162 2023-11-24T08:06:48.838 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:06:48.893 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T08:06:48.938 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T08:06:48.955 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 15/162 2023-11-24T08:06:49.018 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T08:06:49.142 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T08:06:49.277 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 16/162 2023-11-24T08:06:49.320 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T08:06:49.422 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T08:06:49.426 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:06:49.547 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 17/162 2023-11-24T08:06:49.743 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:06:49.812 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:06:50.147 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T08:06:50.259 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.45.6-1.el9.x86_64 18/162 2023-11-24T08:06:50.331 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/162 2023-11-24T08:06:50.449 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/162 2023-11-24T08:06:50.510 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T08:06:50.543 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/162 2023-11-24T08:06:50.578 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T08:06:50.589 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/162 2023-11-24T08:06:50.638 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T08:06:50.647 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/162 2023-11-24T08:06:50.692 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T08:06:50.700 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/162 2023-11-24T08:06:50.765 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T08:06:50.803 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/162 2023-11-24T08:06:50.849 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/162 2023-11-24T08:06:50.881 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/162 2023-11-24T08:06:50.971 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T08:06:50.992 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/162 2023-11-24T08:06:51.019 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T08:06:51.038 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 29/162 2023-11-24T08:06:51.088 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 30/162 2023-11-24T08:06:51.161 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T08:06:51.191 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 31/162 2023-11-24T08:06:51.223 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T08:06:51.305 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T08:06:51.376 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/162 2023-11-24T08:06:51.429 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T08:06:51.447 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/162 2023-11-24T08:06:51.473 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T08:06:51.514 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T08:06:51.585 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T08:06:51.646 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T08:06:51.713 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T08:06:51.921 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T08:06:51.972 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T08:06:52.018 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T08:06:52.147 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T08:06:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/162 2023-11-24T08:06:52.197 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T08:06:52.246 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T08:06:52.259 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/162 2023-11-24T08:06:52.287 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T08:06:52.310 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/162 2023-11-24T08:06:52.370 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/162 2023-11-24T08:06:52.417 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 38/162 2023-11-24T08:06:52.437 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T08:06:52.475 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/162 2023-11-24T08:06:52.492 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T08:06:52.579 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T08:06:52.659 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T08:06:52.667 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/162 2023-11-24T08:06:52.771 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/162 2023-11-24T08:06:52.911 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T08:06:52.933 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/162 2023-11-24T08:06:52.976 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-9.0.0-9.el9.x86_64 43/162 2023-11-24T08:06:52.983 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T08:06:53.048 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/162 2023-11-24T08:06:53.134 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/162 2023-11-24T08:06:53.160 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T08:06:53.174 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/162 2023-11-24T08:06:53.213 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/162 2023-11-24T08:06:53.216 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T08:06:53.289 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/162 2023-11-24T08:06:53.299 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T08:06:53.349 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T08:06:53.357 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/162 2023-11-24T08:06:53.394 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T08:06:53.418 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 50/162 2023-11-24T08:06:53.445 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T08:06:53.497 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T08:06:53.556 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T08:06:53.598 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 51/162 2023-11-24T08:06:53.628 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T08:06:53.647 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 52/162 2023-11-24T08:06:53.692 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 53/162 2023-11-24T08:06:53.702 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T08:06:53.814 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 54/162 2023-11-24T08:06:53.848 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T08:06:53.864 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 55/162 2023-11-24T08:06:53.897 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T08:06:53.914 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 56/162 2023-11-24T08:06:53.932 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T08:06:53.954 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 57/162 2023-11-24T08:06:53.997 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T08:06:54.035 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T08:06:54.097 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libarrow-doc-9.0.0-9.el9.noarch 58/162 2023-11-24T08:06:54.151 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 59/162 2023-11-24T08:06:54.215 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T08:06:54.233 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-ply-3.11-14.el9.noarch 60/162 2023-11-24T08:06:54.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T08:06:54.309 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 61/162 2023-11-24T08:06:54.402 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T08:06:54.466 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T08:06:54.566 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 62/162 2023-11-24T08:06:54.622 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 63/162 2023-11-24T08:06:54.794 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 64/162 2023-11-24T08:06:54.849 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 65/162 2023-11-24T08:06:54.874 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 66/162 2023-11-24T08:06:54.912 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 67/162 2023-11-24T08:06:54.956 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 68/162 2023-11-24T08:06:54.993 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : fuse-2.9.9-15.el9.x86_64 69/162 2023-11-24T08:06:55.044 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 70/162 2023-11-24T08:06:55.110 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/162 2023-11-24T08:06:55.178 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/162 2023-11-24T08:06:55.230 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/162 2023-11-24T08:06:55.358 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/162 2023-11-24T08:06:55.406 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 75/162 2023-11-24T08:06:55.475 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/162 2023-11-24T08:06:55.477 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T08:06:55.530 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T08:06:55.543 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/162 2023-11-24T08:06:55.572 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T08:06:55.588 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/162 2023-11-24T08:06:55.614 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T08:06:55.733 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/162 2023-11-24T08:06:55.791 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 80/162 2023-11-24T08:06:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T08:06:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T08:06:55.946 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/162 2023-11-24T08:06:56.013 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T08:06:56.015 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/162 2023-11-24T08:06:56.940 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T08:06:56.990 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/162 2023-11-24T08:06:57.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/162 2023-11-24T08:06:57.080 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/162 2023-11-24T08:06:57.082 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T08:06:57.123 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/162 2023-11-24T08:06:57.366 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/162 2023-11-24T08:06:57.402 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/162 2023-11-24T08:06:57.506 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/162 2023-11-24T08:06:58.293 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/162 2023-11-24T08:06:58.420 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/162 2023-11-24T08:06:58.703 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T08:06:58.866 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T08:06:58.903 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T08:06:58.934 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T08:06:58.980 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T08:06:59.039 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T08:06:59.421 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T08:06:59.456 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:06:59.568 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:06:59.604 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T08:06:59.657 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T08:06:59.951 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/162 2023-11-24T08:07:00.046 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/162 2023-11-24T08:07:00.085 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 94/162 2023-11-24T08:07:00.116 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : jq-1.6-16.el9.x86_64 95/162 2023-11-24T08:07:00.131 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T08:07:00.166 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/162 2023-11-24T08:07:00.199 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:07:00.218 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 97/162 2023-11-24T08:07:00.302 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:07:00.337 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T08:07:00.595 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/162 2023-11-24T08:07:00.630 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:07:00.749 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 99/162 2023-11-24T08:07:00.777 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 100/162 2023-11-24T08:07:00.831 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/162 2023-11-24T08:07:01.303 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : parquet-libs-9.0.0-9.el9.x86_64 102/162 2023-11-24T08:07:01.341 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:07:01.432 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 103/162 2023-11-24T08:07:01.460 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 104/162 2023-11-24T08:07:02.814 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:07:02.858 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:07:03.483 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:07:03.528 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:07:03.694 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:07:03.694 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T08:07:03.695 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T08:07:03.695 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:03.771 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:07:03.963 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:07:03.998 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:07:04.619 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 105/162 2023-11-24T08:07:04.660 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:07:04.854 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 106/162 2023-11-24T08:07:04.854 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-11-24T08:07:04.854 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-11-24T08:07:04.855 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:04.894 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-11-24T08:07:16.334 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /root 2023-11-24T08:07:16.335 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-11-24T08:07:16.335 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:16.553 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:07:16.603 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:07:16.603 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:16.603 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T08:07:16.603 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:17.460 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 107/162 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /sys 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /proc 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /mnt 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /var/tmp 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /home 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /root 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout:skipping the directory /tmp 2023-11-24T08:07:17.461 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:17.679 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:07:17.727 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 108/162 2023-11-24T08:07:17.727 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:17.727 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-11-24T08:07:17.727 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:18.289 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:07:18.367 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:07:18.367 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:18.367 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T08:07:18.368 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:07:18.368 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:07:18.368 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:18.614 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:07:18.694 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:07:18.694 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:18.695 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T08:07:18.695 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:07:18.695 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:07:18.695 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:19.181 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:07:19.253 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:07:19.253 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:19.253 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T08:07:19.253 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:07:19.254 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:07:19.254 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:19.367 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T08:07:19.476 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:07:19.558 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 109/162 2023-11-24T08:07:19.558 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:19.559 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-11-24T08:07:19.559 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:07:19.559 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-11-24T08:07:19.559 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:19.801 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:07:19.882 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 110/162 2023-11-24T08:07:19.882 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:19.882 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-11-24T08:07:19.883 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:07:19.883 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-11-24T08:07:19.883 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:20.039 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:07:20.067 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:07:20.105 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T08:07:20.197 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T08:07:20.230 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:07:20.359 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:07:20.441 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 111/162 2023-11-24T08:07:20.441 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-11-24T08:07:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:07:20.442 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-11-24T08:07:20.442 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:20.547 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:07:20.553 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : automake-1.16.2-8.el9.noarch 112/162 2023-11-24T08:07:20.590 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:07:21.221 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:07:21.255 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 113/162 2023-11-24T08:07:21.299 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 114/162 2023-11-24T08:07:21.393 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 115/162 2023-11-24T08:07:21.434 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:07:21.720 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:07:21.752 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 116/162 2023-11-24T08:07:21.764 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:07:21.802 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:07:21.911 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:07:22.072 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T08:07:22.100 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:07:22.175 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:07:22.175 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:22.175 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T08:07:22.175 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:07:22.175 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:07:22.175 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:22.225 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:07:22.264 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:07:22.353 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T08:07:22.929 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 117/162 2023-11-24T08:07:22.960 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:07:23.101 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 118/162 2023-11-24T08:07:23.220 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T08:07:23.253 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:07:23.268 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 119/162 2023-11-24T08:07:23.321 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:07:23.326 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:07:23.326 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:23.326 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T08:07:23.326 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:07:23.326 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:07:23.326 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:23.380 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:07:23.412 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 120/162 2023-11-24T08:07:23.412 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:23.413 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-11-24T08:07:23.413 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:07:23.413 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-11-24T08:07:23.413 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:23.469 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:07:23.469 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:23.469 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T08:07:23.469 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:23.471 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:07:23.511 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 121/162 2023-11-24T08:07:23.539 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:07:23.599 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 122/162 2023-11-24T08:07:23.646 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:07:23.646 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:23.646 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T08:07:23.646 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:07:23.647 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:07:23.647 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:24.015 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:07:24.097 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:07:24.097 INFO:teuthology.orchestra.run.smithi156.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:24.097 INFO:teuthology.orchestra.run.smithi156.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T08:07:24.097 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:07:24.097 INFO:teuthology.orchestra.run.smithi156.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:07:24.097 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:24.466 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libtool-2.4.6-45.el9.x86_64 123/162 2023-11-24T08:07:24.498 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:07:24.577 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 124/162 2023-11-24T08:07:24.578 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:24.578 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-11-24T08:07:24.578 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:07:24.578 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-11-24T08:07:24.578 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:24.632 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:07:24.706 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 125/162 2023-11-24T08:07:24.706 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:24.706 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-11-24T08:07:24.706 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:24.775 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:07:24.852 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 126/162 2023-11-24T08:07:24.852 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:24.852 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-11-24T08:07:24.852 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:07:24.852 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-11-24T08:07:24.853 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:25.209 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:07:25.278 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 127/162 2023-11-24T08:07:25.279 INFO:teuthology.orchestra.run.smithi106.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-11-24T08:07:25.279 INFO:teuthology.orchestra.run.smithi106.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-11-24T08:07:25.279 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:07:25.279 INFO:teuthology.orchestra.run.smithi106.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-11-24T08:07:25.279 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:28.262 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T08:07:28.306 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T08:07:28.362 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T08:07:28.426 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T08:07:28.713 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T08:07:28.837 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T08:07:28.874 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T08:07:28.908 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T08:07:28.939 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T08:07:28.976 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T08:07:29.092 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T08:07:29.161 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T08:07:29.226 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T08:07:29.389 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 128/162 2023-11-24T08:07:29.434 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 129/162 2023-11-24T08:07:29.488 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 130/162 2023-11-24T08:07:29.521 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 131/162 2023-11-24T08:07:29.787 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 132/162 2023-11-24T08:07:29.914 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 133/162 2023-11-24T08:07:29.950 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 134/162 2023-11-24T08:07:29.983 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 135/162 2023-11-24T08:07:30.023 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 136/162 2023-11-24T08:07:30.068 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-team-1:1.45.6-1.el9.x86_64 137/162 2023-11-24T08:07:30.166 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : NetworkManager-tui-1:1.45.6-1.el9.x86_64 138/162 2023-11-24T08:07:30.231 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:07:30.234 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 139/162 2023-11-24T08:07:30.277 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 140/162 2023-11-24T08:07:30.283 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:07:30.341 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:07:31.228 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:07:31.275 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:07:31.342 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 141/162 2023-11-24T08:07:31.749 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T08:07:31.865 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T08:07:31.950 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T08:07:32.044 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T08:07:32.203 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T08:07:32.251 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T08:07:32.284 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T08:07:32.347 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T08:07:32.394 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T08:07:32.395 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:07:32.506 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:07:32.553 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T08:07:32.598 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T08:07:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T08:07:32.639 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:07:32.668 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 142/162 2023-11-24T08:07:32.791 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 143/162 2023-11-24T08:07:32.793 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:07:32.824 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T08:07:32.853 INFO:teuthology.orchestra.run.smithi106.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 144/162 2023-11-24T08:07:32.864 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:07:32.865 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:07:32.923 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 145/162 2023-11-24T08:07:33.061 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : flex-2.6.4-9.el9.x86_64 146/162 2023-11-24T08:07:33.099 INFO:teuthology.orchestra.run.smithi106.stdout: Installing : bison-3.7.4-5.el9.x86_64 147/162 2023-11-24T08:07:33.132 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 148/162 2023-11-24T08:07:33.166 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/162 2023-11-24T08:07:33.172 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:07:33.173 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:07:33.207 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 150/162 2023-11-24T08:07:33.208 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:07:33.324 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:07:33.349 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/162 2023-11-24T08:07:33.376 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T08:07:33.391 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 152/162 2023-11-24T08:07:33.410 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T08:07:33.438 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 153/162 2023-11-24T08:07:33.445 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T08:07:33.470 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 154/162 2023-11-24T08:07:33.471 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:07:33.571 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:07:33.608 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/162 2023-11-24T08:07:33.659 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 156/162 2023-11-24T08:07:33.698 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:07:33.699 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:07:33.838 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:07:33.838 INFO:teuthology.orchestra.run.smithi156.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T08:07:33.839 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:34.029 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 157/162 2023-11-24T08:07:34.029 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:07:34.180 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 158/162 2023-11-24T08:07:34.216 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 159/162 2023-11-24T08:07:34.250 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 160/162 2023-11-24T08:07:34.312 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 161/162 2023-11-24T08:07:34.445 INFO:teuthology.orchestra.run.smithi106.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:07:34.715 INFO:teuthology.orchestra.run.smithi106.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 162/162 2023-11-24T08:07:34.716 INFO:teuthology.orchestra.run.smithi106.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-11-24T08:07:34.716 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:07:38.604 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T08:07:38.605 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T08:07:38.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T08:07:38.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T08:07:38.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T08:07:38.606 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T08:07:38.618 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T08:07:38.619 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T08:07:38.620 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T08:07:38.621 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T08:07:38.622 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T08:07:38.623 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T08:07:38.624 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T08:07:38.625 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T08:07:38.625 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T08:07:38.625 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T08:07:38.626 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T08:07:38.627 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T08:07:38.628 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T08:07:38.629 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T08:07:38.630 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T08:07:39.312 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 1/162 2023-11-24T08:07:39.312 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 3/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 4/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-7484.ga13ea 5/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 6/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 7/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 8/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 9/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 10/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 11/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 12/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 13/162 2023-11-24T08:07:39.313 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_ 14/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 15/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 16/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_6 17/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86 18/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 19/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el 20/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9. 21/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_6 22/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 23/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 24/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 25/162 2023-11-24T08:07:39.314 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 26/162 2023-11-24T08:07:39.315 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 27/162 2023-11-24T08:07:39.315 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 28/162 2023-11-24T08:07:39.315 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff. 29/162 2023-11-24T08:07:39.315 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noa 30/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.n 31/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13 32/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el 33/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 34/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.e 35/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 36/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 37/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 38/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : automake-1.16.2-8.el9.noarch 39/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : bison-3.7.4-5.el9.x86_64 40/162 2023-11-24T08:07:39.326 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 41/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 42/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : jq-1.6-16.el9.x86_64 47/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 48/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 49/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 50/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 51/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 52/162 2023-11-24T08:07:39.327 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 53/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 54/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 55/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 56/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 57/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 58/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-13.11-1.el9.x86_64 59/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 60/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 61/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 62/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 63/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 64/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/162 2023-11-24T08:07:39.328 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 72/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 73/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 74/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 75/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 76/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 77/162 2023-11-24T08:07:39.329 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 78/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 79/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 80/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 81/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 82/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 83/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 84/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 85/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 86/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/162 2023-11-24T08:07:39.330 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 91/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 92/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 93/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 94/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 95/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-9.0.0-9.el9.x86_64 96/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libarrow-doc-9.0.0-9.el9.noarch 97/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 98/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 99/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : parquet-libs-9.0.0-9.el9.x86_64 100/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 101/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 102/162 2023-11-24T08:07:39.331 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 103/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 104/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 105/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 106/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 107/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 108/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 109/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 110/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 111/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 112/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 113/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 114/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 115/162 2023-11-24T08:07:39.332 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 116/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 117/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 118/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 119/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 120/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 121/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 122/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 123/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 124/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 125/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 126/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 127/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 128/162 2023-11-24T08:07:39.333 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 129/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 130/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : re2-1:20211101-3.el9.x86_64 131/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 132/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 133/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 134/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 135/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 136/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 137/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 138/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.45.6-1.el9.x86_64 139/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 140/162 2023-11-24T08:07:39.334 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.45.6-1.el9.x86_64 141/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 142/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.45.6-1.el9.x86_64 143/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 144/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.45.6-1.el9.x86_64 145/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 146/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-4.el9.x86_64 147/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : acl-2.3.1-3.el9.x86_64 148/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 149/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 150/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 151/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 152/162 2023-11-24T08:07:39.335 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 153/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 154/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 155/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 156/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 157/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 158/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 159/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 160/162 2023-11-24T08:07:39.336 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 161/162 2023-11-24T08:07:40.868 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T08:07:40.869 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.870 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.871 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T08:07:40.872 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-16.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.873 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T08:07:40.874 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T08:07:40.875 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T08:07:40.876 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T08:07:40.877 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T08:07:40.878 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T08:07:40.879 INFO:teuthology.orchestra.run.smithi156.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T08:07:40.880 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T08:07:40.881 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T08:07:40.881 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T08:07:40.881 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:40.881 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-11-24T08:07:41.445 INFO:teuthology.orchestra.run.smithi106.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 162/162 2023-11-24T08:07:41.445 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout:Upgraded: 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-libnm-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-team-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: NetworkManager-tui-1:1.45.6-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: acl-2.3.1-4.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-2.3.1-4.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-11-24T08:07:41.446 INFO:teuthology.orchestra.run.smithi106.stdout: librados2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: librbd1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout:Installed: 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: automake-1.16.2-8.el9.noarch 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: bison-3.7.4-5.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-base-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.447 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-grafana-dashboards-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-immutable-object-cache-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mds-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-dashboard-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-diskprediction-local-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-modules-core-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mgr-rook-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-mon-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-osd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-prometheus-alerts-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.448 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-radosgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-selinux-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-test-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: ceph-volume-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: cephadm-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: cephfs-top-2:18.0.0-7484.ga13ea5ff.el9.noarch 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: flex-2.6.4-9.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: fuse-2.9.9-15.el9.x86_64 2023-11-24T08:07:41.449 INFO:teuthology.orchestra.run.smithi106.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: jq-1.6-16.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-9.0.0-9.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libarrow-doc-9.0.0-9.el9.noarch 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libcephfs2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libcephsqlite-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: liboath-2.6.7-2.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-11-24T08:07:41.450 INFO:teuthology.orchestra.run.smithi106.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: librados-devel-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: libradosstriper1-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: librgw2-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: libtool-2.4.6-45.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: libunwind-1.6.2-1.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-0.3.21-2.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-11-24T08:07:41.451 INFO:teuthology.orchestra.run.smithi106.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: parquet-libs-9.0.0-9.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-13.11-1.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: postgresql-server-13.11-1.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-babel-2.9.1-2.el9.noarch 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-argparse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ceph-common-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cephfs-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.452 INFO:teuthology.orchestra.run.smithi106.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-11-24T08:07:41.453 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-mako-1.1.4-6.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-11-24T08:07:41.454 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-ply-3.11-14.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-portend-3.1.0-2.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pycparser-2.20-6.el9.noarch 2023-11-24T08:07:41.455 INFO:teuthology.orchestra.run.smithi106.stdout: python3-pytz-2021.1-5.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rados-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-2.25.1-7.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rgw-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-routes-2.5.1-5.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-rsa-4.9-2.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-toml-0.10.2-6.el9.noarch 2023-11-24T08:07:41.456 INFO:teuthology.orchestra.run.smithi106.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: python3-webob-1.8.7-6.el9.noarch 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-fuse-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-mirror-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: rbd-nbd-2:18.0.0-7484.ga13ea5ff.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: re2-1:20211101-3.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: socat-1.7.4.1-5.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: thrift-0.15.0-2.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: uuid-1.6.2-55.el9.x86_64 2023-11-24T08:07:41.457 INFO:teuthology.orchestra.run.smithi106.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-11-24T08:07:41.458 INFO:teuthology.orchestra.run.smithi106.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-11-24T08:07:41.458 INFO:teuthology.orchestra.run.smithi106.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-11-24T08:07:41.458 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:41.459 INFO:teuthology.orchestra.run.smithi106.stdout:Complete! 2023-11-24T08:07:41.459 DEBUG:teuthology.parallel:result is None 2023-11-24T08:07:42.033 DEBUG:teuthology.parallel:result is None 2023-11-24T08:07:42.034 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:07:42.258 DEBUG:teuthology.orchestra.run.smithi106:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T08:07:42.312 INFO:teuthology.orchestra.run.smithi106.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:07:42.313 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:07:42.313 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T08:07:42.313 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a13ea5ff00cb4df7a9cc941166ddeb358881144c 2023-11-24T08:07:42.526 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-24T08:07:42.583 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:07:42.583 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-7484.ga13ea5ff.el9 2023-11-24T08:07:42.583 INFO:teuthology.task.install:The correct ceph version 18.0.0-7484.ga13ea5ff is installed. 2023-11-24T08:07:42.584 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-24T08:07:42.584 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:42.584 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T08:07:42.663 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:42.664 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-24T08:07:42.740 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-24T08:07:42.740 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:42.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T08:07:42.821 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T08:07:42.940 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:42.940 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-11-24T08:07:43.040 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-24T08:07:43.119 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-24T08:07:43.120 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:43.120 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T08:07:43.197 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T08:07:43.309 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:43.309 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-24T08:07:43.392 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-24T08:07:43.523 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-24T08:07:43.523 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:43.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T08:07:43.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T08:07:43.720 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:43.720 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-11-24T08:07:43.802 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-24T08:07:43.918 INFO:teuthology.run_tasks:Running task ceph... 2023-11-24T08:07:43.986 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-24T08:07:43.986 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 777 /var/log/ceph 2023-11-24T08:07:43.994 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /var/log/ceph 2023-11-24T08:07:44.073 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-24T08:07:44.074 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T08:07:44.079 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-24T08:07:44.152 INFO:tasks.ceph:Creating extra log directories... 2023-11-24T08:07:44.152 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T08:07:44.157 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-24T08:07:44.237 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-24T08:07:44.238 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-24T08:07:44.238 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-24T08:07:44.238 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:44.238 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T08:07:44.278 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:44.509 DEBUG:teuthology.orchestra.run.smithi106:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:44.585 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:44.585 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-24T08:07:44.628 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:44.837 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-24T08:07:44.920 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-24T08:07:44.920 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-24T08:07:44.920 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585', 'branch': 'rishabh-mgr-vol-misc', 'description': 'fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}}', 'email': None, 'first_in_suite': False, 'job_id': '7465585', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'rishabh-mgr-vol-misc'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-vol-tests', 'sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da'}}, 'owner': 'scheduled_rishabh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9764, 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'sleep_before_teardown': 0, 'suite': 'fs:volumes', 'suite_branch': 'mgr-vol-tests', 'suite_path': '/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/rishabh-d-dave/ceph', 'suite_sha1': '73843baf45f8a92d8e3fcbea3f024c97e73036da', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi106.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDeEurGEUMjEgvIefvhfRLqIsBnEA6le92tQs8FB72yfJmmwh1/UqB6PkW7dcVCmTDQStpYE1qXQP8mt1ol3+FI=', 'smithi156.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPAasaiQ3DwUeCiBZ8JtwfQIq+3w7DB62xjr7R10augtjzRlAoThP/oLGKfHN4PsXUMZlqfzp0C6b7q90NfQJU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': True, 'sha1': '8247bbb5b470f7e2a1f3eefcc16e24cf1338f606'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'a13ea5ff00cb4df7a9cc941166ddeb358881144c', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots', 'tasks.cephfs.test_volumes.TestSubvolumeSnapshots']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-24_07:31:48', 'tube': 'smithi', 'user': 'rishabh', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-24T08:07:44.921 DEBUG:teuthology.orchestra.run.smithi106:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T08:07:44.927 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-24T08:07:44.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T08:07:45.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m0777 -- /var/run/ceph 2023-11-24T08:07:45.084 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:45.084 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T08:07:45.123 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T08:07:45.123 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_1 2023-11-24T08:07:45.208 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T08:07:45.208 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:45.208 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 632 Links: 1 2023-11-24T08:07:45.208 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:45.209 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:45.209 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2023-11-24 08:07:35.160916884 +0000 2023-11-24T08:07:45.209 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2023-11-24 08:05:02.591675531 +0000 2023-11-24T08:07:45.209 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2023-11-24 08:05:02.591675531 +0000 2023-11-24T08:07:45.209 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2023-11-24 08:05:02.591675531 +0000 2023-11-24T08:07:45.209 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T08:07:45.319 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2023-11-24T08:07:45.319 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2023-11-24T08:07:45.320 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000433748 s, 1.2 MB/s 2023-11-24T08:07:45.322 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T08:07:45.364 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_2 2023-11-24T08:07:45.442 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 651 Links: 1 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2023-11-24 08:07:35.161916861 +0000 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2023-11-24 08:05:03.637650218 +0000 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2023-11-24 08:05:03.637650218 +0000 2023-11-24T08:07:45.443 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2023-11-24 08:05:03.637650218 +0000 2023-11-24T08:07:45.443 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T08:07:45.552 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2023-11-24T08:07:45.552 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2023-11-24T08:07:45.552 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000295962 s, 1.7 MB/s 2023-11-24T08:07:45.554 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T08:07:45.593 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_3 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2023-11-24 08:07:35.162916837 +0000 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2023-11-24 08:05:04.651625680 +0000 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2023-11-24 08:05:04.651625680 +0000 2023-11-24T08:07:45.674 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2023-11-24 08:05:04.651625680 +0000 2023-11-24T08:07:45.675 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T08:07:45.782 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2023-11-24T08:07:45.782 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2023-11-24T08:07:45.782 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000298375 s, 1.7 MB/s 2023-11-24T08:07:45.785 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T08:07:45.826 DEBUG:teuthology.orchestra.run.smithi106:> stat /dev/vg_nvme/lv_4 2023-11-24T08:07:45.907 INFO:teuthology.orchestra.run.smithi106.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T08:07:45.907 INFO:teuthology.orchestra.run.smithi106.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout:Device: 5h/5d Inode: 690 Links: 1 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout:Access: 2023-11-24 08:07:35.162916837 +0000 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout:Modify: 2023-11-24 08:05:05.678600826 +0000 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout:Change: 2023-11-24 08:05:05.678600826 +0000 2023-11-24T08:07:45.908 INFO:teuthology.orchestra.run.smithi106.stdout: Birth: 2023-11-24 08:05:05.678600826 +0000 2023-11-24T08:07:45.908 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T08:07:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records in 2023-11-24T08:07:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:1+0 records out 2023-11-24T08:07:46.014 INFO:teuthology.orchestra.run.smithi106.stderr:512 bytes copied, 0.000273739 s, 1.9 MB/s 2023-11-24T08:07:46.016 DEBUG:teuthology.orchestra.run.smithi106:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T08:07:46.056 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:07:46.057 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:46.057 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-11-24T08:07:46.105 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-24T08:07:46.105 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-11-24T08:07:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-24T08:07:46.192 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:46.192 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 634 Links: 1 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-24 08:07:34.285068870 +0000 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-24 08:05:02.581889296 +0000 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-24 08:05:02.581889296 +0000 2023-11-24T08:07:46.193 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-24 08:05:02.581889296 +0000 2023-11-24T08:07:46.193 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-24T08:07:46.302 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-24T08:07:46.302 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-24T08:07:46.302 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000297075 s, 1.7 MB/s 2023-11-24T08:07:46.305 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-24T08:07:46.347 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 652 Links: 1 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-24 08:07:34.286068856 +0000 2023-11-24T08:07:46.443 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-24 08:05:03.602873951 +0000 2023-11-24T08:07:46.444 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-24 08:05:03.602873951 +0000 2023-11-24T08:07:46.444 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-24 08:05:03.602873951 +0000 2023-11-24T08:07:46.444 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-24T08:07:46.532 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-24T08:07:46.532 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-24T08:07:46.532 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000269632 s, 1.9 MB/s 2023-11-24T08:07:46.535 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-24T08:07:46.594 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-11-24T08:07:46.643 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 671 Links: 1 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-24 08:07:34.287068841 +0000 2023-11-24T08:07:46.644 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-24 08:05:04.589859117 +0000 2023-11-24T08:07:46.645 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-24 08:05:04.589859117 +0000 2023-11-24T08:07:46.645 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-24 08:05:04.589859117 +0000 2023-11-24T08:07:46.645 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-24T08:07:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-24T08:07:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-24T08:07:46.727 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000243861 s, 2.1 MB/s 2023-11-24T08:07:46.730 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-24T08:07:46.784 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 689 Links: 1 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-11-24 08:07:34.287068841 +0000 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-11-24 08:05:05.570844374 +0000 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-11-24 08:05:05.570844374 +0000 2023-11-24T08:07:46.831 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-11-24 08:05:05.570844374 +0000 2023-11-24T08:07:46.832 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-24T08:07:46.968 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-11-24T08:07:46.968 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-11-24T08:07:46.968 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000229905 s, 2.2 MB/s 2023-11-24T08:07:46.973 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-24T08:07:47.027 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:07:47.028 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T08:07:47.028 INFO:tasks.ceph:Generating config... 2023-11-24T08:07:47.029 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-24T08:07:47.029 INFO:tasks.ceph:[client] debug client = 20 2023-11-24T08:07:47.029 INFO:tasks.ceph:[client] debug ms = 1 2023-11-24T08:07:47.029 INFO:tasks.ceph:[client] pid file = 2023-11-24T08:07:47.029 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-24T08:07:47.030 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-24T08:07:47.030 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-24T08:07:47.030 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-24T08:07:47.031 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-24T08:07:47.032 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-24T08:07:47.032 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-24T08:07:47.033 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-24T08:07:47.033 INFO:tasks.ceph:Setting up mon.a... 2023-11-24T08:07:47.033 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-24T08:07:47.182 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.keyring 2023-11-24T08:07:47.190 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-24T08:07:47.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T08:07:47.409 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.106'), ('mon.b', '172.21.15.156'), ('mon.c', '[v2:172.21.15.156:3301,v1:172.21.15.156:6790]')] 2023-11-24T08:07:47.409 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.106,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-24T08:07:47.410 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:47.410 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T08:07:47.462 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.106 --add b 172.21.15.156 --addv c '[v2:172.21.15.156:3301,v1:172.21.15.156:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:07:47.617 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:07:47.617 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: generated fsid d1a897c2-15ce-4272-ad66-4b3723970cb1 2023-11-24T08:07:47.617 INFO:teuthology.orchestra.run.smithi106.stdout:setting min_mon_release = pacific 2023-11-24T08:07:47.617 INFO:teuthology.orchestra.run.smithi106.stdout:epoch 0 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:fsid d1a897c2-15ce-4272-ad66-4b3723970cb1 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:last_changed 2023-11-24T08:07:47.616609+0000 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:created 2023-11-24T08:07:47.616609+0000 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:min_mon_release 16 (pacific) 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:election_strategy: 1 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:0: [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] mon.a 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:1: [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon.b 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:2: [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] mon.c 2023-11-24T08:07:47.618 INFO:teuthology.orchestra.run.smithi106.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-24T08:07:47.621 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-24T08:07:47.675 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID d1a897c2-15ce-4272-ad66-4b3723970cb1... 2023-11-24T08:07:47.676 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T08:07:47.682 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout:[global] 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout: chdir = "" 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout: auth supported = cephx 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout: filestore xattr use omap = true 2023-11-24T08:07:47.838 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: mon clock drift allowed = 1.000 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: osd crush chooseleaf type = 0 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: auth debug = true 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on old message = true 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on bug = true 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: mon pg warn max object skew = 0 2023-11-24T08:07:47.839 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default size = 2 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow primary affinity = true 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd allow pg remap = true 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on legacy crush tunables = false 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on crush straw calc version zero = false 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on no sortbitwise = false 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on osd down out interval zero = false 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon warn on too few osds = false 2023-11-24T08:07:47.840 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: mon_allow_pool_size_one = true 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: osd default data pool replay window = 5 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: mon allow pool delete = true 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: mon cluster log file level = debug 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: debug asserts on shutdown = true 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: mon health detail to clog = false 2023-11-24T08:07:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: mon host = "172.21.15.106,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: ms die on skipped message = False 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: fsid = d1a897c2-15ce-4272-ad66-4b3723970cb1 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout:[osd] 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: osd journal size = 100 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub load threshold = 5.0 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: osd scrub max interval = 600 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: osd mclock profile = high_recovery_ops 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: osd recover clone overlap = true 2023-11-24T08:07:47.842 INFO:teuthology.orchestra.run.smithi106.stdout: osd recovery max chunk = 1048576 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug shutdown = true 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug op order = true 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug verify stray on activate = true 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug trim objects = true 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd open classes on start = true 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug pg log writeout = true 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd deep scrub update digest min age = 30 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: osd map max advance = 10 2023-11-24T08:07:47.843 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: journal zero on create = true 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: filestore ondisk finisher threads = 3 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: filestore apply finisher threads = 3 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: bdev debug aio = true 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: osd debug misdirected ops = true 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: bdev async discard = True 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: bdev enable discard = True 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore allocator = bitmap 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore block size = 96636764160 2023-11-24T08:07:47.844 INFO:teuthology.orchestra.run.smithi106.stdout: bluestore fsck on mount = True 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluefs = 1/20 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: debug bluestore = 1/20 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: debug osd = 20 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: debug rocksdb = 4/10 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd full ratio = 0.9 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd nearfull ratio = 0.8 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: osd failsafe full ratio = 0.95 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: osd objectstore = bluestore 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.845 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr] 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug mgr = 20 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: mgr/telemetry/nag = false 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug finisher = 20 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout:[mon] 2023-11-24T08:07:47.846 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: debug mon = 20 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: debug paxos = 20 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: debug auth = 20 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: mon data avail warn = 5 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: mon mgr mkfs grace = 240 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd reporter subtree level = osd 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: mon osd prime pg temp = true 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.847 INFO:teuthology.orchestra.run.smithi106.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: # 1m isn't quite enough 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: mon_down_mkfs_grace = 2m 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: mon_warn_on_filestore_osds = false 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: mon op complaint time = 120 2023-11-24T08:07:47.848 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout:[client] 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: rgw cache enabled = true 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable ops log = true 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: rgw enable usage log = true 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: client mount timeout = 600 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: debug client = 20 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: pid file = "" 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2023-11-24T08:07:47.849 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.a] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.b] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mon.c] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout:[mds] 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds = 20 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout: debug mds balancer = 20 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout: debug ms = 1 2023-11-24T08:07:47.850 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug frag = True 2023-11-24T08:07:47.851 INFO:teuthology.orchestra.run.smithi106.stdout: mds debug scatterstat = True 2023-11-24T08:07:47.851 INFO:teuthology.orchestra.run.smithi106.stdout: mds op complaint time = 180 2023-11-24T08:07:47.851 INFO:teuthology.orchestra.run.smithi106.stdout: mds verify scatter = True 2023-11-24T08:07:47.851 INFO:teuthology.orchestra.run.smithi106.stdout: osd op complaint time = 180 2023-11-24T08:07:47.851 INFO:teuthology.orchestra.run.smithi106.stdout: rados mon op timeout = 900 2023-11-24T08:07:47.851 INFO:teuthology.orchestra.run.smithi106.stdout: rados osd op timeout = 900 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout:[global] 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: chdir = "" 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: auth supported = cephx 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: filestore xattr use omap = true 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: mon clock drift allowed = 1.000 2023-11-24T08:07:47.861 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: osd crush chooseleaf type = 0 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: auth debug = true 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on old message = true 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on bug = true 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: mon pg warn max object skew = 0 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: # disable pg_autoscaler by default for new pools 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-24T08:07:47.862 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default size = 2 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow primary affinity = true 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd allow pg remap = true 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on legacy crush tunables = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on crush straw calc version zero = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on no sortbitwise = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on osd down out interval zero = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon warn on too few osds = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-24T08:07:47.863 INFO:teuthology.orchestra.run.smithi156.stdout: mon_allow_pool_size_one = true 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: osd default data pool replay window = 5 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: mon allow pool delete = true 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: mon cluster log file level = debug 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: debug asserts on shutdown = true 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: mon health detail to clog = false 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: mon host = "172.21.15.106,172.21.15.156,[v2:172.21.15.156:3301,v1:172.21.15.156:6790]" 2023-11-24T08:07:47.864 INFO:teuthology.orchestra.run.smithi156.stdout: ms die on skipped message = False 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: fsid = d1a897c2-15ce-4272-ad66-4b3723970cb1 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout:[osd] 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: osd journal size = 100 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub load threshold = 5.0 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: osd scrub max interval = 600 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: osd mclock profile = high_recovery_ops 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: osd recover clone overlap = true 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: osd recovery max chunk = 1048576 2023-11-24T08:07:47.865 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug shutdown = true 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug op order = true 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug verify stray on activate = true 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug trim objects = true 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd open classes on start = true 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug pg log writeout = true 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd deep scrub update digest min age = 30 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: osd map max advance = 10 2023-11-24T08:07:47.866 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: journal zero on create = true 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: filestore ondisk finisher threads = 3 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: filestore apply finisher threads = 3 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: bdev debug aio = true 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: osd debug misdirected ops = true 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: bdev async discard = True 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: bdev enable discard = True 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore allocator = bitmap 2023-11-24T08:07:47.867 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore block size = 96636764160 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: bluestore fsck on mount = True 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluefs = 1/20 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: debug bluestore = 1/20 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: debug osd = 20 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: debug rocksdb = 4/10 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd backfillfull_ratio = 0.85 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd full ratio = 0.9 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd nearfull ratio = 0.8 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: osd failsafe full ratio = 0.95 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: osd objectstore = bluestore 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-11-24T08:07:47.868 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr] 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug mgr = 20 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: mgr/telemetry/nag = false 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug finisher = 20 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout:[mon] 2023-11-24T08:07:47.869 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: debug mon = 20 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: debug paxos = 20 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: debug auth = 20 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: mon data avail warn = 5 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: mon mgr mkfs grace = 240 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min pgs per osd = 4 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd reporter subtree level = osd 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: mon osd prime pg temp = true 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: mon reweight min bytes per osd = 10 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: auth mon ticket ttl = 660 # 11m 2023-11-24T08:07:47.870 INFO:teuthology.orchestra.run.smithi156.stdout: auth service ticket ttl = 240 # 4m 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: # don't complain about insecure global_id in the test suite 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: # 1m isn't quite enough 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: mon_down_mkfs_grace = 2m 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: mon_warn_on_filestore_osds = false 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: mon op complaint time = 120 2023-11-24T08:07:47.871 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout:[client] 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: rgw cache enabled = true 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable ops log = true 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: rgw enable usage log = true 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: client mount timeout = 600 2023-11-24T08:07:47.872 INFO:teuthology.orchestra.run.smithi156.stdout: debug client = 20 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout: pid file = "" 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.a] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.b] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mon.c] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-11-24T08:07:47.873 INFO:teuthology.orchestra.run.smithi156.stdout:[mds] 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds = 20 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: debug mds balancer = 20 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: debug ms = 1 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug frag = True 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: mds debug scatterstat = True 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: mds op complaint time = 180 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: mds verify scatter = True 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: osd op complaint time = 180 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: rados mon op timeout = 900 2023-11-24T08:07:47.874 INFO:teuthology.orchestra.run.smithi156.stdout: rados osd op timeout = 900 2023-11-24T08:07:47.905 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-24T08:07:47.905 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-24T08:07:48.054 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-24T08:07:48.055 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:48.055 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-24T08:07:48.104 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:48.104 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-24T08:07:48.200 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:07:48.201 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:48.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T08:07:48.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T08:07:48.359 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:07:48.359 DEBUG:teuthology.orchestra.run.smithi106:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:07:48.457 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:07:48.457 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:48.457 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-24T08:07:48.457 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-24T08:07:48.592 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:07:48.592 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:07:48.685 INFO:tasks.ceph:Setting up mon nodes... 2023-11-24T08:07:48.685 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-24T08:07:48.685 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T08:07:48.876 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-24T08:07:48.886 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T08:07:49.067 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-24T08:07:49.075 INFO:tasks.ceph:Setting up mds nodes... 2023-11-24T08:07:49.075 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T08:07:49.268 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-24T08:07:49.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-24T08:07:49.357 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T08:07:49.510 INFO:teuthology.orchestra.run.smithi106.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-24T08:07:49.519 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-24T08:07:49.603 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T08:07:49.794 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-24T08:07:49.803 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-24T08:07:49.900 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T08:07:50.082 INFO:teuthology.orchestra.run.smithi156.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-24T08:07:50.092 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-24T08:07:50.188 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-24T08:07:50.188 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-24T08:07:50.307 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-24T08:07:50.351 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-24T08:07:50.463 INFO:teuthology.orchestra.run.smithi106.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-24T08:07:50.505 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-24T08:07:50.642 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-24T08:07:50.694 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-24T08:07:50.823 INFO:teuthology.orchestra.run.smithi156.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-24T08:07:50.874 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-24T08:07:50.874 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi106.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi156.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-24T08:07:50.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-24T08:07:50.969 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:07:50.969 INFO:tasks.ceph:role: osd.0 2023-11-24T08:07:50.969 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:07:50.969 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T08:07:51.063 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:07:51.064 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:07:51.579 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2023-11-24T08:07:51.585 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2023-11-24T08:07:51.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-24T08:07:51.776 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-24T08:07:51.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-24T08:07:51.978 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:07:51.978 INFO:tasks.ceph:role: osd.1 2023-11-24T08:07:51.978 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:07:51.978 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T08:07:52.074 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:07:52.074 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:07:52.074 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:07:52.075 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:07:52.075 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:07:52.075 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:07:52.075 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:07:52.075 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:07:52.075 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:07:52.076 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:07:52.553 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2023-11-24T08:07:52.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2023-11-24T08:07:52.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-24T08:07:52.711 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-24T08:07:52.809 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-24T08:07:52.906 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:07:52.906 INFO:tasks.ceph:role: osd.2 2023-11-24T08:07:52.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:07:52.906 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T08:07:53.003 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:07:53.004 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:07:53.513 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2023-11-24T08:07:53.519 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2023-11-24T08:07:53.519 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-24T08:07:53.708 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-24T08:07:53.817 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-24T08:07:53.912 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-24T08:07:53.912 INFO:tasks.ceph:role: osd.3 2023-11-24T08:07:53.912 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:07:53.912 DEBUG:teuthology.orchestra.run.smithi106:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T08:07:54.011 INFO:teuthology.orchestra.run.smithi106.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:07:54.012 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:07:54.012 INFO:teuthology.orchestra.run.smithi106.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:07:54.012 INFO:teuthology.orchestra.run.smithi106.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:07:54.012 INFO:teuthology.orchestra.run.smithi106.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:07:54.013 INFO:teuthology.orchestra.run.smithi106.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:07:54.013 INFO:teuthology.orchestra.run.smithi106.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:07:54.013 INFO:teuthology.orchestra.run.smithi106.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:07:54.013 INFO:teuthology.orchestra.run.smithi106.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:07:54.013 INFO:teuthology.orchestra.run.smithi106.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:07:54.472 INFO:teuthology.orchestra.run.smithi106.stdout:Discarding blocks...Done. 2023-11-24T08:07:54.478 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi106.front.sepia.ceph.com -o noatime 2023-11-24T08:07:54.478 DEBUG:teuthology.orchestra.run.smithi106:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-24T08:07:54.611 DEBUG:teuthology.orchestra.run.smithi106:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-24T08:07:54.701 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:07:54.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:54.842+0000 7fd33068a500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-24T08:07:54.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:54.844+0000 7fd33068a500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-24T08:07:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:54.844+0000 7fd33068a500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T08:07:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:54.844+0000 7fd33068a500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T08:07:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:54.844+0000 7fd33068a500 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-24T08:07:54.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:54.853+0000 7fd33068a500 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-24T08:07:57.776 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-24T08:07:57.877 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:07:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:58.016+0000 7f4ce91ad500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-24T08:07:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:58.017+0000 7f4ce91ad500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-24T08:07:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:58.017+0000 7f4ce91ad500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T08:07:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:58.017+0000 7f4ce91ad500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T08:07:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:58.017+0000 7f4ce91ad500 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-24T08:07:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:07:58.019+0000 7f4ce91ad500 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-24T08:08:00.950 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-24T08:08:01.047 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:01.188+0000 7f6c94cd2500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-24T08:08:01.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:01.188+0000 7f6c94cd2500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-24T08:08:01.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:01.188+0000 7f6c94cd2500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T08:08:01.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:01.188+0000 7f6c94cd2500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T08:08:01.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:01.188+0000 7f6c94cd2500 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-24T08:08:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:01.190+0000 7f6c94cd2500 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-24T08:08:04.105 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-24T08:08:04.204 DEBUG:teuthology.orchestra.run.smithi106:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:04.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:04.343+0000 7f4eec212500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-24T08:08:04.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:04.344+0000 7f4eec212500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-24T08:08:04.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:04.344+0000 7f4eec212500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T08:08:04.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:04.344+0000 7f4eec212500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T08:08:04.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:04.344+0000 7f4eec212500 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-24T08:08:04.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:04.346+0000 7f4eec212500 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-24T08:08:07.268 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-24T08:08:07.368 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-24T08:08:07.472 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:08:07.472 INFO:tasks.ceph:role: osd.4 2023-11-24T08:08:07.472 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:08:07.472 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-24T08:08:07.567 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:08:07.567 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:08:07.567 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:08:07.567 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:08:07.567 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:08:07.567 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:08:07.568 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:08:07.568 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:08:07.568 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:08:07.568 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:08:07.601 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-24T08:08:07.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-24T08:08:07.607 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-24T08:08:07.717 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-24T08:08:07.809 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-24T08:08:07.890 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:08:07.891 INFO:tasks.ceph:role: osd.5 2023-11-24T08:08:07.891 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:08:07.891 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-24T08:08:07.974 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:08:07.974 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:08:07.974 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:08:07.975 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:08:08.008 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-24T08:08:08.013 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-24T08:08:08.013 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-24T08:08:08.114 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-24T08:08:08.197 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-24T08:08:08.276 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:08:08.276 INFO:tasks.ceph:role: osd.6 2023-11-24T08:08:08.276 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:08:08.276 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:08:08.350 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:08:08.351 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:08:08.351 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:08:08.351 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:08:08.383 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-24T08:08:08.388 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-24T08:08:08.388 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-24T08:08:08.484 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-24T08:08:08.562 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-24T08:08:08.638 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-24T08:08:08.638 INFO:tasks.ceph:role: osd.7 2023-11-24T08:08:08.638 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:08:08.638 DEBUG:teuthology.orchestra.run.smithi156:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-24T08:08:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-24T08:08:08.721 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-24T08:08:08.721 INFO:teuthology.orchestra.run.smithi156.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-24T08:08:08.721 INFO:teuthology.orchestra.run.smithi156.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-11-24T08:08:08.721 INFO:teuthology.orchestra.run.smithi156.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-24T08:08:08.721 INFO:teuthology.orchestra.run.smithi156.stdout: = sunit=0 swidth=0 blks 2023-11-24T08:08:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-24T08:08:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-11-24T08:08:08.722 INFO:teuthology.orchestra.run.smithi156.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-24T08:08:08.722 INFO:teuthology.orchestra.run.smithi156.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-24T08:08:08.755 INFO:teuthology.orchestra.run.smithi156.stdout:Discarding blocks...Done. 2023-11-24T08:08:08.760 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi156.front.sepia.ceph.com -o noatime 2023-11-24T08:08:08.760 DEBUG:teuthology.orchestra.run.smithi156:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-24T08:08:08.859 DEBUG:teuthology.orchestra.run.smithi156:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-24T08:08:08.935 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:09.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:09.078+0000 7fde88614500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-24T08:08:09.080 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:09.079+0000 7fde88614500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-24T08:08:09.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:09.079+0000 7fde88614500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T08:08:09.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:09.079+0000 7fde88614500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T08:08:09.081 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:09.079+0000 7fde88614500 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-24T08:08:09.082 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:09.081+0000 7fde88614500 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-24T08:08:12.250 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-24T08:08:12.351 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:12.490 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:12.489+0000 7ffa03c0f500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-24T08:08:12.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:12.489+0000 7ffa03c0f500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-24T08:08:12.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:12.489+0000 7ffa03c0f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T08:08:12.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:12.490+0000 7ffa03c0f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T08:08:12.491 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:12.490+0000 7ffa03c0f500 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-24T08:08:12.493 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:12.491+0000 7ffa03c0f500 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-24T08:08:14.922 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:14.932 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:15.655 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-24T08:08:15.744 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:15.881 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:15.880+0000 7f9caa000500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-24T08:08:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:15.880+0000 7f9caa000500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-24T08:08:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:15.880+0000 7f9caa000500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T08:08:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:15.881+0000 7f9caa000500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T08:08:15.882 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:15.881+0000 7f9caa000500 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-24T08:08:15.884 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:15.882+0000 7f9caa000500 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-24T08:08:19.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-24T08:08:19.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:19.288 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:19.287+0000 7f7268a0c500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-24T08:08:19.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:19.288+0000 7f7268a0c500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-24T08:08:19.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:19.288+0000 7f7268a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T08:08:19.289 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:19.288+0000 7f7268a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T08:08:19.290 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:19.288+0000 7f7268a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-24T08:08:19.291 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:19.290+0000 7f7268a0c500 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-24T08:08:22.460 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-24T08:08:22.561 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-24T08:08:22.561 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:22.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-24T08:08:22.665 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:22.665 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-24T08:08:22.765 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:22.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-24T08:08:22.854 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:22.855 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-24T08:08:22.935 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:22.935 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-24T08:08:23.068 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:23.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-24T08:08:23.161 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:23.161 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-24T08:08:23.247 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.247 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-24T08:08:23.351 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.351 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-24T08:08:23.446 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.446 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-24T08:08:23.542 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.543 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-24T08:08:23.640 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.641 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-24T08:08:23.734 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.734 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-24T08:08:23.831 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:23.831 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-24T08:08:23.930 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:23.930 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-24T08:08:23.984 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:23.984 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-24T08:08:24.028 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:24.028 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-24T08:08:24.087 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:24.087 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-24T08:08:24.140 INFO:tasks.ceph:Adding keys to all mons... 2023-11-24T08:08:24.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T08:08:24.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-24T08:08:24.226 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.y] 2023-11-24T08:08:24.226 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA== 2023-11-24T08:08:24.226 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.a] 2023-11-24T08:08:24.226 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A== 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.c] 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q== 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.0] 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA== 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.1] 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA== 2023-11-24T08:08:24.227 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.2] 2023-11-24T08:08:24.228 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA== 2023-11-24T08:08:24.228 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.3] 2023-11-24T08:08:24.228 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw== 2023-11-24T08:08:24.228 INFO:teuthology.orchestra.run.smithi106.stdout:[mgr.x] 2023-11-24T08:08:24.228 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ== 2023-11-24T08:08:24.228 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.b] 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg== 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout:[mds.d] 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw== 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.4] 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw== 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.5] 2023-11-24T08:08:24.229 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw== 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.6] 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA== 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout:[osd.7] 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug== 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:08:24.230 INFO:teuthology.orchestra.run.smithi106.stdout:[client.2] 2023-11-24T08:08:24.231 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg== 2023-11-24T08:08:24.231 INFO:teuthology.orchestra.run.smithi106.stdout:[client.1] 2023-11-24T08:08:24.231 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg== 2023-11-24T08:08:24.231 INFO:teuthology.orchestra.run.smithi106.stdout:[client.3] 2023-11-24T08:08:24.231 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w== 2023-11-24T08:08:24.241 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.y] 2023-11-24T08:08:24.241 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA== 2023-11-24T08:08:24.241 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.a] 2023-11-24T08:08:24.241 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A== 2023-11-24T08:08:24.241 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.c] 2023-11-24T08:08:24.242 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q== 2023-11-24T08:08:24.242 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.0] 2023-11-24T08:08:24.242 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA== 2023-11-24T08:08:24.242 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.1] 2023-11-24T08:08:24.242 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA== 2023-11-24T08:08:24.242 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.2] 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA== 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.3] 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw== 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout:[mgr.x] 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ== 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.b] 2023-11-24T08:08:24.243 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg== 2023-11-24T08:08:24.244 INFO:teuthology.orchestra.run.smithi156.stdout:[mds.d] 2023-11-24T08:08:24.244 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw== 2023-11-24T08:08:24.244 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.4] 2023-11-24T08:08:24.244 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw== 2023-11-24T08:08:24.244 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.5] 2023-11-24T08:08:24.244 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw== 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.6] 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA== 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout:[osd.7] 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug== 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout:[client.0] 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:08:24.245 INFO:teuthology.orchestra.run.smithi156.stdout:[client.2] 2023-11-24T08:08:24.246 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg== 2023-11-24T08:08:24.246 INFO:teuthology.orchestra.run.smithi156.stdout:[client.1] 2023-11-24T08:08:24.246 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg== 2023-11-24T08:08:24.246 INFO:teuthology.orchestra.run.smithi156.stdout:[client.3] 2023-11-24T08:08:24.246 INFO:teuthology.orchestra.run.smithi156.stdout: key = AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w== 2023-11-24T08:08:24.247 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:08:24.281 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:08:24.425 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:24.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:24.578 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:24.583 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:24.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:24.733 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:24.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:24.882 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.027 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.176 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.182 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:08:25.329 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-24T08:08:25.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:25.480 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:25.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:25.636 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-24T08:08:25.777 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.783 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.933 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:25.938 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:26.084 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:26.090 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:26.225 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:26.230 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-24T08:08:26.368 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.377 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.522 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.678 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.684 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.829 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.834 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-24T08:08:26.981 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-24T08:08:26.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-24T08:08:27.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T08:08:27.226 DEBUG:teuthology.orchestra.run.smithi106:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-24T08:08:27.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-24T08:08:27.422 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T08:08:27.584 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-24T08:08:27.665 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-24T08:08:27.759 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-24T08:08:27.912 DEBUG:teuthology.orchestra.run.smithi156:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-24T08:08:27.994 DEBUG:teuthology.orchestra.run.smithi106:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:28.003 DEBUG:teuthology.orchestra.run.smithi156:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-24T08:08:28.055 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-24T08:08:28.056 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-24T08:08:28.056 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-24T08:08:28.061 INFO:tasks.ceph.mon.a:Started 2023-11-24T08:08:28.062 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-24T08:08:28.062 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-24T08:08:28.067 INFO:tasks.ceph.mon.b:Started 2023-11-24T08:08:28.067 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-24T08:08:28.067 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-24T08:08:28.072 INFO:tasks.ceph.mon.c:Started 2023-11-24T08:08:28.072 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-24T08:08:28.073 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-24T08:08:28.073 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-24T08:08:28.078 INFO:tasks.ceph.mgr.y:Started 2023-11-24T08:08:28.078 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-24T08:08:28.078 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-24T08:08:28.083 INFO:tasks.ceph.mgr.x:Started 2023-11-24T08:08:28.083 INFO:tasks.ceph:Setting crush tunables to default 2023-11-24T08:08:28.084 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph osd crush tunables default 2023-11-24T08:08:28.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.269+0000 7f4ebc20e640 1 Processor -- start 2023-11-24T08:08:28.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.269+0000 7f4ebc20e640 1 -- start start 2023-11-24T08:08:28.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.270+0000 7f4ebc20e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 0x7f4eb406adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.270+0000 7f4ebc20e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb406dcf0 con 0x7f4eb40b3330 2023-11-24T08:08:28.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.270+0000 7f4ebc20e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eb4161ec0 con 0x7f4eb4161060 2023-11-24T08:08:28.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.270+0000 7f4ebc20e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eb406ec10 con 0x7f4eb40b3790 2023-11-24T08:08:28.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eb9faa640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f4eb4161060 0x7f4eb4161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:51412/0 (socket says 172.21.15.106:51412) 2023-11-24T08:08:28.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eb9faa640 1 -- 172.21.15.106:0/2335555694 learned_addr learned my addr 172.21.15.106:0/2335555694 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:28.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eba7ab640 1 --2- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 0x7f4eb406adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eba7ab640 1 -- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 msgr2=0x7f4eb406adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-24T08:08:28.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eba7ab640 1 -- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 msgr2=0x7f4eb406adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-24T08:08:28.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eba7ab640 1 --2- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 0x7f4eb406adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-24T08:08:28.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.271+0000 7f4eba7ab640 1 --2- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 0x7f4eb406adc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-24T08:08:28.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.275+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569335965 0 0) 0x7f4eb406dcf0 con 0x7f4eb40b3330 2023-11-24T08:08:28.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.275+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea0003520 con 0x7f4eb40b3330 2023-11-24T08:08:28.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805359153 0 0) 0x7f4eb4161ec0 con 0x7f4eb4161060 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eb406dcf0 con 0x7f4eb4161060 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1035864889 0 0) 0x7f4ea0003520 con 0x7f4eb40b3330 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 msgr2=0x7f4eb406adc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 --2- 172.21.15.106:0/2335555694 >> v2:172.21.15.106:3300/0 conn(0x7f4eb40b3790 0x7f4eb406adc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 >> v1:172.21.15.156:6789/0 conn(0x7f4eb4161060 legacy=0x7f4eb4161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb406df90 con 0x7f4eb40b3330 2023-11-24T08:08:28.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f4ea40016e0 con 0x7f4eb40b3330 2023-11-24T08:08:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 >> v1:172.21.15.106:6789/0 conn(0x7f4eb40b3330 legacy=0x7f4eb406a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ea0003b50 0x7f4ea0003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ea00047c0 0x7f4ea0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea0005250 0x7f4ea000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb406ec10 con 0x7f4ea00047c0 2023-11-24T08:08:28.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea000c1a0 con 0x7f4ea0005250 2023-11-24T08:08:28.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea000c360 con 0x7f4ea0003b50 2023-11-24T08:08:28.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eba7ab640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea0005250 0x7f4ea000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ea00047c0 0x7f4ea0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.276+0000 7f4eb9faa640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ea0003b50 0x7f4ea0003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.277+0000 7f4eb97a9640 1 -- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ea0003b50 msgr2=0x7f4ea0003f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.277+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4ea0003b50 0x7f4ea0003f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.277+0000 7f4eb97a9640 1 -- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea0005250 msgr2=0x7f4ea000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.277+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea0005250 0x7f4ea000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.277+0000 7f4eb97a9640 1 -- 172.21.15.106:0/2335555694 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea0003520 con 0x7f4ea00047c0 2023-11-24T08:08:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.277+0000 7f4eba7ab640 1 -- 172.21.15.106:0/2335555694 reap_dead start 2023-11-24T08:08:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.278+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ea00047c0 0x7f4ea0004c40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4007e80 tx=0x7f4ea00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b27fea670e5238 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.278+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4005270 con 0x7f4ea00047c0 2023-11-24T08:08:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.278+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea40053d0 con 0x7f4ea00047c0 2023-11-24T08:08:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.278+0000 7f4eb8fa8640 1 -- 172.21.15.106:0/2335555694 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea402c8c0 con 0x7f4ea00047c0 2023-11-24T08:08:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 -- 172.21.15.106:0/2335555694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ea00047c0 msgr2=0x7f4ea0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ea00047c0 0x7f4ea0004c40 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4007e80 tx=0x7f4ea00057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 -- 172.21.15.106:0/2335555694 shutdown_connections 2023-11-24T08:08:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 --2- 172.21.15.106:0/2335555694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4ea00047c0 0x7f4ea0004c40 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 -- 172.21.15.106:0/2335555694 >> 172.21.15.106:0/2335555694 conn(0x7f4eb40cf900 msgr2=0x7f4eb40cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 -- 172.21.15.106:0/2335555694 shutdown_connections 2023-11-24T08:08:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.279+0000 7f4ebc20e640 1 -- 172.21.15.106:0/2335555694 wait complete. 2023-11-24T08:08:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.281+0000 7f4ebc20e640 1 Processor -- start 2023-11-24T08:08:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.281+0000 7f4ebc20e640 1 -- start start 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4ebc20e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea00047c0 0x7f4eb41f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4ebc20e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 0x7f4eb41515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4ebc20e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4eb414c460 0x7f4eb406ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4ebc20e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41f73c0 con 0x7f4eb414fbc0 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4ebc20e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb4151b00 con 0x7f4ea00047c0 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4ebc20e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eb41f6b20 con 0x7f4eb414c460 2023-11-24T08:08:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4eb97a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 0x7f4eb41515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4eb9faa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea00047c0 0x7f4eb41f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4eb97a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 0x7f4eb41515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42962/0 (socket says 172.21.15.106:42962) 2023-11-24T08:08:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4eb97a9640 1 -- 172.21.15.106:0/864358884 learned_addr learned my addr 172.21.15.106:0/864358884 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.282+0000 7f4eba7ab640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4eb414c460 0x7f4eb406ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eb97a9640 1 -- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4eb414c460 msgr2=0x7f4eb406ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4eb414c460 0x7f4eb406ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eb97a9640 1 -- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea00047c0 msgr2=0x7f4eb41f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea00047c0 0x7f4eb41f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eb97a9640 1 -- 172.21.15.106:0/864358884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eb41f6e00 con 0x7f4eb414fbc0 2023-11-24T08:08:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eb97a9640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 0x7f4eb41515c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4007e80 tx=0x7f4ea40019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea403c070 con 0x7f4eb414fbc0 2023-11-24T08:08:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ea4000ed0 con 0x7f4eb414fbc0 2023-11-24T08:08:28.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ea40035f0 con 0x7f4eb414fbc0 2023-11-24T08:08:28.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eb41f70e0 con 0x7f4eb414fbc0 2023-11-24T08:08:28.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.283+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eb4150040 con 0x7f4eb414fbc0 2023-11-24T08:08:28.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.284+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f4ea4032070 con 0x7f4eb414fbc0 2023-11-24T08:08:28.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.284+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f4ea4035630 con 0x7f4eb414fbc0 2023-11-24T08:08:28.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.285+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eb4067990 con 0x7f4eb414fbc0 2023-11-24T08:08:28.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.287+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f4ea4045070 con 0x7f4eb414fbc0 2023-11-24T08:08:28.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.358+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f4eb414d380 con 0x7f4eb414fbc0 2023-11-24T08:08:28.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.361+0000 7f4eaa7fc640 1 -- 172.21.15.106:0/864358884 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f4eb414d380 con 0x7f4eb414fbc0 2023-11-24T08:08:28.362 INFO:teuthology.orchestra.run.smithi106.stderr:adjusted tunables profile to default 2023-11-24T08:08:28.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 msgr2=0x7f4eb41515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:28.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 0x7f4eb41515c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4007e80 tx=0x7f4ea40019a0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 shutdown_connections 2023-11-24T08:08:28.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4eb414c460 0x7f4eb406ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4eb414fbc0 0x7f4eb41515c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 --2- 172.21.15.106:0/864358884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ea00047c0 0x7f4eb41f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:28.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 >> 172.21.15.106:0/864358884 conn(0x7f4eb40cf900 msgr2=0x7f4eb415d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:28.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 shutdown_connections 2023-11-24T08:08:28.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:28.362+0000 7f4ebc20e640 1 -- 172.21.15.106:0/864358884 wait complete. 2023-11-24T08:08:28.381 INFO:tasks.ceph:check_enable_crimson: False 2023-11-24T08:08:28.381 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-24T08:08:28.382 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:28.382 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-24T08:08:28.462 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:28.462 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-24T08:08:28.588 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:28.588 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-24T08:08:28.720 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:08:28.720 DEBUG:teuthology.orchestra.run.smithi106:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-24T08:08:28.798 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:28.798 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-24T08:08:28.889 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:28.889 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-24T08:08:28.976 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:28.976 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-24T08:08:28.977 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:28.974+0000 7f682b62b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T08:08:29.042 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.040+0000 7f682b62b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T08:08:29.050 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:29.048+0000 7fb8cfc56040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-24T08:08:29.111 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:08:29.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-24T08:08:29.117 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:29.116+0000 7fb8cfc56040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-24T08:08:29.206 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 22cb81b7-10cd-4ba1-9b9c-6f5a45440014 0 2023-11-24T08:08:29.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.397+0000 7f504b59e640 1 Processor -- start 2023-11-24T08:08:29.399 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.398+0000 7f504b59e640 1 -- start start 2023-11-24T08:08:29.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f504c0d1550 0x7f504c0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504b59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f504c0d3500 0x7f504c0ce660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.400 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504b59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504c0d1ae0 con 0x7f504c160a00 2023-11-24T08:08:29.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504b59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0d1dc0 con 0x7f504c0d1550 2023-11-24T08:08:29.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504b59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504c0d20a0 con 0x7f504c0d3500 2023-11-24T08:08:29.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504ad9d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f504c0d3500 0x7f504c0ce660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504a59c640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f504c160a00 0x7f504c0d0e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:56286/0 (socket says 172.21.15.156:56286) 2023-11-24T08:08:29.401 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504a59c640 1 -- 172.21.15.156:0/3811006010 learned_addr learned my addr 172.21.15.156:0/3811006010 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-24T08:08:29.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f5049d9b640 1 -- 172.21.15.156:0/3811006010 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926267875 0 0) 0x7f504c0d1ae0 con 0x7f504c160a00 2023-11-24T08:08:29.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f5049d9b640 1 -- 172.21.15.156:0/3811006010 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5038003520 con 0x7f504c160a00 2023-11-24T08:08:29.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f5043fff640 1 --2- 172.21.15.156:0/3811006010 >> v2:172.21.15.106:3300/0 conn(0x7f504c0d1550 0x7f504c0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504ad9d640 1 -- 172.21.15.156:0/3811006010 >> v2:172.21.15.106:3300/0 conn(0x7f504c0d1550 msgr2=0x7f504c0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:29.402 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504ad9d640 1 --2- 172.21.15.156:0/3811006010 >> v2:172.21.15.106:3300/0 conn(0x7f504c0d1550 0x7f504c0d2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.399+0000 7f504ad9d640 1 -- 172.21.15.156:0/3811006010 >> v1:172.21.15.156:6789/0 conn(0x7f504c160a00 legacy=0x7f504c0d0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.400+0000 7f504ad9d640 1 -- 172.21.15.156:0/3811006010 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c0ce140 con 0x7f504c0d3500 2023-11-24T08:08:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.401+0000 7f504ad9d640 1 --2- 172.21.15.156:0/3811006010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c0d3500 0x7f504c0ce660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f504c08c6e0 tx=0x7f504402bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=681685009c9366a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.401+0000 7f5049d9b640 1 -- 172.21.15.156:0/3811006010 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504402cb50 con 0x7f504c0d3500 2023-11-24T08:08:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.401+0000 7f5049d9b640 1 -- 172.21.15.156:0/3811006010 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f504402ccb0 con 0x7f504c0d3500 2023-11-24T08:08:29.403 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.401+0000 7f5049d9b640 1 -- 172.21.15.156:0/3811006010 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044034660 con 0x7f504c0d3500 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.401+0000 7f504b59e640 1 -- 172.21.15.156:0/3811006010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c0d3500 msgr2=0x7f504c0ce660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.401+0000 7f504b59e640 1 --2- 172.21.15.156:0/3811006010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c0d3500 0x7f504c0ce660 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f504c08c6e0 tx=0x7f504402bda0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.402+0000 7f504b59e640 1 -- 172.21.15.156:0/3811006010 shutdown_connections 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.402+0000 7f504b59e640 1 --2- 172.21.15.156:0/3811006010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c0d3500 0x7f504c0ce660 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.402+0000 7f504b59e640 1 --2- 172.21.15.156:0/3811006010 >> v2:172.21.15.106:3300/0 conn(0x7f504c0d1550 0x7f504c0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.402+0000 7f504b59e640 1 -- 172.21.15.156:0/3811006010 >> 172.21.15.156:0/3811006010 conn(0x7f504c08cc90 msgr2=0x7f504c09c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.404 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.402+0000 7f504b59e640 1 -- 172.21.15.156:0/3811006010 shutdown_connections 2023-11-24T08:08:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.402+0000 7f504b59e640 1 -- 172.21.15.156:0/3811006010 wait complete. 2023-11-24T08:08:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.403+0000 7f504b59e640 1 Processor -- start 2023-11-24T08:08:29.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.404+0000 7f504b59e640 1 -- start start 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504c0d1550 0x7f504c0ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504c0d3500 0x7f504c15da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 0x7f504c0b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504b59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0cd290 con 0x7f504c0d1550 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504b59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f504c1f63d0 con 0x7f504c160a00 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c0cb820 con 0x7f504c0d3500 2023-11-24T08:08:29.407 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f5043fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504c0d3500 0x7f504c15da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 0x7f504c0b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 0x7f504c0b69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57040/0 (socket says 172.21.15.156:57040) 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 -- 172.21.15.156:0/109171056 learned_addr learned my addr 172.21.15.156:0/109171056 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504a59c640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504c0d1550 0x7f504c0ccd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 -- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504c0d3500 msgr2=0x7f504c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504c0d3500 0x7f504c15da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 -- 172.21.15.156:0/109171056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504c0d1550 msgr2=0x7f504c0ccd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504c0d1550 0x7f504c0ccd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f504ad9d640 1 -- 172.21.15.156:0/109171056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5044009a70 con 0x7f504c160a00 2023-11-24T08:08:29.408 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.405+0000 7f5043fff640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504c0d3500 0x7f504c15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:29.409 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f504ad9d640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 0x7f504c0b69d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5044009e80 tx=0x7f5044004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044040650 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50440407b0 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f504c0cbb00 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044040b60 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f504c0cc0c0 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.406+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5044049400 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.407+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f504403e400 con 0x7f504c160a00 2023-11-24T08:08:29.410 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.407+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504c0ceb20 con 0x7f504c160a00 2023-11-24T08:08:29.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.410+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5044067ce0 con 0x7f504c160a00 2023-11-24T08:08:29.489 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.488+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "22cb81b7-10cd-4ba1-9b9c-6f5a45440014", "id": 0} v 0) v1 -- 0x7f504c04e840 con 0x7f504c160a00 2023-11-24T08:08:29.495 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.494+0000 7f5042ffd640 1 -- 172.21.15.156:0/109171056 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "22cb81b7-10cd-4ba1-9b9c-6f5a45440014", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5044056030 con 0x7f504c160a00 2023-11-24T08:08:29.496 INFO:teuthology.orchestra.run.smithi156.stdout:0 2023-11-24T08:08:29.497 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 msgr2=0x7f504c0b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 0x7f504c0b69d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5044009e80 tx=0x7f5044004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 shutdown_connections 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504c160a00 0x7f504c0b69d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504c0d3500 0x7f504c15da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 --2- 172.21.15.156:0/109171056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504c0d1550 0x7f504c0ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 >> 172.21.15.156:0/109171056 conn(0x7f504c08cc90 msgr2=0x7f504c15e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 shutdown_connections 2023-11-24T08:08:29.498 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.496+0000 7f504b59e640 1 -- 172.21.15.156:0/109171056 wait complete. 2023-11-24T08:08:29.513 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd 1 2023-11-24T08:08:29.646 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.644+0000 7f682b62b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T08:08:29.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.702+0000 7f49c10ea640 1 Processor -- start 2023-11-24T08:08:29.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.702+0000 7f49c10ea640 1 -- start start 2023-11-24T08:08:29.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49c10ea640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49bc160a00 0x7f49bc0d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.706 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49c10ea640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49bc0d1380 0x7f49bc0d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49c10ea640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc0d1810 con 0x7f49bc0d3500 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49c10ea640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0d1ae0 con 0x7f49bc0d1380 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49c10ea640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0d1dc0 con 0x7f49bc160a00 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49bbfff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49bc160a00 0x7f49bc0d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49bbfff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49bc160a00 0x7f49bc0d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57046/0 (socket says 172.21.15.156:57046) 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.704+0000 7f49bbfff640 1 -- 172.21.15.156:0/2251731218 learned_addr learned my addr 172.21.15.156:0/2251731218 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.705+0000 7f49bb7fe640 1 --2- 172.21.15.156:0/2251731218 >> v2:172.21.15.106:3300/0 conn(0x7f49bc0d1380 0x7f49bc0d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.705+0000 7f49bbfff640 1 -- 172.21.15.156:0/2251731218 >> v2:172.21.15.106:3300/0 conn(0x7f49bc0d1380 msgr2=0x7f49bc0d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.705+0000 7f49bbfff640 1 --2- 172.21.15.156:0/2251731218 >> v2:172.21.15.106:3300/0 conn(0x7f49bc0d1380 0x7f49bc0d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.705+0000 7f49bbfff640 1 -- 172.21.15.156:0/2251731218 >> v1:172.21.15.156:6789/0 conn(0x7f49bc0d3500 legacy=0x7f49bc0cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.707 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.705+0000 7f49bbfff640 1 -- 172.21.15.156:0/2251731218 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc0d20a0 con 0x7f49bc160a00 2023-11-24T08:08:29.708 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.705+0000 7f49baffd640 1 -- 172.21.15.156:0/2251731218 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801289965 0 0) 0x7f49bc0d1810 con 0x7f49bc0d3500 2023-11-24T08:08:29.708 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.705+0000 7f682b62b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T08:08:29.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.706+0000 7f49bbfff640 1 --2- 172.21.15.156:0/2251731218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc160a00 0x7f49bc0d0e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b5db0 tx=0x7f49ac02b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5829adedd00e71d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.707+0000 7f49baffd640 1 -- 172.21.15.156:0/2251731218 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac0317e0 con 0x7f49bc160a00 2023-11-24T08:08:29.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.707+0000 7f49baffd640 1 -- 172.21.15.156:0/2251731218 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49ac031940 con 0x7f49bc160a00 2023-11-24T08:08:29.709 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.707+0000 7f49baffd640 1 -- 172.21.15.156:0/2251731218 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac031cd0 con 0x7f49bc160a00 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.707+0000 7f49c10ea640 1 -- 172.21.15.156:0/2251731218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc160a00 msgr2=0x7f49bc0d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.707+0000 7f49c10ea640 1 --2- 172.21.15.156:0/2251731218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc160a00 0x7f49bc0d0e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b5db0 tx=0x7f49ac02b880 comp rx=0 tx=0).stop 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.708+0000 7f49c10ea640 1 -- 172.21.15.156:0/2251731218 shutdown_connections 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.708+0000 7f49c10ea640 1 --2- 172.21.15.156:0/2251731218 >> v2:172.21.15.106:3300/0 conn(0x7f49bc0d1380 0x7f49bc0d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.708+0000 7f49c10ea640 1 --2- 172.21.15.156:0/2251731218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc160a00 0x7f49bc0d0e40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.708+0000 7f49c10ea640 1 -- 172.21.15.156:0/2251731218 >> 172.21.15.156:0/2251731218 conn(0x7f49bc08cc90 msgr2=0x7f49bc09c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.708+0000 7f49c10ea640 1 -- 172.21.15.156:0/2251731218 shutdown_connections 2023-11-24T08:08:29.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.708+0000 7f49c10ea640 1 -- 172.21.15.156:0/2251731218 wait complete. 2023-11-24T08:08:29.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.709+0000 7f49c10ea640 1 Processor -- start 2023-11-24T08:08:29.711 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.709+0000 7f49c10ea640 1 -- start start 2023-11-24T08:08:29.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.710+0000 7f49c10ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49bc0d1380 0x7f49bc0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.710+0000 7f49c10ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 0x7f49bc0c77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.710+0000 7f49c10ea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49bc160a00 0x7f49bc1f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.710+0000 7f49c10ea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cbec0 con 0x7f49bc0d1380 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.710+0000 7f49c10ea640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cb240 con 0x7f49bc0d3500 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.710+0000 7f49c10ea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0cb520 con 0x7f49bc160a00 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 0x7f49bc0c77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 0x7f49bc0c77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:57048/0 (socket says 172.21.15.156:57048) 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 -- 172.21.15.156:0/4069546147 learned_addr learned my addr 172.21.15.156:0/4069546147 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49c08e9640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49bc160a00 0x7f49bc1f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bbfff640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49bc0d1380 0x7f49bc0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:29.714 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 -- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49bc160a00 msgr2=0x7f49bc1f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49bc160a00 0x7f49bc1f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 -- 172.21.15.156:0/4069546147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49bc0d1380 msgr2=0x7f49bc0cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49bc0d1380 0x7f49bc0cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 -- 172.21.15.156:0/4069546147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac0094d0 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.711+0000 7f49bb7fe640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 0x7f49bc0c77b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d5f0 tx=0x7f49b000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b00125f0 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0012750 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0012b00 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc0cb800 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc0c98d0 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f49b001e460 con 0x7f49bc0d3500 2023-11-24T08:08:29.715 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.712+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f49b0014910 con 0x7f49bc0d3500 2023-11-24T08:08:29.716 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.713+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc04e840 con 0x7f49bc0d3500 2023-11-24T08:08:29.717 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.716+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f49b0014ad0 con 0x7f49bc0d3500 2023-11-24T08:08:29.758 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.757+0000 7f682b62b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T08:08:29.760 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:29.759+0000 7fb8cfc56040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-24T08:08:29.796 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.794+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd", "id": 1} v 0) v1 -- 0x7f49bc0b29d0 con 0x7f49bc0d3500 2023-11-24T08:08:29.801 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.800+0000 7f49b8ff9640 1 -- 172.21.15.156:0/4069546147 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f49b0014d40 con 0x7f49bc0d3500 2023-11-24T08:08:29.801 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 msgr2=0x7f49bc0c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 0x7f49bc0c77b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f49b000d5f0 tx=0x7f49b000dab0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 shutdown_connections 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49bc160a00 0x7f49bc1f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49bc0d3500 0x7f49bc0c77b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 --2- 172.21.15.156:0/4069546147 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49bc0d1380 0x7f49bc0cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 >> 172.21.15.156:0/4069546147 conn(0x7f49bc08cc90 msgr2=0x7f49bc0b18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 shutdown_connections 2023-11-24T08:08:29.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:29.801+0000 7f49c10ea640 1 -- 172.21.15.156:0/4069546147 wait complete. 2023-11-24T08:08:29.820 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 3891f879-7b91-401a-83f2-fcf5bde64519 2 2023-11-24T08:08:29.821 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.820+0000 7f682b62b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T08:08:29.826 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:29.824+0000 7fb8cfc56040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-24T08:08:29.871 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.869+0000 7f682b62b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T08:08:29.881 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:29.879+0000 7fb8cfc56040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-24T08:08:29.949 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:29.947+0000 7fb8cfc56040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-24T08:08:29.984 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:29.982+0000 7f682b62b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T08:08:30.002 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.000+0000 7fb8cfc56040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-24T08:08:30.006 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.005+0000 7fc194aad640 1 Processor -- start 2023-11-24T08:08:30.007 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.005+0000 7fc194aad640 1 -- start start 2023-11-24T08:08:30.009 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.007+0000 7fc194aad640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc19015d720 0x7fc1900d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.007+0000 7fc194aad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc19006ca90 0x7fc1900b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.007+0000 7fc194aad640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19015db00 con 0x7fc1900d3870 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.007+0000 7fc194aad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1900d3d20 con 0x7fc19006ca90 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.007+0000 7fc194aad640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1900d0ce0 con 0x7fc19015d720 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.007+0000 7fc18e59c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc19015d720 0x7fc1900d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18e59c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc19015d720 0x7fc1900d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48600/0 (socket says 172.21.15.156:48600) 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18e59c640 1 -- 172.21.15.156:0/1053996990 learned_addr learned my addr 172.21.15.156:0/1053996990 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18ed9d640 1 --2- 172.21.15.156:0/1053996990 >> v2:172.21.15.106:3300/0 conn(0x7fc19006ca90 0x7fc1900b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18e59c640 1 -- 172.21.15.156:0/1053996990 >> v2:172.21.15.106:3300/0 conn(0x7fc19006ca90 msgr2=0x7fc1900b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.010 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18e59c640 1 --2- 172.21.15.156:0/1053996990 >> v2:172.21.15.106:3300/0 conn(0x7fc19006ca90 0x7fc1900b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18e59c640 1 -- 172.21.15.156:0/1053996990 >> v1:172.21.15.106:6789/0 conn(0x7fc1900d3870 legacy=0x7fc1900b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.011 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.008+0000 7fc18e59c640 1 -- 172.21.15.156:0/1053996990 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19006d400 con 0x7fc19015d720 2023-11-24T08:08:30.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.009+0000 7fc18e59c640 1 --2- 172.21.15.156:0/1053996990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc1900d3330 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc19008b890 tx=0x7fc18002b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c00e6d5be624c470 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.009+0000 7fc18dd9b640 1 -- 172.21.15.156:0/1053996990 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800317e0 con 0x7fc19015d720 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.009+0000 7fc18dd9b640 1 -- 172.21.15.156:0/1053996990 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc180031940 con 0x7fc19015d720 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.009+0000 7fc18dd9b640 1 -- 172.21.15.156:0/1053996990 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180031cd0 con 0x7fc19015d720 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 -- 172.21.15.156:0/1053996990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 msgr2=0x7fc1900d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 --2- 172.21.15.156:0/1053996990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc1900d3330 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc19008b890 tx=0x7fc18002b880 comp rx=0 tx=0).stop 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 -- 172.21.15.156:0/1053996990 shutdown_connections 2023-11-24T08:08:30.013 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 --2- 172.21.15.156:0/1053996990 >> v2:172.21.15.106:3300/0 conn(0x7fc19006ca90 0x7fc1900b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 --2- 172.21.15.156:0/1053996990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc1900d3330 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 -- 172.21.15.156:0/1053996990 >> 172.21.15.156:0/1053996990 conn(0x7fc19008af60 msgr2=0x7fc19009c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 -- 172.21.15.156:0/1053996990 shutdown_connections 2023-11-24T08:08:30.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.010+0000 7fc194aad640 1 -- 172.21.15.156:0/1053996990 wait complete. 2023-11-24T08:08:30.014 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.013+0000 7fc194aad640 1 Processor -- start 2023-11-24T08:08:30.015 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.013+0000 7fc194aad640 1 -- start start 2023-11-24T08:08:30.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc194aad640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 0x7fc1900c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc194aad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1900d3870 0x7fc19014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.017 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc194aad640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc19015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc194aad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c7fa0 con 0x7fc1900d3870 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc194aad640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c9da0 con 0x7fc19015d720 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc194aad640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1900c9ac0 con 0x7fc19006ca90 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 0x7fc1900c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc19015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 0x7fc1900c8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52174/0 (socket says 172.21.15.156:52174) 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18ed9d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc19015a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48602/0 (socket says 172.21.15.156:48602) 2023-11-24T08:08:30.018 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 -- 172.21.15.156:0/1429133181 learned_addr learned my addr 172.21.15.156:0/1429133181 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc187fff640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1900d3870 0x7fc19014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 -- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 msgr2=0x7fc19015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc19015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 -- 172.21.15.156:0/1429133181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1900d3870 msgr2=0x7fc19014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1900d3870 0x7fc19014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18e59c640 1 -- 172.21.15.156:0/1429133181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800094d0 con 0x7fc19006ca90 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.015+0000 7fc18ed9d640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc19015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:30.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.016+0000 7fc18e59c640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 0x7fc1900c8430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc180000c00 tx=0x7fc1800046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.016+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180032880 con 0x7fc19006ca90 2023-11-24T08:08:30.020 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.016+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1800329e0 con 0x7fc19006ca90 2023-11-24T08:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.016+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc180032d90 con 0x7fc19006ca90 2023-11-24T08:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.016+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1900c7950 con 0x7fc19006ca90 2023-11-24T08:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.016+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19014ade0 con 0x7fc19006ca90 2023-11-24T08:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.017+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc1800456f0 con 0x7fc19006ca90 2023-11-24T08:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.017+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc180045dc0 con 0x7fc19006ca90 2023-11-24T08:08:30.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.018+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19004e8b0 con 0x7fc19006ca90 2023-11-24T08:08:30.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.020+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fc180063ce0 con 0x7fc19006ca90 2023-11-24T08:08:30.064 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.063+0000 7f682b62b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T08:08:30.100 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.099+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3891f879-7b91-401a-83f2-fcf5bde64519", "id": 2} v 0) v1 -- 0x7fc1900cee70 con 0x7fc19006ca90 2023-11-24T08:08:30.107 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.106+0000 7fc186ffd640 1 -- 172.21.15.156:0/1429133181 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3891f879-7b91-401a-83f2-fcf5bde64519", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc180052070 con 0x7fc19006ca90 2023-11-24T08:08:30.107 INFO:teuthology.orchestra.run.smithi156.stdout:2 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 msgr2=0x7fc1900c8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 0x7fc1900c8430 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc180000c00 tx=0x7fc1800046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 shutdown_connections 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc19015d720 0x7fc19015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1900d3870 0x7fc19014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 --2- 172.21.15.156:0/1429133181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc19006ca90 0x7fc1900c8430 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 >> 172.21.15.156:0/1429133181 conn(0x7fc19008af60 msgr2=0x7fc1900cf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.109 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 shutdown_connections 2023-11-24T08:08:30.110 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.107+0000 7fc194aad640 1 -- 172.21.15.156:0/1429133181 wait complete. 2023-11-24T08:08:30.118 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.117+0000 7f682b62b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T08:08:30.125 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new e9297628-723d-459b-8077-39d9e0b212ee 3 2023-11-24T08:08:30.126 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.124+0000 7fb8cfc56040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-11-24T08:08:30.179 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.177+0000 7f682b62b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T08:08:30.212 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.211+0000 7fb8cfc56040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-24T08:08:30.233 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.231+0000 7f682b62b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T08:08:30.269 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.267+0000 7fb8cfc56040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-24T08:08:30.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.311+0000 7f612bfff640 1 Processor -- start 2023-11-24T08:08:30.313 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.311+0000 7f612bfff640 1 -- start start 2023-11-24T08:08:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612bfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c0d12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f612c160dd0 0x7f612c0d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612bfff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c0b41b0 con 0x7f612c0b2050 2023-11-24T08:08:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612bfff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c0b4490 con 0x7f612c160dd0 2023-11-24T08:08:30.315 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612bfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0b0090 con 0x7f612c1609f0 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c0d12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c0d12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52180/0 (socket says 172.21.15.156:52180) 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 -- 172.21.15.156:0/377061873 learned_addr learned my addr 172.21.15.156:0/377061873 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612a7fc640 1 --2- 172.21.15.156:0/377061873 >> v2:172.21.15.106:3300/0 conn(0x7f612c160dd0 0x7f612c0d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 -- 172.21.15.156:0/377061873 >> v2:172.21.15.106:3300/0 conn(0x7f612c160dd0 msgr2=0x7f612c0d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 --2- 172.21.15.156:0/377061873 >> v2:172.21.15.106:3300/0 conn(0x7f612c160dd0 0x7f612c0d1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 -- 172.21.15.156:0/377061873 >> v1:172.21.15.106:6789/0 conn(0x7f612c0b2050 legacy=0x7f612c0b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.313+0000 7f612affd640 1 -- 172.21.15.156:0/377061873 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0b0370 con 0x7f612c1609f0 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.314+0000 7f612affd640 1 --2- 172.21.15.156:0/377061873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c0d12f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f611c009870 tx=0x7f611c02b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=abff4b5a5b5dd547 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f6129ffb640 1 -- 172.21.15.156:0/377061873 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c031640 con 0x7f612c1609f0 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f6129ffb640 1 -- 172.21.15.156:0/377061873 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c0317a0 con 0x7f612c1609f0 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f6129ffb640 1 -- 172.21.15.156:0/377061873 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c031b30 con 0x7f612c1609f0 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f612bfff640 1 -- 172.21.15.156:0/377061873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 msgr2=0x7f612c0d12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f612bfff640 1 --2- 172.21.15.156:0/377061873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c0d12f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f611c009870 tx=0x7f611c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f612bfff640 1 -- 172.21.15.156:0/377061873 shutdown_connections 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f612bfff640 1 --2- 172.21.15.156:0/377061873 >> v2:172.21.15.106:3300/0 conn(0x7f612c160dd0 0x7f612c0d1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.318 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f612bfff640 1 --2- 172.21.15.156:0/377061873 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c0d12f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.315+0000 7f612bfff640 1 -- 172.21.15.156:0/377061873 >> 172.21.15.156:0/377061873 conn(0x7f612c09c5b0 msgr2=0x7f612c0d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.316+0000 7f612bfff640 1 -- 172.21.15.156:0/377061873 shutdown_connections 2023-11-24T08:08:30.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.316+0000 7f612bfff640 1 -- 172.21.15.156:0/377061873 wait complete. 2023-11-24T08:08:30.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.317+0000 7f612bfff640 1 Processor -- start 2023-11-24T08:08:30.319 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.317+0000 7f612bfff640 1 -- start start 2023-11-24T08:08:30.321 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.318+0000 7f612bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f612c0b2050 0x7f612c0cae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.318+0000 7f612bfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c1f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.318+0000 7f612bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f612c160dd0 0x7f612c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.318+0000 7f612bfff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0cc060 con 0x7f612c0b2050 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.318+0000 7f612bfff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c1f4240 con 0x7f612c160dd0 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.318+0000 7f612bfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0cb3e0 con 0x7f612c1609f0 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612a7fc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c1f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f612c160dd0 0x7f612c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.322 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612a7fc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c1f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52194/0 (socket says 172.21.15.156:52194) 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612a7fc640 1 -- 172.21.15.156:0/553194135 learned_addr learned my addr 172.21.15.156:0/553194135 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612affd640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f612c0b2050 0x7f612c0cae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 -- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 msgr2=0x7f612c1f3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c1f3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 -- 172.21.15.156:0/553194135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f612c0b2050 msgr2=0x7f612c0cae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f612c0b2050 0x7f612c0cae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 -- 172.21.15.156:0/553194135 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c0094d0 con 0x7f612c160dd0 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612a7fc640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c1f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612affd640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f612c0b2050 0x7f612c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:30.323 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.319+0000 7f612b7fe640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f612c160dd0 0x7f612c15c360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f612400c780 tx=0x7f612400cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400aa50 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f612400abb0 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f612400ce30 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c0cb6c0 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c0cbc80 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f6124004bf0 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.320+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f6124012420 con 0x7f612c160dd0 2023-11-24T08:08:30.324 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.321+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612c0cc1a0 con 0x7f612c160dd0 2023-11-24T08:08:30.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.325+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f6124039ce0 con 0x7f612c160dd0 2023-11-24T08:08:30.336 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.335+0000 7fb8cfc56040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-24T08:08:30.394 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.393+0000 7fb8cfc56040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-24T08:08:30.405 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.404+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e9297628-723d-459b-8077-39d9e0b212ee", "id": 3} v 0) v1 -- 0x7f612c04e840 con 0x7f612c160dd0 2023-11-24T08:08:30.411 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.410+0000 7f61137fe640 1 -- 172.21.15.156:0/553194135 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e9297628-723d-459b-8077-39d9e0b212ee", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6124028070 con 0x7f612c160dd0 2023-11-24T08:08:30.411 INFO:teuthology.orchestra.run.smithi156.stdout:3 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f612c160dd0 msgr2=0x7f612c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f612c160dd0 0x7f612c15c360 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f612400c780 tx=0x7f612400cc40 comp rx=0 tx=0).stop 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 shutdown_connections 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f612c160dd0 0x7f612c15c360 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f612c1609f0 0x7f612c1f3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 --2- 172.21.15.156:0/553194135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f612c0b2050 0x7f612c0cae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 >> 172.21.15.156:0/553194135 conn(0x7f612c09c5b0 msgr2=0x7f612c06cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.413 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 shutdown_connections 2023-11-24T08:08:30.414 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.411+0000 7f612bfff640 1 -- 172.21.15.156:0/553194135 wait complete. 2023-11-24T08:08:30.430 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d 4 2023-11-24T08:08:30.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.609+0000 7fed1a8b4640 1 Processor -- start 2023-11-24T08:08:30.611 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.609+0000 7fed1a8b4640 1 -- start start 2023-11-24T08:08:30.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.611+0000 7fed1a8b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed140d1a00 0x7fed140b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.611+0000 7fed1a8b4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140b41b0 con 0x7fed14160dd0 2023-11-24T08:08:30.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.611+0000 7fed1a8b4640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed140b4490 con 0x7fed141609f0 2023-11-24T08:08:30.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.611+0000 7fed1a8b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed140b0090 con 0x7fed140d1a00 2023-11-24T08:08:30.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.611+0000 7fed198b2640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fed141609f0 0x7fed140d12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33664/0 (socket says 172.21.15.156:33664) 2023-11-24T08:08:30.613 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.611+0000 7fed198b2640 1 -- 172.21.15.156:0/3560957301 learned_addr learned my addr 172.21.15.156:0/3560957301 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-24T08:08:30.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed190b1640 1 --2- 172.21.15.156:0/3560957301 >> v2:172.21.15.106:3300/0 conn(0x7fed140d1a00 0x7fed140b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.615 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 48344540 0 0) 0x7fed140b4490 con 0x7fed141609f0 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed00003520 con 0x7fed141609f0 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3546103135 0 0) 0x7fed00003520 con 0x7fed141609f0 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 >> v2:172.21.15.106:3300/0 conn(0x7fed140d1a00 msgr2=0x7fed140b2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 --2- 172.21.15.156:0/3560957301 >> v2:172.21.15.106:3300/0 conn(0x7fed140d1a00 0x7fed140b2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 >> v1:172.21.15.106:6789/0 conn(0x7fed14160dd0 legacy=0x7fed140b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed140b0370 con 0x7fed141609f0 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fed10001670 con 0x7fed141609f0 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 >> v1:172.21.15.156:6789/0 conn(0x7fed141609f0 legacy=0x7fed140d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed00003970 0x7fed00003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.616 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed00005180 0x7fed0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed140b0090 con 0x7fed00003970 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000c0d0 con 0x7fed00005180 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.612+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000c290 con 0x7fed000047c0 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed190b1640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed1a0b3640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed00005180 0x7fed0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed198b2640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed00003970 0x7fed00003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed190b1640 1 -- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed00005180 msgr2=0x7fed0000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed190b1640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed00005180 0x7fed0000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed190b1640 1 -- 172.21.15.156:0/3560957301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed00003970 msgr2=0x7fed00003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed190b1640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed00003970 0x7fed00003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed190b1640 1 -- 172.21.15.156:0/3560957301 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00003520 con 0x7fed000047c0 2023-11-24T08:08:30.617 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.613+0000 7fed1a0b3640 1 -- 172.21.15.156:0/3560957301 reap_dead start 2023-11-24T08:08:30.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.614+0000 7fed190b1640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed00004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed0400ba30 tx=0x7fed000057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=327588b33aee1572 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.618 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.614+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04032630 con 0x7fed000047c0 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.614+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed04032790 con 0x7fed000047c0 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.614+0000 7fed188b0640 1 -- 172.21.15.156:0/3560957301 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed04032b20 con 0x7fed000047c0 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 msgr2=0x7fed00004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed00004c40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fed0400ba30 tx=0x7fed000057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3560957301 shutdown_connections 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 --2- 172.21.15.156:0/3560957301 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed00004c40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3560957301 >> 172.21.15.156:0/3560957301 conn(0x7fed1409c5b0 msgr2=0x7fed140d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3560957301 shutdown_connections 2023-11-24T08:08:30.619 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.615+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3560957301 wait complete. 2023-11-24T08:08:30.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.618+0000 7fed1a8b4640 1 Processor -- start 2023-11-24T08:08:30.620 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.618+0000 7fed1a8b4640 1 -- start start 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a8b4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed140c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a8b4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed1415c8f0 0x7fed1409d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a8b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1409d560 0x7fed1415a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a8b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1415be20 con 0x7fed1409d560 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a8b4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1409ec20 con 0x7fed1415c8f0 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a8b4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1415fce0 con 0x7fed000047c0 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed140c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed140c74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52212/0 (socket says 172.21.15.156:52212) 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed190b1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed1415c8f0 0x7fed1409d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 -- 172.21.15.156:0/3125928023 learned_addr learned my addr 172.21.15.156:0/3125928023 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 -- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed1415c8f0 msgr2=0x7fed1409d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed1415c8f0 0x7fed1409d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a0b3640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1409d560 0x7fed1415a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 -- 172.21.15.156:0/3125928023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1409d560 msgr2=0x7fed1415a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1409d560 0x7fed1415a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed198b2640 1 -- 172.21.15.156:0/3125928023 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1415ffc0 con 0x7fed000047c0 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.620+0000 7fed1a0b3640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1409d560 0x7fed1415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed198b2640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed140c74d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e6c0 tx=0x7fed1000eb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1000c6c0 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1000c840 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed1409eec0 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed1000cbf0 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed1415f9d0 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.621+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fed1000cd50 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.622+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fed10013a70 con 0x7fed000047c0 2023-11-24T08:08:30.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.622+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed141601b0 con 0x7fed000047c0 2023-11-24T08:08:30.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.625+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7fed10013c30 con 0x7fed000047c0 2023-11-24T08:08:30.686 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.684+0000 7f682b62b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T08:08:30.704 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.702+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d", "id": 4} v 0) v1 -- 0x7fed1404e840 con 0x7fed000047c0 2023-11-24T08:08:30.710 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.709+0000 7fed0a7fc640 1 -- 172.21.15.156:0/3125928023 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fed10013df0 con 0x7fed000047c0 2023-11-24T08:08:30.710 INFO:teuthology.orchestra.run.smithi156.stdout:4 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 msgr2=0x7fed140c74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed140c74d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fed1000e6c0 tx=0x7fed1000eb80 comp rx=0 tx=0).stop 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 shutdown_connections 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed1409d560 0x7fed1415a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed1415c8f0 0x7fed1409d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 --2- 172.21.15.156:0/3125928023 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed000047c0 0x7fed140c74d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.712 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 >> 172.21.15.156:0/3125928023 conn(0x7fed1409c5b0 msgr2=0x7fed140b1890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 shutdown_connections 2023-11-24T08:08:30.713 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.710+0000 7fed1a8b4640 1 -- 172.21.15.156:0/3125928023 wait complete. 2023-11-24T08:08:30.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 6055d743-bcd9-4e65-af0b-5b99fae68c9d 5 2023-11-24T08:08:30.845 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.844+0000 7f682b62b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T08:08:30.887 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:30.886+0000 7fb8cfc56040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-24T08:08:30.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.918+0000 7f5b003c4640 1 Processor -- start 2023-11-24T08:08:30.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.918+0000 7f5b003c4640 1 -- start start 2023-11-24T08:08:30.921 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5b003c4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5af80d3a40 0x7f5af806ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5b003c4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af806bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5b003c4640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5af815db00 con 0x7f5af815d720 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5b003c4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5af80d0ce0 con 0x7f5af80d3a40 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5b003c4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af806d400 con 0x7f5af80b1ed0 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5afe961640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af806bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5afe160640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5af815d720 0x7f5af80d3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33666/0 (socket says 172.21.15.156:33666) 2023-11-24T08:08:30.922 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5afe160640 1 -- 172.21.15.156:0/1344170524 learned_addr learned my addr 172.21.15.156:0/1344170524 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-24T08:08:30.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.920+0000 7f5afd95f640 1 --2- 172.21.15.156:0/1344170524 >> v2:172.21.15.106:3300/0 conn(0x7f5af80d3a40 0x7f5af806ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.921+0000 7f5afd15e640 1 -- 172.21.15.156:0/1344170524 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1875539655 0 0) 0x7f5af815db00 con 0x7f5af815d720 2023-11-24T08:08:30.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.921+0000 7f5afe961640 1 -- 172.21.15.156:0/1344170524 >> v2:172.21.15.106:3300/0 conn(0x7f5af80d3a40 msgr2=0x7f5af806ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:30.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.921+0000 7f5afe961640 1 --2- 172.21.15.156:0/1344170524 >> v2:172.21.15.106:3300/0 conn(0x7f5af80d3a40 0x7f5af806ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.921+0000 7f5afe961640 1 -- 172.21.15.156:0/1344170524 >> v1:172.21.15.156:6789/0 conn(0x7f5af815d720 legacy=0x7f5af80d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.921+0000 7f5afe961640 1 -- 172.21.15.156:0/1344170524 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af80b0100 con 0x7f5af80b1ed0 2023-11-24T08:08:30.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.922+0000 7f5afe961640 1 --2- 172.21.15.156:0/1344170524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af806bfc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5af80d0e80 tx=0x7f5af002be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7a3eb9056ef80c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5afd15e640 1 -- 172.21.15.156:0/1344170524 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af002cbb0 con 0x7f5af80b1ed0 2023-11-24T08:08:30.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5afd15e640 1 -- 172.21.15.156:0/1344170524 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af002cd10 con 0x7f5af80b1ed0 2023-11-24T08:08:30.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5afd15e640 1 -- 172.21.15.156:0/1344170524 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0034660 con 0x7f5af80b1ed0 2023-11-24T08:08:30.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 -- 172.21.15.156:0/1344170524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 msgr2=0x7f5af806bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 --2- 172.21.15.156:0/1344170524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af806bfc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5af80d0e80 tx=0x7f5af002be00 comp rx=0 tx=0).stop 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 -- 172.21.15.156:0/1344170524 shutdown_connections 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 --2- 172.21.15.156:0/1344170524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af806bfc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 --2- 172.21.15.156:0/1344170524 >> v2:172.21.15.106:3300/0 conn(0x7f5af80d3a40 0x7f5af806ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 -- 172.21.15.156:0/1344170524 >> 172.21.15.156:0/1344170524 conn(0x7f5af808af60 msgr2=0x7f5af809c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 -- 172.21.15.156:0/1344170524 shutdown_connections 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.923+0000 7f5b003c4640 1 -- 172.21.15.156:0/1344170524 wait complete. 2023-11-24T08:08:30.926 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.925+0000 7f5b003c4640 1 Processor -- start 2023-11-24T08:08:30.927 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.925+0000 7f5b003c4640 1 -- start start 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5b003c4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af815e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5b003c4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 0x7f5af81f5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5b003c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5af815d720 0x7f5af81594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5b003c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815efe0 con 0x7f5af815d720 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5b003c4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815f2c0 con 0x7f5af80d3a40 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5b003c4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5af815f5a0 con 0x7f5af80b1ed0 2023-11-24T08:08:30.928 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5afd95f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 0x7f5af81f5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5afe160640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af815e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5afd95f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 0x7f5af81f5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48644/0 (socket says 172.21.15.156:48644) 2023-11-24T08:08:30.929 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.926+0000 7f5afd95f640 1 -- 172.21.15.156:0/4053252495 learned_addr learned my addr 172.21.15.156:0/4053252495 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:30.930 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afe961640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5af815d720 0x7f5af81594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afd95f640 1 -- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 msgr2=0x7f5af815e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afd95f640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af815e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afd95f640 1 -- 172.21.15.156:0/4053252495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5af815d720 msgr2=0x7f5af81594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afd95f640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5af815d720 0x7f5af81594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afd95f640 1 -- 172.21.15.156:0/4053252495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5af0009a70 con 0x7f5af80d3a40 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5afd95f640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 0x7f5af81f5a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d7d0 tx=0x7f5ae800dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800b6c0 con 0x7f5af80d3a40 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ae800b820 con 0x7f5af80d3a40 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af815f880 con 0x7f5af80d3a40 2023-11-24T08:08:30.931 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.927+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae800bbb0 con 0x7f5af80d3a40 2023-11-24T08:08:30.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.928+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af8160990 con 0x7f5af80d3a40 2023-11-24T08:08:30.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.928+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5ae801e760 con 0x7f5af80d3a40 2023-11-24T08:08:30.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.928+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f5ae8014520 con 0x7f5af80d3a40 2023-11-24T08:08:30.932 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.929+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af80d1f30 con 0x7f5af80d3a40 2023-11-24T08:08:30.933 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:30.931+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f5ae80146e0 con 0x7f5af80d3a40 2023-11-24T08:08:30.978 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:30.976+0000 7f682b62b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T08:08:31.012 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.011+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6055d743-bcd9-4e65-af0b-5b99fae68c9d", "id": 5} v 0) v1 -- 0x7f5af804e8b0 con 0x7f5af80d3a40 2023-11-24T08:08:31.019 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.018+0000 7f5aeeffd640 1 -- 172.21.15.156:0/4053252495 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6055d743-bcd9-4e65-af0b-5b99fae68c9d", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5ae801ea10 con 0x7f5af80d3a40 2023-11-24T08:08:31.020 INFO:teuthology.orchestra.run.smithi156.stdout:5 2023-11-24T08:08:31.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 msgr2=0x7f5af81f5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 0x7f5af81f5a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800d7d0 tx=0x7f5ae800dc90 comp rx=0 tx=0).stop 2023-11-24T08:08:31.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 shutdown_connections 2023-11-24T08:08:31.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5af815d720 0x7f5af81594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.021 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5af80d3a40 0x7f5af81f5a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 --2- 172.21.15.156:0/4053252495 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5af80b1ed0 0x7f5af815e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 >> 172.21.15.156:0/4053252495 conn(0x7f5af808af60 msgr2=0x7f5af80b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:31.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 shutdown_connections 2023-11-24T08:08:31.022 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.019+0000 7f5b003c4640 1 -- 172.21.15.156:0/4053252495 wait complete. 2023-11-24T08:08:31.031 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:31.030+0000 7f682b62b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T08:08:31.036 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new 1516397b-f6b3-4f54-b7a2-c5e0de5811c9 6 2023-11-24T08:08:31.062 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.061+0000 7fb8cfc56040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-24T08:08:31.140 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:31.139+0000 7f682b62b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:08:31.217 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.215+0000 7fb8cfc56040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-24T08:08:31.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.222+0000 7f23c55ef640 1 Processor -- start 2023-11-24T08:08:31.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.222+0000 7f23c55ef640 1 -- start start 2023-11-24T08:08:31.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23c55ef640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c0160a00 0x7f23c00d0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.225 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23c55ef640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f23c00d1380 0x7f23c00d2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23c55ef640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23c00d1810 con 0x7f23c00d3500 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23c55ef640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00d1ae0 con 0x7f23c00d1380 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23c55ef640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23c00d1dc0 con 0x7f23c0160a00 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23bffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c0160a00 0x7f23c00d0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.223+0000 7f23c4dee640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f23c00d3500 0x7f23c00cfc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.156:33676/0 (socket says 172.21.15.156:33676) 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23c4dee640 1 -- 172.21.15.156:0/171645301 learned_addr learned my addr 172.21.15.156:0/171645301 (peer_addr_for_me v1:172.21.15.156:0/0) 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23bffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f23c0160a00 0x7f23c00d0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48656/0 (socket says 172.21.15.156:48656) 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23b7fff640 1 --2- 172.21.15.156:0/171645301 >> v2:172.21.15.106:3300/0 conn(0x7f23c00d1380 0x7f23c00d2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23bffff640 1 -- 172.21.15.156:0/171645301 >> v2:172.21.15.106:3300/0 conn(0x7f23c00d1380 msgr2=0x7f23c00d2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23bffff640 1 --2- 172.21.15.156:0/171645301 >> v2:172.21.15.106:3300/0 conn(0x7f23c00d1380 0x7f23c00d2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23bffff640 1 -- 172.21.15.156:0/171645301 >> v1:172.21.15.156:6789/0 conn(0x7f23c00d3500 legacy=0x7f23c00cfc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.224+0000 7f23bffff640 1 -- 172.21.15.156:0/171645301 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c00d20a0 con 0x7f23c0160a00 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.225+0000 7f23bffff640 1 --2- 172.21.15.156:0/171645301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c0160a00 0x7f23c00d0e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23c00b5db0 tx=0x7f23b002b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78cf99d827be0640 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.225+0000 7f23bf7fe640 1 -- 172.21.15.156:0/171645301 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0031520 con 0x7f23c0160a00 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.225+0000 7f23bf7fe640 1 -- 172.21.15.156:0/171645301 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0031680 con 0x7f23c0160a00 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.225+0000 7f23bf7fe640 1 -- 172.21.15.156:0/171645301 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0031a10 con 0x7f23c0160a00 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 -- 172.21.15.156:0/171645301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c0160a00 msgr2=0x7f23c00d0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 --2- 172.21.15.156:0/171645301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c0160a00 0x7f23c00d0e40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f23c00b5db0 tx=0x7f23b002b7c0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 -- 172.21.15.156:0/171645301 shutdown_connections 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 --2- 172.21.15.156:0/171645301 >> v2:172.21.15.106:3300/0 conn(0x7f23c00d1380 0x7f23c00d2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 --2- 172.21.15.156:0/171645301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c0160a00 0x7f23c00d0e40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 -- 172.21.15.156:0/171645301 >> 172.21.15.156:0/171645301 conn(0x7f23c008cc90 msgr2=0x7f23c009c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:31.228 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 -- 172.21.15.156:0/171645301 shutdown_connections 2023-11-24T08:08:31.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.226+0000 7f23c55ef640 1 -- 172.21.15.156:0/171645301 wait complete. 2023-11-24T08:08:31.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.228+0000 7f23c55ef640 1 Processor -- start 2023-11-24T08:08:31.229 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.228+0000 7f23c55ef640 1 -- start start 2023-11-24T08:08:31.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c55ef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 0x7f23c00c7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c55ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 0x7f23c0158ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c55ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23c0160a00 0x7f23c00cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c55ef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0156760 con 0x7f23c0160a00 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c55ef640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c01569b0 con 0x7f23c00d1380 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c55ef640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0156c90 con 0x7f23c00d3500 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 0x7f23c00c7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 0x7f23c0158ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 0x7f23c00c7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48670/0 (socket says 172.21.15.156:48670) 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 -- 172.21.15.156:0/794158625 learned_addr learned my addr 172.21.15.156:0/794158625 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23b7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 0x7f23c0158ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52240/0 (socket says 172.21.15.156:52240) 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c4dee640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23c0160a00 0x7f23c00cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 -- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 msgr2=0x7f23c0158ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 0x7f23c0158ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 -- 172.21.15.156:0/794158625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23c0160a00 msgr2=0x7f23c00cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23c0160a00 0x7f23c00cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23bffff640 1 -- 172.21.15.156:0/794158625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0009400 con 0x7f23c00d1380 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23b7fff640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 0x7f23c0158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.229+0000 7f23c4dee640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23c0160a00 0x7f23c00cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.230+0000 7f23bffff640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 0x7f23c00c7820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.230+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b0033c50 con 0x7f23c00d1380 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.230+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b0033dd0 con 0x7f23c00d1380 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.230+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b003c640 con 0x7f23c00d1380 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.230+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c0156f70 con 0x7f23c00d1380 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.230+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c00c7d60 con 0x7f23c00d1380 2023-11-24T08:08:31.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.231+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f23b003a040 con 0x7f23c00d1380 2023-11-24T08:08:31.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.231+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f23b0048b80 con 0x7f23c00d1380 2023-11-24T08:08:31.235 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.231+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c00c7ef0 con 0x7f23c00d1380 2023-11-24T08:08:31.237 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.235+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f23b004d060 con 0x7f23c00d1380 2023-11-24T08:08:31.251 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:31.249+0000 7f682b62b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T08:08:31.278 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.275+0000 7fb8cfc56040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-24T08:08:31.316 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.315+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1516397b-f6b3-4f54-b7a2-c5e0de5811c9", "id": 6} v 0) v1 -- 0x7f23c0157490 con 0x7f23c00d1380 2023-11-24T08:08:31.325 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.324+0000 7f23bd7fa640 1 -- 172.21.15.156:0/794158625 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1516397b-f6b3-4f54-b7a2-c5e0de5811c9", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f23c0157490 con 0x7f23c00d1380 2023-11-24T08:08:31.326 INFO:teuthology.orchestra.run.smithi156.stdout:6 2023-11-24T08:08:31.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.325+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 msgr2=0x7f23c00c7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.325+0000 7f23c55ef640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 0x7f23c00c7820 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f23b0000c00 tx=0x7f23b00046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.325+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 shutdown_connections 2023-11-24T08:08:31.327 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.325+0000 7f23c55ef640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f23c0160a00 0x7f23c00cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.325+0000 7f23c55ef640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f23c00d3500 0x7f23c0158ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.325+0000 7f23c55ef640 1 --2- 172.21.15.156:0/794158625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23c00d1380 0x7f23c00c7820 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.326+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 >> 172.21.15.156:0/794158625 conn(0x7f23c008cc90 msgr2=0x7f23c006ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.326+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 shutdown_connections 2023-11-24T08:08:31.328 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.326+0000 7f23c55ef640 1 -- 172.21.15.156:0/794158625 wait complete. 2023-11-24T08:08:31.344 DEBUG:teuthology.orchestra.run.smithi156:> sudo ceph --cluster ceph osd new c8bb85e2-5999-4d43-9c54-e60aa7949caf 7 2023-11-24T08:08:31.395 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.394+0000 7fb8cfc56040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:08:31.411 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:31.409+0000 7f682b62b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:08:31.461 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:31.459+0000 7f682b62b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T08:08:31.515 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.513+0000 7fb8cfc56040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-24T08:08:31.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.581+0000 7ffbb7fff640 1 Processor -- start 2023-11-24T08:08:31.583 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.582+0000 7ffbb7fff640 1 -- start start 2023-11-24T08:08:31.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.583+0000 7ffbb7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 0x7ffbb806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.583+0000 7ffbb7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffbb80d09c0 0x7ffbb80cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.583+0000 7ffbb7fff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb806f0d0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.583+0000 7ffbb7fff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb806f970 con 0x7ffbb80d09c0 2023-11-24T08:08:31.585 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.583+0000 7ffbb7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb806f3b0 con 0x7ffbb80c4490 2023-11-24T08:08:31.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbaffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffbb80d09c0 0x7ffbb80cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbb6ffd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 0x7ffbb806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbaffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffbb80d09c0 0x7ffbb80cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:48686/0 (socket says 172.21.15.156:48686) 2023-11-24T08:08:31.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbaffff640 1 -- 172.21.15.156:0/1822911107 learned_addr learned my addr 172.21.15.156:0/1822911107 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:31.586 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbb6ffd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 0x7ffbb806e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52250/0 (socket says 172.21.15.156:52250) 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbb6ffd640 1 -- 172.21.15.156:0/1822911107 >> v2:172.21.15.156:3300/0 conn(0x7ffbb80d09c0 msgr2=0x7ffbb80cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbb6ffd640 1 --2- 172.21.15.156:0/1822911107 >> v2:172.21.15.156:3300/0 conn(0x7ffbb80d09c0 0x7ffbb80cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbb6ffd640 1 -- 172.21.15.156:0/1822911107 >> v1:172.21.15.156:6789/0 conn(0x7ffbb80cbc50 legacy=0x7ffbb80cefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.584+0000 7ffbb6ffd640 1 -- 172.21.15.156:0/1822911107 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb806f690 con 0x7ffbb80c4490 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.585+0000 7ffbb6ffd640 1 --2- 172.21.15.156:0/1822911107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 0x7ffbb806e8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffba80097a0 tx=0x7ffba802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=84726e54d78b756 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.585+0000 7ffbb67fc640 1 -- 172.21.15.156:0/1822911107 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba802d040 con 0x7ffbb80c4490 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.585+0000 7ffbb67fc640 1 -- 172.21.15.156:0/1822911107 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba8031810 con 0x7ffbb80c4490 2023-11-24T08:08:31.587 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.585+0000 7ffbb67fc640 1 -- 172.21.15.156:0/1822911107 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba8031ba0 con 0x7ffbb80c4490 2023-11-24T08:08:31.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 -- 172.21.15.156:0/1822911107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 msgr2=0x7ffbb806e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/1822911107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 0x7ffbb806e8a0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ffba80097a0 tx=0x7ffba802b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 -- 172.21.15.156:0/1822911107 shutdown_connections 2023-11-24T08:08:31.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/1822911107 >> v2:172.21.15.156:3300/0 conn(0x7ffbb80d09c0 0x7ffbb80cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/1822911107 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80c4490 0x7ffbb806e8a0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 -- 172.21.15.156:0/1822911107 >> 172.21.15.156:0/1822911107 conn(0x7ffbb80d10c0 msgr2=0x7ffbb80d14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:31.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 -- 172.21.15.156:0/1822911107 shutdown_connections 2023-11-24T08:08:31.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.586+0000 7ffbb7fff640 1 -- 172.21.15.156:0/1822911107 wait complete. 2023-11-24T08:08:31.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.588+0000 7ffbb7fff640 1 Processor -- start 2023-11-24T08:08:31.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.588+0000 7ffbb7fff640 1 -- start start 2023-11-24T08:08:31.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbb80c4490 0x7ffbb8156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 0x7ffbb8161fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb80d09c0 0x7ffbb8155f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb7fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb81624e0 con 0x7ffbb80c4490 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb7fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8159000 con 0x7ffbb80d09c0 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb8163300 con 0x7ffbb80cbc50 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 0x7ffbb8161fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbaffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 0x7ffbb8161fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.156:52256/0 (socket says 172.21.15.156:52256) 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbaffff640 1 -- 172.21.15.156:0/3301094244 learned_addr learned my addr 172.21.15.156:0/3301094244 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.589+0000 7ffbb77fe640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb80d09c0 0x7ffbb8155f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbb6ffd640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbb80c4490 0x7ffbb8156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbaffff640 1 -- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb80d09c0 msgr2=0x7ffbb8155f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbaffff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb80d09c0 0x7ffbb8155f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbaffff640 1 -- 172.21.15.156:0/3301094244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbb80c4490 msgr2=0x7ffbb8156760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbaffff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbb80c4490 0x7ffbb8156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbaffff640 1 -- 172.21.15.156:0/3301094244 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8009400 con 0x7ffbb80cbc50 2023-11-24T08:08:31.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbaffff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 0x7ffbb8161fa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffba000e870 tx=0x7ffba000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:31.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000c8e0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba000ca40 con 0x7ffbb80cbc50 2023-11-24T08:08:31.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb81626f0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba000cdf0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.590+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb8162cb0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.591+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffba0005c90 con 0x7ffbb80cbc50 2023-11-24T08:08:31.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.591+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7ffba0013ad0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.591+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb804e840 con 0x7ffbb80cbc50 2023-11-24T08:08:31.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.594+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7ffba0021060 con 0x7ffbb80cbc50 2023-11-24T08:08:31.618 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:31.616+0000 7f682b62b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T08:08:31.673 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.672+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c8bb85e2-5999-4d43-9c54-e60aa7949caf", "id": 7} v 0) v1 -- 0x7ffbb80d1ec0 con 0x7ffbb80cbc50 2023-11-24T08:08:31.681 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.679+0000 7ffbb4ff9640 1 -- 172.21.15.156:0/3301094244 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c8bb85e2-5999-4d43-9c54-e60aa7949caf", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffba0029020 con 0x7ffbb80cbc50 2023-11-24T08:08:31.681 INFO:teuthology.orchestra.run.smithi156.stdout:7 2023-11-24T08:08:31.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 msgr2=0x7ffbb8161fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 0x7ffbb8161fa0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffba000e870 tx=0x7ffba000ed30 comp rx=0 tx=0).stop 2023-11-24T08:08:31.682 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 shutdown_connections 2023-11-24T08:08:31.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb80d09c0 0x7ffbb8155f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffbb80cbc50 0x7ffbb8161fa0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 --2- 172.21.15.156:0/3301094244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffbb80c4490 0x7ffbb8156760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 >> 172.21.15.156:0/3301094244 conn(0x7ffbb80d10c0 msgr2=0x7ffbb8070370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:31.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.680+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 shutdown_connections 2023-11-24T08:08:31.683 INFO:teuthology.orchestra.run.smithi156.stderr:2023-11-24T08:08:31.681+0000 7ffbb7fff640 1 -- 172.21.15.156:0/3301094244 wait complete. 2023-11-24T08:08:31.683 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.680+0000 7fb8cfc56040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-24T08:08:31.699 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-24T08:08:31.699 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-24T08:08:31.708 INFO:tasks.ceph.osd.0:Started 2023-11-24T08:08:31.708 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-24T08:08:31.708 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-24T08:08:31.717 INFO:tasks.ceph.osd.1:Started 2023-11-24T08:08:31.717 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-24T08:08:31.717 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-24T08:08:31.722 INFO:tasks.ceph.osd.2:Started 2023-11-24T08:08:31.723 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-24T08:08:31.723 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-24T08:08:31.728 INFO:tasks.ceph.osd.3:Started 2023-11-24T08:08:31.729 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-24T08:08:31.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-24T08:08:31.735 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.734+0000 7fb8cfc56040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-24T08:08:31.743 INFO:tasks.ceph.osd.4:Started 2023-11-24T08:08:31.743 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-24T08:08:31.743 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-24T08:08:31.751 INFO:tasks.ceph.osd.5:Started 2023-11-24T08:08:31.751 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-24T08:08:31.751 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-24T08:08:31.757 INFO:tasks.ceph.osd.6:Started 2023-11-24T08:08:31.757 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-24T08:08:31.757 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-24T08:08:31.764 INFO:tasks.ceph.osd.7:Started 2023-11-24T08:08:31.764 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:31.946 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:31.945+0000 7fb8cfc56040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-24T08:08:31.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.980+0000 7f02d7e9b640 1 Processor -- start 2023-11-24T08:08:31.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.980+0000 7f02d7e9b640 1 -- start start 2023-11-24T08:08:31.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.982+0000 7f02d7e9b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02d015c7f0 0x7f02d006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.982+0000 7f02d7e9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f02d00cf450 0x7f02d00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.982+0000 7f02d7e9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.982+0000 7f02d7e9b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d00cfe40 con 0x7f02d00cf450 2023-11-24T08:08:31.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.982+0000 7f02d7e9b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02d0156010 con 0x7f02d015c7f0 2023-11-24T08:08:31.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.982+0000 7f02d7e9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d006ce30 con 0x7f02d00cdfb0 2023-11-24T08:08:31.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f02d00cf450 0x7f02d00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5c37640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f02d015c7f0 0x7f02d006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f02d00cf450 0x7f02d00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43038/0 (socket says 172.21.15.106:43038) 2023-11-24T08:08:31.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 -- 172.21.15.106:0/4176203814 learned_addr learned my addr 172.21.15.106:0/4176203814 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:31.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d6438640 1 --2- 172.21.15.106:0/4176203814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 -- 172.21.15.106:0/4176203814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 msgr2=0x7f02d00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:31.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 --2- 172.21.15.106:0/4176203814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 -- 172.21.15.106:0/4176203814 >> v2:172.21.15.156:3300/0 conn(0x7f02d015c7f0 msgr2=0x7f02d006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:31.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 --2- 172.21.15.106:0/4176203814 >> v2:172.21.15.156:3300/0 conn(0x7f02d015c7f0 0x7f02d006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.983+0000 7f02d5436640 1 -- 172.21.15.106:0/4176203814 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d00cece0 con 0x7f02d00cf450 2023-11-24T08:08:31.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.984+0000 7f02d5436640 1 --2- 172.21.15.106:0/4176203814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d00cf450 0x7f02d00cf900 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02d00d0130 tx=0x7f02c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d760b0fec03ff5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.985+0000 7f02d4c35640 1 -- 172.21.15.106:0/4176203814 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0031870 con 0x7f02d00cf450 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.985+0000 7f02d4c35640 1 -- 172.21.15.106:0/4176203814 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02c00319d0 con 0x7f02d00cf450 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.985+0000 7f02d4c35640 1 -- 172.21.15.106:0/4176203814 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02c0031d60 con 0x7f02d00cf450 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.985+0000 7f02d7e9b640 1 -- 172.21.15.106:0/4176203814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d00cf450 msgr2=0x7f02d00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.985+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/4176203814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d00cf450 0x7f02d00cf900 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f02d00d0130 tx=0x7f02c002b910 comp rx=0 tx=0).stop 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 -- 172.21.15.106:0/4176203814 shutdown_connections 2023-11-24T08:08:31.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/4176203814 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/4176203814 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d00cf450 0x7f02d00cf900 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/4176203814 >> v2:172.21.15.156:3300/0 conn(0x7f02d015c7f0 0x7f02d006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 -- 172.21.15.106:0/4176203814 >> 172.21.15.106:0/4176203814 conn(0x7f02d006a920 msgr2=0x7f02d006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:31.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 -- 172.21.15.106:0/4176203814 shutdown_connections 2023-11-24T08:08:31.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.986+0000 7f02d7e9b640 1 -- 172.21.15.106:0/4176203814 wait complete. 2023-11-24T08:08:31.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.987+0000 7f02d7e9b640 1 Processor -- start 2023-11-24T08:08:31.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.987+0000 7f02d7e9b640 1 -- start start 2023-11-24T08:08:31.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.988+0000 7f02d7e9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d0147770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.988+0000 7f02d7e9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d00cf450 0x7f02d014c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.988+0000 7f02d7e9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d015c7f0 0x7f02d01596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:31.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.988+0000 7f02d7e9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d00accd0 con 0x7f02d015c7f0 2023-11-24T08:08:31.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.988+0000 7f02d7e9b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d014feb0 con 0x7f02d00cf450 2023-11-24T08:08:31.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.988+0000 7f02d7e9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0150130 con 0x7f02d00cdfb0 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d0147770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d0147770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:59604/0 (socket says 172.21.15.106:59604) 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 -- 172.21.15.106:0/1273042687 learned_addr learned my addr 172.21.15.106:0/1273042687 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5436640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d00cf450 0x7f02d014c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d6438640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d015c7f0 0x7f02d01596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 -- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d00cf450 msgr2=0x7f02d014c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d00cf450 0x7f02d014c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 -- 172.21.15.106:0/1273042687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d015c7f0 msgr2=0x7f02d01596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d015c7f0 0x7f02d01596b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:31.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 -- 172.21.15.106:0/1273042687 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02c00094d0 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5436640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d00cf450 0x7f02d014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d6438640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d015c7f0 0x7f02d01596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.989+0000 7f02d5c37640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d0147770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e8a0 tx=0x7f02cc00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc0094f0 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02cc00c940 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d014c690 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02cc00ccf0 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d01503d0 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f02cc005c90 con 0x7f02d00cdfb0 2023-11-24T08:08:31.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.990+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f02cc0067f0 con 0x7f02d00cdfb0 2023-11-24T08:08:31.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.991+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d0067470 con 0x7f02d00cdfb0 2023-11-24T08:08:31.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:31.995+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f02cc021060 con 0x7f02d00cdfb0 2023-11-24T08:08:32.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.058+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02d014bd70 con 0x7f02d00cdfb0 2023-11-24T08:08:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.059+0000 7f02c67fc640 1 -- 172.21.15.106:0/1273042687 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f02cc029020 con 0x7f02d00cdfb0 2023-11-24T08:08:32.060 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:32.060 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":10,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:31.674429+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 msgr2=0x7f02d0147770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d0147770 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02cc00e8a0 tx=0x7f02cc00ed60 comp rx=0 tx=0).stop 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 shutdown_connections 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f02d015c7f0 0x7f02d01596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02d00cf450 0x7f02d014c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 --2- 172.21.15.106:0/1273042687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f02d00cdfb0 0x7f02d0147770 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 >> 172.21.15.106:0/1273042687 conn(0x7f02d006a920 msgr2=0x7f02d006ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 shutdown_connections 2023-11-24T08:08:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:32.060+0000 7f02d7e9b640 1 -- 172.21.15.106:0/1273042687 wait complete. 2023-11-24T08:08:32.078 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-24T08:08:32.078 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-24T08:08:32.436 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-11-24T08:08:32.434+0000 7fb67ce0f500 -1 Falling back to public interface 2023-11-24T08:08:32.439 INFO:tasks.ceph.osd.1.smithi106.stderr:2023-11-24T08:08:32.438+0000 7f618241c500 -1 Falling back to public interface 2023-11-24T08:08:32.444 INFO:tasks.ceph.osd.0.smithi106.stderr:2023-11-24T08:08:32.443+0000 7f1da840f500 -1 Falling back to public interface 2023-11-24T08:08:32.445 INFO:tasks.ceph.osd.2.smithi106.stderr:2023-11-24T08:08:32.443+0000 7f63f9c18500 -1 Falling back to public interface 2023-11-24T08:08:32.445 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-11-24T08:08:32.444+0000 7f2c80a27500 -1 Falling back to public interface 2023-11-24T08:08:32.449 INFO:tasks.ceph.osd.3.smithi106.stderr:2023-11-24T08:08:32.447+0000 7f07a6815500 -1 Falling back to public interface 2023-11-24T08:08:32.469 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-11-24T08:08:32.468+0000 7f87cbc16500 -1 Falling back to public interface 2023-11-24T08:08:32.477 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-11-24T08:08:32.476+0000 7f6f40cfa500 -1 Falling back to public interface 2023-11-24T08:08:32.764 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:32.762+0000 7f682b62b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T08:08:32.883 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:33.046 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:33.044+0000 7fb8cfc56040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-24T08:08:33.053 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.052+0000 7f1e09859640 1 Processor -- start 2023-11-24T08:08:33.054 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.052+0000 7f1e09859640 1 -- start start 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e09859640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e09859640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e040b4ac0 con 0x7f1e0415d720 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e09859640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e040ce1c0 con 0x7f1e040cdd40 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e09859640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040cf260 con 0x7f1e0415d340 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e09058640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1e0415d720 0x7f1e04167360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51978/0 (socket says 172.21.15.106:51978) 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e09058640 1 -- 172.21.15.106:0/1710308323 learned_addr learned my addr 172.21.15.106:0/1710308323 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:33.055 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.053+0000 7f1e08857640 1 --2- 172.21.15.106:0/1710308323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954218 0 0) 0x7f1e040b4ac0 con 0x7f1e0415d720 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1df4003520 con 0x7f1e0415d720 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461196674 0 0) 0x7f1e040ce1c0 con 0x7f1e040cdd40 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e040b4ac0 con 0x7f1e040cdd40 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e08857640 1 -- 172.21.15.106:0/1710308323 >> v1:172.21.15.156:6789/0 conn(0x7f1e040cdd40 legacy=0x7f1e040cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e08857640 1 -- 172.21.15.106:0/1710308323 >> v1:172.21.15.106:6789/0 conn(0x7f1e0415d720 legacy=0x7f1e04167360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:33.056 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.054+0000 7f1e08857640 1 -- 172.21.15.106:0/1710308323 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e040ce4a0 con 0x7f1e0415d340 2023-11-24T08:08:33.057 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.055+0000 7f1e08857640 1 --2- 172.21.15.106:0/1710308323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040d1a40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1df80097d0 tx=0x7f1df802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfdfea4e0ba2b833 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:33.057 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.055+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8031890 con 0x7f1e0415d340 2023-11-24T08:08:33.057 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df80319f0 con 0x7f1e0415d340 2023-11-24T08:08:33.057 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e037fe640 1 -- 172.21.15.106:0/1710308323 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df8031d80 con 0x7f1e0415d340 2023-11-24T08:08:33.058 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e09859640 1 -- 172.21.15.106:0/1710308323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 msgr2=0x7f1e040d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:33.058 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e09859640 1 --2- 172.21.15.106:0/1710308323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040d1a40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1df80097d0 tx=0x7f1df802b930 comp rx=0 tx=0).stop 2023-11-24T08:08:33.058 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e09859640 1 -- 172.21.15.106:0/1710308323 shutdown_connections 2023-11-24T08:08:33.058 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e09859640 1 --2- 172.21.15.106:0/1710308323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040d1a40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:33.058 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.056+0000 7f1e09859640 1 -- 172.21.15.106:0/1710308323 >> 172.21.15.106:0/1710308323 conn(0x7f1e0409c500 msgr2=0x7f1e040d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:33.059 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.057+0000 7f1e09859640 1 -- 172.21.15.106:0/1710308323 shutdown_connections 2023-11-24T08:08:33.059 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.057+0000 7f1e09859640 1 -- 172.21.15.106:0/1710308323 wait complete. 2023-11-24T08:08:33.060 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.059+0000 7f1e09859640 1 Processor -- start 2023-11-24T08:08:33.061 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.059+0000 7f1e09859640 1 -- start start 2023-11-24T08:08:33.064 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09859640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e040cdd40 0x7f1e0406cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:33.064 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09859640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040b0f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:33.064 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09859640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 0x7f1e04151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:33.064 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09859640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040c90a0 con 0x7f1e0415d720 2023-11-24T08:08:33.064 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09859640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040b14b0 con 0x7f1e040cdd40 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09859640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e040b8570 con 0x7f1e0415d340 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09058640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 0x7f1e04151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09058640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 0x7f1e04151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43052/0 (socket says 172.21.15.106:43052) 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e09058640 1 -- 172.21.15.106:0/1401907515 learned_addr learned my addr 172.21.15.106:0/1401907515 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e03fff640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040b0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.061+0000 7f1e08857640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e040cdd40 0x7f1e0406cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09058640 1 -- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 msgr2=0x7f1e040b0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09058640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040b0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09058640 1 -- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e040cdd40 msgr2=0x7f1e0406cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09058640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e040cdd40 0x7f1e0406cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09058640 1 -- 172.21.15.106:0/1401907515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df8009400 con 0x7f1e0415d720 2023-11-24T08:08:33.065 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09058640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 0x7f1e04151620 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00ca70 tx=0x7f1dfc00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc00abd0 con 0x7f1e0415d720 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dfc00ad30 con 0x7f1e0415d720 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dfc012600 con 0x7f1e0415d720 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e040b8850 con 0x7f1e0415d720 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.062+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e04151b60 con 0x7f1e0415d720 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.063+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1dfc012a60 con 0x7f1e0415d720 2023-11-24T08:08:33.066 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.063+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f1dfc012d10 con 0x7f1e0415d720 2023-11-24T08:08:33.067 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.064+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e040c91e0 con 0x7f1e0415d720 2023-11-24T08:08:33.067 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.066+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72898 (secure 0 0 0) 0x7f1e040c91e0 con 0x7f1e0415d720 2023-11-24T08:08:33.078 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:33.077+0000 7f682b62b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T08:08:33.134 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.132+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1e04068450 con 0x7f1e0415d720 2023-11-24T08:08:33.134 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.133+0000 7f1e017fa640 1 -- 172.21.15.106:0/1401907515 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f1dfc021060 con 0x7f1e0415d720 2023-11-24T08:08:33.134 INFO:teuthology.misc.health.smithi106.stdout: 2023-11-24T08:08:33.134 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":10,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:31.674429+0000","last_up_change":"0.000000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 msgr2=0x7f1e04151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 0x7f1e04151620 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1dfc00ca70 tx=0x7f1dfc00cf30 comp rx=0 tx=0).stop 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 shutdown_connections 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1e0415d720 0x7f1e04151620 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1e0415d340 0x7f1e040b0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 --2- 172.21.15.106:0/1401907515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1e040cdd40 0x7f1e0406cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 >> 172.21.15.106:0/1401907515 conn(0x7f1e0409c500 msgr2=0x7f1e040d0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 shutdown_connections 2023-11-24T08:08:33.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:33.134+0000 7f1e09859640 1 -- 172.21.15.106:0/1401907515 wait complete. 2023-11-24T08:08:33.140 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:33.138+0000 7f682b62b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T08:08:33.148 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-24T08:08:33.256 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:33.254+0000 7f682b62b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T08:08:33.365 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:33.363+0000 7fb8cfc56040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-24T08:08:33.425 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:33.423+0000 7fb8cfc56040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-24T08:08:33.539 INFO:tasks.ceph.mgr.x.smithi156.stderr:2023-11-24T08:08:33.537+0000 7fb8cfc56040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-24T08:08:34.300 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:34.298+0000 7f67cc283640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-24T08:08:34.301 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:08:34.299+0000 7f67cc283640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-24T08:08:35.994 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-11-24T08:08:35.992+0000 7f2c80a27500 -1 osd.6 0 log_to_monitors true 2023-11-24T08:08:36.034 INFO:tasks.ceph.osd.3.smithi106.stderr:2023-11-24T08:08:36.032+0000 7f07a6815500 -1 osd.3 0 log_to_monitors true 2023-11-24T08:08:36.038 INFO:tasks.ceph.osd.0.smithi106.stderr:2023-11-24T08:08:36.037+0000 7f1da840f500 -1 osd.0 0 log_to_monitors true 2023-11-24T08:08:36.040 INFO:tasks.ceph.osd.2.smithi106.stderr:2023-11-24T08:08:36.039+0000 7f63f9c18500 -1 osd.2 0 log_to_monitors true 2023-11-24T08:08:36.041 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-11-24T08:08:36.040+0000 7f6f40cfa500 -1 osd.7 0 log_to_monitors true 2023-11-24T08:08:36.214 INFO:tasks.ceph.osd.1.smithi106.stderr:2023-11-24T08:08:36.212+0000 7f618241c500 -1 osd.1 0 log_to_monitors true 2023-11-24T08:08:36.290 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-11-24T08:08:36.289+0000 7fb67ce0f500 -1 osd.4 0 log_to_monitors true 2023-11-24T08:08:36.316 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-11-24T08:08:36.314+0000 7f87cbc16500 -1 osd.5 0 log_to_monitors true 2023-11-24T08:08:38.810 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-11-24T08:08:38.808+0000 7fb677eab640 -1 osd.4 0 waiting for initial osdmap 2023-11-24T08:08:38.819 INFO:tasks.ceph.osd.4.smithi156.stderr:2023-11-24T08:08:38.817+0000 7fb6704e4640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:38.839 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-11-24T08:08:38.837+0000 7f6f3c467640 -1 osd.7 0 waiting for initial osdmap 2023-11-24T08:08:38.853 INFO:tasks.ceph.osd.7.smithi156.stderr:2023-11-24T08:08:38.851+0000 7f6f34aa0640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:38.855 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-11-24T08:08:38.854+0000 7f2c7bbf0640 -1 osd.6 0 waiting for initial osdmap 2023-11-24T08:08:38.860 INFO:tasks.ceph.osd.6.smithi156.stderr:2023-11-24T08:08:38.858+0000 7f2c74229640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:39.589 INFO:tasks.ceph.osd.2.smithi106.stderr:2023-11-24T08:08:39.588+0000 7f63f4de2640 -1 osd.2 0 waiting for initial osdmap 2023-11-24T08:08:39.597 INFO:tasks.ceph.osd.2.smithi106.stderr:2023-11-24T08:08:39.596+0000 7f63ed41b640 -1 osd.2 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:39.613 INFO:tasks.ceph.osd.1.smithi106.stderr:2023-11-24T08:08:39.611+0000 7f617d5c9640 -1 osd.1 0 waiting for initial osdmap 2023-11-24T08:08:39.620 INFO:tasks.ceph.osd.1.smithi106.stderr:2023-11-24T08:08:39.618+0000 7f6175c02640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:39.627 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-11-24T08:08:39.626+0000 7f87c6d5b640 -1 osd.5 0 waiting for initial osdmap 2023-11-24T08:08:39.636 INFO:tasks.ceph.osd.5.smithi156.stderr:2023-11-24T08:08:39.634+0000 7f87bf394640 -1 osd.5 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:39.653 INFO:tasks.ceph.osd.0.smithi106.stderr:2023-11-24T08:08:39.652+0000 7f1da35aa640 -1 osd.0 0 waiting for initial osdmap 2023-11-24T08:08:39.665 INFO:tasks.ceph.osd.0.smithi106.stderr:2023-11-24T08:08:39.664+0000 7f1d9bbe3640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:39.715 INFO:tasks.ceph.osd.3.smithi106.stderr:2023-11-24T08:08:39.713+0000 7f07a1937640 -1 osd.3 0 waiting for initial osdmap 2023-11-24T08:08:39.721 INFO:tasks.ceph.osd.3.smithi106.stderr:2023-11-24T08:08:39.720+0000 7f0799f70640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-24T08:08:39.955 DEBUG:teuthology.orchestra.run.smithi106:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:40.135 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.133+0000 7f50b0def640 1 Processor -- start 2023-11-24T08:08:40.136 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.134+0000 7f50b0def640 1 -- start start 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.135+0000 7f50b0def640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f50ac140090 0x7f50ac0b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.135+0000 7f50b0def640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac15e310 0x7f50ac0ac840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.135+0000 7f50b0def640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50ac0ce970 con 0x7f50ac0b2400 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.135+0000 7f50b0def640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50ac0b2890 con 0x7f50ac140090 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.135+0000 7f50b0def640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac0b4a40 con 0x7f50ac15e310 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50ab7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f50ac140090 0x7f50ac0b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.137 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50a3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f50ac0b2400 0x7f50ac15dc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58172/0 (socket says 172.21.15.106:58172) 2023-11-24T08:08:40.138 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50a3fff640 1 -- 172.21.15.106:0/3028594061 learned_addr learned my addr 172.21.15.106:0/3028594061 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:40.138 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50ab7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f50ac140090 0x7f50ac0b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38218/0 (socket says 172.21.15.106:38218) 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50abfff640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac15e310 0x7f50ac0ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50ab7fe640 1 -- 172.21.15.106:0/3028594061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac15e310 msgr2=0x7f50ac0ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50ab7fe640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac15e310 0x7f50ac0ac840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50ab7fe640 1 -- 172.21.15.106:0/3028594061 >> v1:172.21.15.106:6789/0 conn(0x7f50ac0b2400 legacy=0x7f50ac15dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50ab7fe640 1 -- 172.21.15.106:0/3028594061 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0b4110 con 0x7f50ac140090 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.136+0000 7f50abfff640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac15e310 0x7f50ac0ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.137+0000 7f50ab7fe640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac140090 0x7f50ac0b1ec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f509c0097a0 tx=0x7f509c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f363d4caf0c32f79 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.140 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.137+0000 7f50aaffd640 1 -- 172.21.15.106:0/3028594061 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c02d040 con 0x7f50ac140090 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.137+0000 7f50aaffd640 1 -- 172.21.15.106:0/3028594061 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f509c031810 con 0x7f50ac140090 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.137+0000 7f50aaffd640 1 -- 172.21.15.106:0/3028594061 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f509c031ba0 con 0x7f50ac140090 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 -- 172.21.15.106:0/3028594061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac140090 msgr2=0x7f50ac0b1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac140090 0x7f50ac0b1ec0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f509c0097a0 tx=0x7f509c02b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 -- 172.21.15.106:0/3028594061 shutdown_connections 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac15e310 0x7f50ac0ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 --2- 172.21.15.106:0/3028594061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac140090 0x7f50ac0b1ec0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 -- 172.21.15.106:0/3028594061 >> 172.21.15.106:0/3028594061 conn(0x7f50ac06c460 msgr2=0x7f50ac06c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 -- 172.21.15.106:0/3028594061 shutdown_connections 2023-11-24T08:08:40.141 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.138+0000 7f50b0def640 1 -- 172.21.15.106:0/3028594061 wait complete. 2023-11-24T08:08:40.144 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.140+0000 7f50b0def640 1 Processor -- start 2023-11-24T08:08:40.144 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.140+0000 7f50b0def640 1 -- start start 2023-11-24T08:08:40.144 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50b0def640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac0b2400 0x7f50ac1f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.144 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50b0def640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50ac140090 0x7f50ac0ad500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.144 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50b0def640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 0x7f50ac13a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.144 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50b0def640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac1580a0 con 0x7f50ac15e310 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50b0def640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac158380 con 0x7f50ac140090 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50b0def640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac158660 con 0x7f50ac0b2400 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 0x7f50ac13a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.141+0000 7f50abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 0x7f50ac13a1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38220/0 (socket says 172.21.15.106:38220) 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 -- 172.21.15.106:0/1607731825 learned_addr learned my addr 172.21.15.106:0/1607731825 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50ab7fe640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac0b2400 0x7f50ac1f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50a3fff640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50ac140090 0x7f50ac0ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 -- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac0b2400 msgr2=0x7f50ac1f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac0b2400 0x7f50ac1f6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.145 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 -- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50ac140090 msgr2=0x7f50ac0ad500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50ac140090 0x7f50ac0ad500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 -- 172.21.15.106:0/1607731825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c009400 con 0x7f50ac15e310 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50abfff640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 0x7f50ac13a1c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f50a400cd50 tx=0x7f50a400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400f680 con 0x7f50ac15e310 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50a400f800 con 0x7f50ac15e310 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50a400fbb0 con 0x7f50ac15e310 2023-11-24T08:08:40.146 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.142+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac158940 con 0x7f50ac15e310 2023-11-24T08:08:40.147 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.143+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac15b010 con 0x7f50ac15e310 2023-11-24T08:08:40.147 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.144+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac0b4eb0 con 0x7f50ac15e310 2023-11-24T08:08:40.147 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.146+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f50a400b040 con 0x7f50ac15e310 2023-11-24T08:08:40.148 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.147+0000 7f50a8ff9640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f508c06adb0 0x7f508c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.148 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.147+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f50a4089a50 con 0x7f50ac15e310 2023-11-24T08:08:40.149 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.147+0000 7f50ab7fe640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f508c06adb0 0x7f508c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.149 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.148+0000 7f50ab7fe640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f508c06adb0 0x7f508c06d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f509c036000 tx=0x7f509c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.153 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.152+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50ac0b4eb0 con 0x7f50ac15e310 2023-11-24T08:08:40.329 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.327+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50ac0b4eb0 con 0x7f50ac15e310 2023-11-24T08:08:40.330 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.329+0000 7f50a8ff9640 1 -- 172.21.15.106:0/1607731825 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10956 (secure 0 0 0) 0x7f50ac0b4eb0 con 0x7f50ac15e310 2023-11-24T08:08:40.330 INFO:teuthology.misc.health.smithi106.stdout: 2023-11-24T08:08:40.330 INFO:teuthology.misc.health.smithi106.stdout:{"epoch":14,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:39.815404+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f508c06adb0 msgr2=0x7f508c06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f508c06adb0 0x7f508c06d270 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f509c036000 tx=0x7f509c03c040 comp rx=0 tx=0).stop 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 msgr2=0x7f50ac13a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 0x7f50ac13a1c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f50a400cd50 tx=0x7f50a400a650 comp rx=0 tx=0).stop 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 shutdown_connections 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50ac15e310 0x7f50ac13a1c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.333 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50ac140090 0x7f50ac0ad500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.334 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f508c06adb0 0x7f508c06d270 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.334 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 --2- 172.21.15.106:0/1607731825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50ac0b2400 0x7f50ac1f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.334 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 >> 172.21.15.106:0/1607731825 conn(0x7f50ac06c460 msgr2=0x7f50ac0703a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:40.334 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 shutdown_connections 2023-11-24T08:08:40.334 INFO:teuthology.misc.health.smithi106.stderr:2023-11-24T08:08:40.331+0000 7f50b0def640 1 -- 172.21.15.106:0/1607731825 wait complete. 2023-11-24T08:08:40.348 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-24T08:08:40.348 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-24T08:08:40.348 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:08:40.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:08:40.355 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:08:40.355 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:08:40.355 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:08:40.360 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:08:40.360 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:08:40.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:08:40.369 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:08:40.369 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:08:40.369 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:08:40.379 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:08:40.379 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-24T08:08:40.379 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:40.500 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:08:40.501 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:08:40.530 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-11-24T08:08:40.541 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:08:40.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.554+0000 7f9c94e94640 1 Processor -- start 2023-11-24T08:08:40.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.554+0000 7f9c94e94640 1 -- start start 2023-11-24T08:08:40.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c94e94640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c9015c7f0 0x7f9c9006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c94e94640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9c900cf450 0x7f9c900cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c94e94640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c900cece0 con 0x7f9c900cdfb0 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c94e94640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c90156010 con 0x7f9c900cf450 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c94e94640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9006ce30 con 0x7f9c9015c7f0 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c8dd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9c900cf450 0x7f9c900cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c8ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9c900cdfb0 0x7f9c900ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58186/0 (socket says 172.21.15.106:58186) 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.555+0000 7f9c8ed9d640 1 -- 172.21.15.106:0/1149326227 learned_addr learned my addr 172.21.15.106:0/1149326227 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.556+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/1149326227 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c9015c7f0 msgr2=0x7f9c9006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.556+0000 7f9c8e59c640 1 --2- 172.21.15.106:0/1149326227 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c9015c7f0 0x7f9c9006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.556+0000 7f9c8dd9b640 1 --2- 172.21.15.106:0/1149326227 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c9015c7f0 0x7f9c9006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.556+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/1149326227 >> v1:172.21.15.106:6789/0 conn(0x7f9c900cdfb0 legacy=0x7f9c900ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.556+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/1149326227 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9006d040 con 0x7f9c900cf450 2023-11-24T08:08:40.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.556+0000 7f9c8d59a640 1 -- 172.21.15.106:0/1149326227 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582690755 0 0) 0x7f9c900cece0 con 0x7f9c900cdfb0 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.557+0000 7f9c8dd9b640 1 --2- 172.21.15.106:0/1149326227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c900cf8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097a0 tx=0x7f9c8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22608740d51dc0b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.557+0000 7f9c8d59a640 1 -- 172.21.15.106:0/1149326227 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84031830 con 0x7f9c900cf450 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.557+0000 7f9c8d59a640 1 -- 172.21.15.106:0/1149326227 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c84031990 con 0x7f9c900cf450 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.557+0000 7f9c8d59a640 1 -- 172.21.15.106:0/1149326227 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c84031d20 con 0x7f9c900cf450 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 -- 172.21.15.106:0/1149326227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 msgr2=0x7f9c900cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 --2- 172.21.15.106:0/1149326227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c900cf8d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9c840097a0 tx=0x7f9c8402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 -- 172.21.15.106:0/1149326227 shutdown_connections 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 --2- 172.21.15.106:0/1149326227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c900cf8d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 --2- 172.21.15.106:0/1149326227 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c9015c7f0 0x7f9c9006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 -- 172.21.15.106:0/1149326227 >> 172.21.15.106:0/1149326227 conn(0x7f9c9006a920 msgr2=0x7f9c9006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:40.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 -- 172.21.15.106:0/1149326227 shutdown_connections 2023-11-24T08:08:40.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.558+0000 7f9c94e94640 1 -- 172.21.15.106:0/1149326227 wait complete. 2023-11-24T08:08:40.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.559+0000 7f9c94e94640 1 Processor -- start 2023-11-24T08:08:40.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.560+0000 7f9c94e94640 1 -- start start 2023-11-24T08:08:40.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.560+0000 7f9c94e94640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c900cdfb0 0x7f9c90149500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.560+0000 7f9c94e94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c9015ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c94e94640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c9015c7f0 0x7f9c9015e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c94e94640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c90149a40 con 0x7f9c900cf450 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c94e94640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901f73f0 con 0x7f9c9015c7f0 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c94e94640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c901f67e0 con 0x7f9c900cdfb0 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c9015ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c9015ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38262/0 (socket says 172.21.15.106:38262) 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/3932164581 learned_addr learned my addr 172.21.15.106:0/3932164581 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8e59c640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c900cdfb0 0x7f9c90149500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c900cdfb0 msgr2=0x7f9c90149500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8ed9d640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c9015c7f0 0x7f9c9015e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c900cdfb0 0x7f9c90149500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c9015c7f0 msgr2=0x7f9c9015e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c9015c7f0 0x7f9c9015e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 -- 172.21.15.106:0/3932164581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c84009400 con 0x7f9c900cf450 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8e59c640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c900cdfb0 0x7f9c90149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8ed9d640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c9015c7f0 0x7f9c9015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.561+0000 7f9c8dd9b640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c9015ddd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009890 tx=0x7f9c840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.562+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8403baa0 con 0x7f9c900cf450 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.562+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8403bc20 con 0x7f9c900cf450 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.562+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c840433f0 con 0x7f9c900cf450 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.562+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c901f6ac0 con 0x7f9c900cf450 2023-11-24T08:08:40.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.562+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c901f7080 con 0x7f9c900cf450 2023-11-24T08:08:40.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.563+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c901f7210 con 0x7f9c900cf450 2023-11-24T08:08:40.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.564+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f9c84030380 con 0x7f9c900cf450 2023-11-24T08:08:40.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.564+0000 7f9c7effd640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9c7006adb0 0x7f9c7006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.565+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f9c840aac90 con 0x7f9c900cf450 2023-11-24T08:08:40.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.565+0000 7f9c8e59c640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9c7006adb0 0x7f9c7006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.565+0000 7f9c8e59c640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9c7006adb0 0x7f9c7006d270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000b820 tx=0x7f9c80008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.569+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9c840762e0 con 0x7f9c900cf450 2023-11-24T08:08:40.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.748+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c90067470 con 0x7f9c900cf450 2023-11-24T08:08:40.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.749+0000 7f9c7effd640 1 -- 172.21.15.106:0/3932164581 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+10956 (secure 0 0 0) 0x7f9c84079900 con 0x7f9c900cf450 2023-11-24T08:08:40.751 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:40.751 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":14,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:39.815404+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:40.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.751+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9c7006adb0 msgr2=0x7f9c7006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.751+0000 7f9c94e94640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9c7006adb0 0x7f9c7006d270 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c8000b820 tx=0x7f9c80008040 comp rx=0 tx=0).stop 2023-11-24T08:08:40.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.751+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 msgr2=0x7f9c9015ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.751+0000 7f9c94e94640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c9015ddd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9c84009890 tx=0x7f9c840046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 shutdown_connections 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9c9015c7f0 0x7f9c9015e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9c900cf450 0x7f9c9015ddd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9c7006adb0 0x7f9c7006d270 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 --2- 172.21.15.106:0/3932164581 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9c900cdfb0 0x7f9c90149500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 >> 172.21.15.106:0/3932164581 conn(0x7f9c9006a920 msgr2=0x7f9c900ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 shutdown_connections 2023-11-24T08:08:40.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.752+0000 7f9c94e94640 1 -- 172.21.15.106:0/3932164581 wait complete. 2023-11-24T08:08:40.770 INFO:tasks.cephfs.filesystem.ceph_manager:[] 2023-11-24T08:08:40.770 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:40.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.971+0000 7f7038fa9640 1 Processor -- start 2023-11-24T08:08:40.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.972+0000 7f7038fa9640 1 -- start start 2023-11-24T08:08:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f7038fa9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f703415c7f0 0x7f703406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f7038fa9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70340cf450 0x7f70340cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f7038fa9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70340cece0 con 0x7f70340cdfb0 2023-11-24T08:08:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f7038fa9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7034156010 con 0x7f703415c7f0 2023-11-24T08:08:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f7038fa9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703406ce30 con 0x7f70340cf450 2023-11-24T08:08:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f703259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f703415c7f0 0x7f703406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f703259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f703415c7f0 0x7f703406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38268/0 (socket says 172.21.15.106:38268) 2023-11-24T08:08:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.973+0000 7f703259c640 1 -- 172.21.15.106:0/1324184948 learned_addr learned my addr 172.21.15.106:0/1324184948 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.974+0000 7f703259c640 1 -- 172.21.15.106:0/1324184948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70340cf450 msgr2=0x7f70340cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.974+0000 7f703259c640 1 --2- 172.21.15.106:0/1324184948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70340cf450 0x7f70340cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.974+0000 7f703259c640 1 -- 172.21.15.106:0/1324184948 >> v1:172.21.15.156:6789/0 conn(0x7f70340cdfb0 legacy=0x7f70340ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:40.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.974+0000 7f703259c640 1 -- 172.21.15.106:0/1324184948 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f703406d040 con 0x7f703415c7f0 2023-11-24T08:08:40.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.975+0000 7f703259c640 1 --2- 172.21.15.106:0/1324184948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703415c7f0 0x7f703406db20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7024009870 tx=0x7f702402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e241ce6f13b52d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.975+0000 7f7031d9b640 1 -- 172.21.15.106:0/1324184948 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024031640 con 0x7f703415c7f0 2023-11-24T08:08:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.975+0000 7f7031d9b640 1 -- 172.21.15.106:0/1324184948 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70240317a0 con 0x7f703415c7f0 2023-11-24T08:08:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.975+0000 7f7031d9b640 1 -- 172.21.15.106:0/1324184948 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7024031b30 con 0x7f703415c7f0 2023-11-24T08:08:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.975+0000 7f7038fa9640 1 -- 172.21.15.106:0/1324184948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703415c7f0 msgr2=0x7f703406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.975+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1324184948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703415c7f0 0x7f703406db20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7024009870 tx=0x7f702402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.976+0000 7f7038fa9640 1 -- 172.21.15.106:0/1324184948 shutdown_connections 2023-11-24T08:08:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.976+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1324184948 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70340cf450 0x7f70340cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.976+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1324184948 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f703415c7f0 0x7f703406db20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.976+0000 7f7038fa9640 1 -- 172.21.15.106:0/1324184948 >> 172.21.15.106:0/1324184948 conn(0x7f703406a920 msgr2=0x7f703406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.976+0000 7f7038fa9640 1 -- 172.21.15.106:0/1324184948 shutdown_connections 2023-11-24T08:08:40.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.976+0000 7f7038fa9640 1 -- 172.21.15.106:0/1324184948 wait complete. 2023-11-24T08:08:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.977+0000 7f7038fa9640 1 Processor -- start 2023-11-24T08:08:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.977+0000 7f7038fa9640 1 -- start start 2023-11-24T08:08:40.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.978+0000 7f7038fa9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70340cdfb0 0x7f7034155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.978+0000 7f7038fa9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 0x7f703414ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.978+0000 7f7038fa9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f703415c7f0 0x7f70341e4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.978+0000 7f7038fa9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034155600 con 0x7f70340cf450 2023-11-24T08:08:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.978+0000 7f7038fa9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7034153df0 con 0x7f70340cdfb0 2023-11-24T08:08:40.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.978+0000 7f7038fa9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f703414f750 con 0x7f703415c7f0 2023-11-24T08:08:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 0x7f703414ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 0x7f703414ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38270/0 (socket says 172.21.15.106:38270) 2023-11-24T08:08:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 -- 172.21.15.106:0/1601424498 learned_addr learned my addr 172.21.15.106:0/1601424498 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f7032d9d640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f703415c7f0 0x7f70341e4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f703259c640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70340cdfb0 0x7f7034155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 -- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f703415c7f0 msgr2=0x7f70341e4ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f703415c7f0 0x7f70341e4ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 -- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70340cdfb0 msgr2=0x7f7034155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70340cdfb0 0x7f7034155020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 -- 172.21.15.106:0/1601424498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70240094d0 con 0x7f70340cf450 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f7032d9d640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f703415c7f0 0x7f70341e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f703259c640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70340cdfb0 0x7f7034155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:40.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.979+0000 7f702bfff640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 0x7f703414ee50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f701c00b4a0 tx=0x7f701c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.980+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c010610 con 0x7f70340cf450 2023-11-24T08:08:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.980+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f701c010770 con 0x7f70340cf450 2023-11-24T08:08:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.980+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f701c010b20 con 0x7f70340cf450 2023-11-24T08:08:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.980+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70341534c0 con 0x7f70340cf450 2023-11-24T08:08:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.980+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7034153a80 con 0x7f70340cf450 2023-11-24T08:08:40.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.981+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7034067470 con 0x7f70340cf450 2023-11-24T08:08:40.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.982+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f701c010c80 con 0x7f70340cf450 2023-11-24T08:08:40.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.982+0000 7f702affd640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f701406adb0 0x7f701406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:40.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.983+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f701c088c40 con 0x7f70340cf450 2023-11-24T08:08:40.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.983+0000 7f703259c640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f701406adb0 0x7f701406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:40.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.983+0000 7f703259c640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f701406adb0 0x7f701406d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7024036000 tx=0x7f702403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:40.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:40.987+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f701c054290 con 0x7f70340cf450 2023-11-24T08:08:41.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.196+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70340cd730 con 0x7f70340cf450 2023-11-24T08:08:41.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.197+0000 7f702affd640 1 -- 172.21.15.106:0/1601424498 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1327 (secure 0 0 0) 0x7f701c01e540 con 0x7f70340cf450 2023-11-24T08:08:41.199 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:41.199 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-11-24T08:08:41.200 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 2 2023-11-24T08:08:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.199+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f701406adb0 msgr2=0x7f701406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:41.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.199+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f701406adb0 0x7f701406d270 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f7024036000 tx=0x7f702403c040 comp rx=0 tx=0).stop 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.199+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 msgr2=0x7f703414ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.199+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 0x7f703414ee50 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f701c00b4a0 tx=0x7f701c00b960 comp rx=0 tx=0).stop 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.199+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 shutdown_connections 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f701406adb0 0x7f701406d270 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f703415c7f0 0x7f70341e4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70340cf450 0x7f703414ee50 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:41.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 --2- 172.21.15.106:0/1601424498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f70340cdfb0 0x7f7034155020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 >> 172.21.15.106:0/1601424498 conn(0x7f703406a920 msgr2=0x7f70340afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 shutdown_connections 2023-11-24T08:08:41.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:41.200+0000 7f7038fa9640 1 -- 172.21.15.106:0/1601424498 wait complete. 2023-11-24T08:08:43.218 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:43.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.419+0000 7f6d68802640 1 Processor -- start 2023-11-24T08:08:43.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.419+0000 7f6d68802640 1 -- start start 2023-11-24T08:08:43.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.420+0000 7f6d68802640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d6406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.420+0000 7f6d68802640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d640ce010 0x7f6d640ce460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.420+0000 7f6d68802640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6406c1f0 con 0x7f6d640d31a0 2023-11-24T08:08:43.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d68802640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d640d1070 con 0x7f6d640ce010 2023-11-24T08:08:43.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d68802640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640cedc0 con 0x7f6d641609e0 2023-11-24T08:08:43.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d640ce010 0x7f6d640ce460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6d640ce010 0x7f6d640ce460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38284/0 (socket says 172.21.15.106:38284) 2023-11-24T08:08:43.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 -- 172.21.15.106:0/1037687914 learned_addr learned my addr 172.21.15.106:0/1037687914 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:43.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d6259c640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d6406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 -- 172.21.15.106:0/1037687914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 msgr2=0x7f6d6406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d6406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 -- 172.21.15.106:0/1037687914 >> v1:172.21.15.156:6789/0 conn(0x7f6d640d31a0 legacy=0x7f6d640d3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d62d9d640 1 -- 172.21.15.106:0/1037687914 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d640cf0a0 con 0x7f6d640ce010 2023-11-24T08:08:43.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.421+0000 7f6d6259c640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d6406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:43.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.423+0000 7f6d62d9d640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640ce010 0x7f6d640ce460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009e50 tx=0x7f6d5c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f543bcf28f488350 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.423+0000 7f6d61d9b640 1 -- 172.21.15.106:0/1037687914 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c02cb00 con 0x7f6d640ce010 2023-11-24T08:08:43.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.423+0000 7f6d61d9b640 1 -- 172.21.15.106:0/1037687914 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c02cc60 con 0x7f6d640ce010 2023-11-24T08:08:43.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.423+0000 7f6d61d9b640 1 -- 172.21.15.106:0/1037687914 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c0343f0 con 0x7f6d640ce010 2023-11-24T08:08:43.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 -- 172.21.15.106:0/1037687914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640ce010 msgr2=0x7f6d640ce460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640ce010 0x7f6d640ce460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009e50 tx=0x7f6d5c02bf60 comp rx=0 tx=0).stop 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 -- 172.21.15.106:0/1037687914 shutdown_connections 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640ce010 0x7f6d640ce460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 --2- 172.21.15.106:0/1037687914 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d6406bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 -- 172.21.15.106:0/1037687914 >> 172.21.15.106:0/1037687914 conn(0x7f6d6408af60 msgr2=0x7f6d6409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 -- 172.21.15.106:0/1037687914 shutdown_connections 2023-11-24T08:08:43.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.424+0000 7f6d68802640 1 -- 172.21.15.106:0/1037687914 wait complete. 2023-11-24T08:08:43.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.426+0000 7f6d68802640 1 Processor -- start 2023-11-24T08:08:43.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.426+0000 7f6d68802640 1 -- start start 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d68802640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d640ce010 0x7f6d640b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d68802640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 0x7f6d640b8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d68802640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d64151750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d68802640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d640b9260 con 0x7f6d640d31a0 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d68802640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6415acc0 con 0x7f6d640ce010 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d68802640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d641f6af0 con 0x7f6d641609e0 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 0x7f6d640b8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d5bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 0x7f6d640b8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38288/0 (socket says 172.21.15.106:38288) 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d5bfff640 1 -- 172.21.15.106:0/2805955137 learned_addr learned my addr 172.21.15.106:0/2805955137 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d6259c640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d640ce010 0x7f6d640b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d62d9d640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d64151750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d5bfff640 1 -- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 msgr2=0x7f6d64151750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.427+0000 7f6d5bfff640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d64151750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5bfff640 1 -- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d640ce010 msgr2=0x7f6d640b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5bfff640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d640ce010 0x7f6d640b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5bfff640 1 -- 172.21.15.106:0/2805955137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009b30 con 0x7f6d640d31a0 2023-11-24T08:08:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d62d9d640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d64151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d6259c640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d640ce010 0x7f6d640b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5bfff640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 0x7f6d640b8d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c4d0 tx=0x7f6d4c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0052d0 con 0x7f6d640d31a0 2023-11-24T08:08:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d4c005430 con 0x7f6d640d31a0 2023-11-24T08:08:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c012850 con 0x7f6d640d31a0 2023-11-24T08:08:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d641f6dd0 con 0x7f6d640d31a0 2023-11-24T08:08:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.428+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d641f7390 con 0x7f6d640d31a0 2023-11-24T08:08:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.429+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d64067470 con 0x7f6d640d31a0 2023-11-24T08:08:43.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.430+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f6d4c0129b0 con 0x7f6d640d31a0 2023-11-24T08:08:43.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.431+0000 7f6d5affd640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6d4406adb0 0x7f6d4406d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.431+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6d4c088cd0 con 0x7f6d640d31a0 2023-11-24T08:08:43.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.432+0000 7f6d6259c640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6d4406adb0 0x7f6d4406d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.432+0000 7f6d6259c640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6d4406adb0 0x7f6d4406d270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d640d3ad0 tx=0x7f6d54008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.437+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6d4c0541b0 con 0x7f6d640d31a0 2023-11-24T08:08:43.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.673+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d6406b750 con 0x7f6d640d31a0 2023-11-24T08:08:43.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.674+0000 7f6d5affd640 1 -- 172.21.15.106:0/2805955137 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3703 (secure 0 0 0) 0x7f6d4c0577d0 con 0x7f6d640d31a0 2023-11-24T08:08:43.677 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:43.677 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4276,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-11-24T08:08:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 3 2023-11-24T08:08:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6d4406adb0 msgr2=0x7f6d4406d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6d4406adb0 0x7f6d4406d270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d640d3ad0 tx=0x7f6d54008040 comp rx=0 tx=0).stop 2023-11-24T08:08:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 msgr2=0x7f6d640b8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 0x7f6d640b8d20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c4d0 tx=0x7f6d4c00c990 comp rx=0 tx=0).stop 2023-11-24T08:08:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 shutdown_connections 2023-11-24T08:08:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6d4406adb0 0x7f6d4406d270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6d641609e0 0x7f6d64151750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6d640d31a0 0x7f6d640b8d20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 --2- 172.21.15.106:0/2805955137 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d640ce010 0x7f6d640b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.677+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 >> 172.21.15.106:0/2805955137 conn(0x7f6d6408af60 msgr2=0x7f6d640b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:43.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.678+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 shutdown_connections 2023-11-24T08:08:43.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.678+0000 7f6d68802640 1 -- 172.21.15.106:0/2805955137 wait complete. 2023-11-24T08:08:43.696 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:43.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.885+0000 7f1b60dc5640 1 Processor -- start 2023-11-24T08:08:43.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.885+0000 7f1b60dc5640 1 -- start start 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.886+0000 7f1b60dc5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 0x7f1b5c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.886+0000 7f1b60dc5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1b5c0cf450 0x7f1b5c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.886+0000 7f1b60dc5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b5c0cece0 con 0x7f1b5c0cdfb0 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.886+0000 7f1b60dc5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b5c156010 con 0x7f1b5c0cf450 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.886+0000 7f1b60dc5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c06ce30 con 0x7f1b5c15c7f0 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b5ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1b5c0cdfb0 0x7f1b5c0ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58194/0 (socket says 172.21.15.106:58194) 2023-11-24T08:08:43.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b5ad9d640 1 -- 172.21.15.106:0/1295667745 learned_addr learned my addr 172.21.15.106:0/1295667745 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:43.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b53fff640 1 --2- 172.21.15.106:0/1295667745 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c0cf450 0x7f1b5c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 0x7f1b5c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2054125565 0 0) 0x7f1b5c0cece0 con 0x7f1b5c0cdfb0 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b48003520 con 0x7f1b5c0cdfb0 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2182387372 0 0) 0x7f1b48003520 con 0x7f1b5c0cdfb0 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 msgr2=0x7f1b5c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 0x7f1b5c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c0cf450 msgr2=0x7f1b5c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 --2- 172.21.15.106:0/1295667745 >> v2:172.21.15.156:3300/0 conn(0x7f1b5c0cf450 0x7f1b5c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.887+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c06d040 con 0x7f1b5c0cdfb0 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f1b540019f0 con 0x7f1b5c0cdfb0 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 >> v1:172.21.15.106:6789/0 conn(0x7f1b5c0cdfb0 legacy=0x7f1b5c0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b48003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 0x7f1b480047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b48004d00 0x7f1b48005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c156010 con 0x7f1b48003970 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c06ce30 con 0x7f1b48004d00 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4800b980 con 0x7f1b5c15c7f0 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b48003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b53fff640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 0x7f1b480047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5a59c640 1 -- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 msgr2=0x7f1b480047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c15c7f0 0x7f1b480047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5a59c640 1 -- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b48004d00 msgr2=0x7f1b48005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5ad9d640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b48004d00 0x7f1b48005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b48004d00 0x7f1b48005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.888+0000 7f1b5a59c640 1 -- 172.21.15.106:0/1295667745 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b48003520 con 0x7f1b48003970 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.889+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b48003d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009ac0 tx=0x7f1b480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9a58663ddb7a34 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c02c6b0 con 0x7f1b48003970 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c02c810 con 0x7f1b48003970 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b59d9b640 1 -- 172.21.15.106:0/1295667745 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c02cba0 con 0x7f1b48003970 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b60dc5640 1 -- 172.21.15.106:0/1295667745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 msgr2=0x7f1b48003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/1295667745 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b48003d90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c009ac0 tx=0x7f1b480057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b5ad9d640 1 -- 172.21.15.106:0/1295667745 reap_dead start 2023-11-24T08:08:43.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b60dc5640 1 -- 172.21.15.106:0/1295667745 shutdown_connections 2023-11-24T08:08:43.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b60dc5640 1 -- 172.21.15.106:0/1295667745 >> 172.21.15.106:0/1295667745 conn(0x7f1b5c06a920 msgr2=0x7f1b5c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:43.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.890+0000 7f1b60dc5640 1 -- 172.21.15.106:0/1295667745 shutdown_connections 2023-11-24T08:08:43.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.891+0000 7f1b60dc5640 1 -- 172.21.15.106:0/1295667745 wait complete. 2023-11-24T08:08:43.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.892+0000 7f1b60dc5640 1 Processor -- start 2023-11-24T08:08:43.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.892+0000 7f1b60dc5640 1 -- start start 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.893+0000 7f1b60dc5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b5c0b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.893+0000 7f1b60dc5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c0b2af0 0x7f1b5c154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.893+0000 7f1b60dc5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0ad5b0 0x7f1b5c0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.893+0000 7f1b60dc5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c0adf70 con 0x7f1b48003970 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.893+0000 7f1b60dc5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c1547a0 con 0x7f1b5c0b2af0 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.893+0000 7f1b60dc5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5c156280 con 0x7f1b5c0ad5b0 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b5c0b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b5c0b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38310/0 (socket says 172.21.15.106:38310) 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 -- 172.21.15.106:0/874512274 learned_addr learned my addr 172.21.15.106:0/874512274 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:43.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b53fff640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c0b2af0 0x7f1b5c154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 -- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0ad5b0 msgr2=0x7f1b5c0ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5ad9d640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0ad5b0 0x7f1b5c0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0ad5b0 0x7f1b5c0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 -- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c0b2af0 msgr2=0x7f1b5c154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c0b2af0 0x7f1b5c154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 -- 172.21.15.106:0/874512274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b5c156560 con 0x7f1b48003970 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b53fff640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c0b2af0 0x7f1b5c154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.894+0000 7f1b5a59c640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b5c0b3d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c02c250 tx=0x7f1b4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.895+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c003810 con 0x7f1b48003970 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.895+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b4c003970 con 0x7f1b48003970 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.895+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b4c003ce0 con 0x7f1b48003970 2023-11-24T08:08:43.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.895+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5c156840 con 0x7f1b48003970 2023-11-24T08:08:43.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.895+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5c156e00 con 0x7f1b48003970 2023-11-24T08:08:43.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.896+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b5c06d8b0 con 0x7f1b48003970 2023-11-24T08:08:43.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.897+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f1b4c047450 con 0x7f1b48003970 2023-11-24T08:08:43.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.898+0000 7f1b52ffd640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b3c06ade0 0x7f1b3c06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:43.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.898+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1b4c0ab390 con 0x7f1b48003970 2023-11-24T08:08:43.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.898+0000 7f1b53fff640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b3c06ade0 0x7f1b3c06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:43.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.899+0000 7f1b53fff640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b3c06ade0 0x7f1b3c06d2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c157e20 tx=0x7f1b44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:43.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:43.902+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b4c0769b0 con 0x7f1b48003970 2023-11-24T08:08:44.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.113+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b5c0adf70 con 0x7f1b48003970 2023-11-24T08:08:44.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.115+0000 7f1b52ffd640 1 -- 172.21.15.106:0/874512274 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+12898 (secure 0 0 0) 0x7f1b5c0adf70 con 0x7f1b48003970 2023-11-24T08:08:44.117 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:44.117 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":18,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:43.836643+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:44.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b3c06ade0 msgr2=0x7f1b3c06d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b3c06ade0 0x7f1b3c06d2a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c157e20 tx=0x7f1b44008040 comp rx=0 tx=0).stop 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 msgr2=0x7f1b5c0b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b5c0b3d50 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c02c250 tx=0x7f1b4c004730 comp rx=0 tx=0).stop 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 shutdown_connections 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b3c06ade0 0x7f1b3c06d2a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b5c0ad5b0 0x7f1b5c0ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b5c0b2af0 0x7f1b5c154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 --2- 172.21.15.106:0/874512274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b48003970 0x7f1b5c0b3d50 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 >> 172.21.15.106:0/874512274 conn(0x7f1b5c06a920 msgr2=0x7f1b5c0ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 shutdown_connections 2023-11-24T08:08:44.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.118+0000 7f1b60dc5640 1 -- 172.21.15.106:0/874512274 wait complete. 2023-11-24T08:08:44.137 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:44.138 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:44.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.325+0000 7f8302ce7640 1 Processor -- start 2023-11-24T08:08:44.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.325+0000 7f8302ce7640 1 -- start start 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8302ce7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82fc15b440 0x7f82fc0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8302ce7640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f82fc0c59b0 0x7f82fc06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8302ce7640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc0c5dc0 con 0x7f82fc0cbf20 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8302ce7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc0cc3d0 con 0x7f82fc15b440 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8302ce7640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc15b820 con 0x7f82fc0c59b0 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8300a83640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82fc15b440 0x7f82fc0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8300a83640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82fc15b440 0x7f82fc0cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47594/0 (socket says 172.21.15.106:47594) 2023-11-24T08:08:44.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.326+0000 7f8300a83640 1 -- 172.21.15.106:0/44738758 learned_addr learned my addr 172.21.15.106:0/44738758 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:44.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.327+0000 7f8301284640 1 --2- 172.21.15.106:0/44738758 >> v2:172.21.15.156:3300/0 conn(0x7f82fc0c59b0 0x7f82fc06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.327+0000 7f8300a83640 1 -- 172.21.15.106:0/44738758 >> v2:172.21.15.156:3300/0 conn(0x7f82fc0c59b0 msgr2=0x7f82fc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:44.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.327+0000 7f8300a83640 1 --2- 172.21.15.106:0/44738758 >> v2:172.21.15.156:3300/0 conn(0x7f82fc0c59b0 0x7f82fc06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.327+0000 7f8300a83640 1 -- 172.21.15.106:0/44738758 >> v1:172.21.15.156:6789/0 conn(0x7f82fc0cbf20 legacy=0x7f82fc0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:44.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.327+0000 7f8300a83640 1 -- 172.21.15.106:0/44738758 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc15eea0 con 0x7f82fc15b440 2023-11-24T08:08:44.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.328+0000 7f8300a83640 1 --2- 172.21.15.106:0/44738758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc0cb9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0cb910 tx=0x7f82f002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a441b4789a5a8a02 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.328+0000 7f82fb7fe640 1 -- 172.21.15.106:0/44738758 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f00317e0 con 0x7f82fc15b440 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.328+0000 7f82fb7fe640 1 -- 172.21.15.106:0/44738758 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f0031940 con 0x7f82fc15b440 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.328+0000 7f82fb7fe640 1 -- 172.21.15.106:0/44738758 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f0031cd0 con 0x7f82fc15b440 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 -- 172.21.15.106:0/44738758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 msgr2=0x7f82fc0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 --2- 172.21.15.106:0/44738758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc0cb9e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0cb910 tx=0x7f82f002b880 comp rx=0 tx=0).stop 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 -- 172.21.15.106:0/44738758 shutdown_connections 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 --2- 172.21.15.106:0/44738758 >> v2:172.21.15.156:3300/0 conn(0x7f82fc0c59b0 0x7f82fc06c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 --2- 172.21.15.106:0/44738758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc0cb9e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 -- 172.21.15.106:0/44738758 >> 172.21.15.106:0/44738758 conn(0x7f82fc0b0c50 msgr2=0x7f82fc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 -- 172.21.15.106:0/44738758 shutdown_connections 2023-11-24T08:08:44.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.329+0000 7f8302ce7640 1 -- 172.21.15.106:0/44738758 wait complete. 2023-11-24T08:08:44.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.331+0000 7f8302ce7640 1 Processor -- start 2023-11-24T08:08:44.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.331+0000 7f8302ce7640 1 -- start start 2023-11-24T08:08:44.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8302ce7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82fc0c59b0 0x7f82fc15a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8302ce7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82fc0cbf20 0x7f82fc15e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8302ce7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8302ce7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0b45a0 con 0x7f82fc15b440 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8302ce7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0b3920 con 0x7f82fc0cbf20 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8302ce7640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0b3c00 con 0x7f82fc0c59b0 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8301284640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8301284640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47610/0 (socket says 172.21.15.106:47610) 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.332+0000 7f8301284640 1 -- 172.21.15.106:0/2356095361 learned_addr learned my addr 172.21.15.106:0/2356095361 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f82fbfff640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82fc0cbf20 0x7f82fc15e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8300a83640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82fc0c59b0 0x7f82fc15a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8301284640 1 -- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82fc0c59b0 msgr2=0x7f82fc15a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8301284640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82fc0c59b0 0x7f82fc15a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8301284640 1 -- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82fc0cbf20 msgr2=0x7f82fc15e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8301284640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82fc0cbf20 0x7f82fc15e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8301284640 1 -- 172.21.15.106:0/2356095361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f00094d0 con 0x7f82fc15b440 2023-11-24T08:08:44.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f82fbfff640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82fc0cbf20 0x7f82fc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:44.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8300a83640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82fc0c59b0 0x7f82fc15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:44.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8301284640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc156740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82f400cd70 tx=0x7f82f400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4003b90 con 0x7f82fc15b440 2023-11-24T08:08:44.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f4003cf0 con 0x7f82fc15b440 2023-11-24T08:08:44.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc0b3ee0 con 0x7f82fc15b440 2023-11-24T08:08:44.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.333+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4004240 con 0x7f82fc15b440 2023-11-24T08:08:44.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.334+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc0b30b0 con 0x7f82fc15b440 2023-11-24T08:08:44.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.335+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc0b46e0 con 0x7f82fc15b440 2023-11-24T08:08:44.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.335+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f82f40043a0 con 0x7f82fc15b440 2023-11-24T08:08:44.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.336+0000 7f82f97fa640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f82e006adb0 0x7f82e006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.337+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f82f4012070 con 0x7f82fc15b440 2023-11-24T08:08:44.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.337+0000 7f8300a83640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f82e006adb0 0x7f82e006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.338+0000 7f8300a83640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f82e006adb0 0x7f82e006d270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82f0036000 tx=0x7f82f003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.341+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82fc0b46e0 con 0x7f82fc15b440 2023-11-24T08:08:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.534+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f82fc0b46e0 con 0x7f82fc15b440 2023-11-24T08:08:44.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.535+0000 7f82f97fa640 1 -- 172.21.15.106:0/2356095361 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f82fc0b46e0 con 0x7f82fc15b440 2023-11-24T08:08:44.536 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f82e006adb0 msgr2=0x7f82e006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f82e006adb0 0x7f82e006d270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f82f0036000 tx=0x7f82f003d040 comp rx=0 tx=0).stop 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 msgr2=0x7f82fc156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc156740 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f82f400cd70 tx=0x7f82f400a660 comp rx=0 tx=0).stop 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 shutdown_connections 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82fc15b440 0x7f82fc156740 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82fc0cbf20 0x7f82fc15e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f82e006adb0 0x7f82e006d270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 --2- 172.21.15.106:0/2356095361 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82fc0c59b0 0x7f82fc15a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 >> 172.21.15.106:0/2356095361 conn(0x7f82fc0b0c50 msgr2=0x7f82fc0cf680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 shutdown_connections 2023-11-24T08:08:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.537+0000 7f8302ce7640 1 -- 172.21.15.106:0/2356095361 wait complete. 2023-11-24T08:08:44.557 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:08:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.771+0000 7f1f7a5e5640 1 Processor -- start 2023-11-24T08:08:44.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.771+0000 7f1f7a5e5640 1 -- start start 2023-11-24T08:08:44.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f7a5e5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f7415c7f0 0x7f1f7406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f7a5e5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f740cf450 0x7f1f740cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f7a5e5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740cece0 con 0x7f1f740cdfb0 2023-11-24T08:08:44.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f7a5e5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740cfe40 con 0x7f1f740cf450 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f7a5e5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740ceaf0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f78b82640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1f740cdfb0 0x7f1f740ce3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54300/0 (socket says 172.21.15.106:54300) 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f78b82640 1 -- 172.21.15.106:0/4012469067 learned_addr learned my addr 172.21.15.106:0/4012469067 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f6bfff640 1 --2- 172.21.15.106:0/4012469067 >> v2:172.21.15.106:3300/0 conn(0x7f1f740cf450 0x7f1f740cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.773+0000 7f1f73fff640 1 --2- 172.21.15.106:0/4012469067 >> v2:172.21.15.156:3300/0 conn(0x7f1f7415c7f0 0x7f1f7406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.774+0000 7f1f6bfff640 1 -- 172.21.15.106:0/4012469067 >> v2:172.21.15.156:3300/0 conn(0x7f1f7415c7f0 msgr2=0x7f1f7406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.774+0000 7f1f6bfff640 1 --2- 172.21.15.106:0/4012469067 >> v2:172.21.15.156:3300/0 conn(0x7f1f7415c7f0 0x7f1f7406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.774+0000 7f1f6bfff640 1 -- 172.21.15.106:0/4012469067 >> v1:172.21.15.106:6789/0 conn(0x7f1f740cdfb0 legacy=0x7f1f740ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.774+0000 7f1f6bfff640 1 -- 172.21.15.106:0/4012469067 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7406d040 con 0x7f1f740cf450 2023-11-24T08:08:44.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.775+0000 7f1f6bfff640 1 --2- 172.21.15.106:0/4012469067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f740cf450 0x7f1f740cf900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f740d0130 tx=0x7f1f5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3052dbb55e6217d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.775+0000 7f1f737fe640 1 -- 172.21.15.106:0/4012469067 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c031830 con 0x7f1f740cf450 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.775+0000 7f1f737fe640 1 -- 172.21.15.106:0/4012469067 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c031990 con 0x7f1f740cf450 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.775+0000 7f1f737fe640 1 -- 172.21.15.106:0/4012469067 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c031d20 con 0x7f1f740cf450 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/4012469067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f740cf450 msgr2=0x7f1f740cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/4012469067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f740cf450 0x7f1f740cf900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1f740d0130 tx=0x7f1f5c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/4012469067 shutdown_connections 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/4012469067 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f740cf450 0x7f1f740cf900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/4012469067 >> v2:172.21.15.156:3300/0 conn(0x7f1f7415c7f0 0x7f1f7406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/4012469067 >> 172.21.15.106:0/4012469067 conn(0x7f1f7406a920 msgr2=0x7f1f7406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/4012469067 shutdown_connections 2023-11-24T08:08:44.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.776+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/4012469067 wait complete. 2023-11-24T08:08:44.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.778+0000 7f1f7a5e5640 1 Processor -- start 2023-11-24T08:08:44.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.778+0000 7f1f7a5e5640 1 -- start start 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f7a5e5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f740cdfb0 0x7f1f74158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f7a5e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f740cf450 0x7f1f74155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f7a5e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 0x7f1f741483b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f7a5e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741570f0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f7a5e5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741564e0 con 0x7f1f740cdfb0 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f7a5e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741567c0 con 0x7f1f740cf450 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f78b82640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 0x7f1f741483b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f78b82640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 0x7f1f741483b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47630/0 (socket says 172.21.15.106:47630) 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.779+0000 7f1f78b82640 1 -- 172.21.15.106:0/2460410844 learned_addr learned my addr 172.21.15.106:0/2460410844 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f78b82640 1 -- 172.21.15.106:0/2460410844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f740cf450 msgr2=0x7f1f74155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f78b82640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f740cf450 0x7f1f74155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f78b82640 1 -- 172.21.15.106:0/2460410844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f740cdfb0 msgr2=0x7f1f74158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f78b82640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f740cdfb0 0x7f1f74158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f78b82640 1 -- 172.21.15.106:0/2460410844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c009400 con 0x7f1f7415c7f0 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f78b82640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 0x7f1f741483b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00ca70 tx=0x7f1f6c00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c00abd0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c00ad30 con 0x7f1f7415c7f0 2023-11-24T08:08:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c012600 con 0x7f1f7415c7f0 2023-11-24T08:08:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f74156aa0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.780+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7415acc0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.781+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f74067470 con 0x7f1f7415c7f0 2023-11-24T08:08:44.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.782+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f1f6c007070 con 0x7f1f7415c7f0 2023-11-24T08:08:44.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.783+0000 7f1f717fa640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f5406ade0 0x7f1f5406d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:44.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.783+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f6c088f60 con 0x7f1f7415c7f0 2023-11-24T08:08:44.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.784+0000 7f1f73fff640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f5406ade0 0x7f1f5406d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:44.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.784+0000 7f1f73fff640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f5406ade0 0x7f1f5406d2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f64006fd0 tx=0x7f1f64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:44.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.788+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f6c0544e0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.981+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1f7415fbf0 con 0x7f1f7415c7f0 2023-11-24T08:08:44.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.981+0000 7f1f717fa640 1 -- 172.21.15.106:0/2460410844 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+65 (secure 0 0 0) 0x7f1f6c017090 con 0x7f1f7415c7f0 2023-11-24T08:08:44.983 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:44.983 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:08:44.983 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:08:44.983 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2023-11-24T08:08:44.983 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2023-11-24T08:08:44.984 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:08:44.984 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:08:44.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f5406ade0 msgr2=0x7f1f5406d2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f5406ade0 0x7f1f5406d2a0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1f64006fd0 tx=0x7f1f64008040 comp rx=0 tx=0).stop 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 msgr2=0x7f1f741483b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 0x7f1f741483b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c00ca70 tx=0x7f1f6c00cf30 comp rx=0 tx=0).stop 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 shutdown_connections 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f7415c7f0 0x7f1f741483b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f5406ade0 0x7f1f5406d2a0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f740cf450 0x7f1f74155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 --2- 172.21.15.106:0/2460410844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f740cdfb0 0x7f1f74158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 >> 172.21.15.106:0/2460410844 conn(0x7f1f7406a920 msgr2=0x7f1f7406c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 shutdown_connections 2023-11-24T08:08:44.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:44.984+0000 7f1f7a5e5640 1 -- 172.21.15.106:0/2460410844 wait complete. 2023-11-24T08:08:45.006 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:08:45.006 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:08:45.101 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:45.110 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:08:45.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.213+0000 7f6fe936f640 1 Processor -- start 2023-11-24T08:08:45.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.213+0000 7f6fe936f640 1 -- start start 2023-11-24T08:08:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.214+0000 7f6fe936f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6fe4157c70 0x7f6fe4161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.214+0000 7f6fe936f640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6fe415e390 0x7f6fe40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe936f640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe40ce0b0 con 0x7f6fe40cf450 2023-11-24T08:08:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe936f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40cfe40 con 0x7f6fe4157c70 2023-11-24T08:08:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe936f640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe4161960 con 0x7f6fe415e390 2023-11-24T08:08:45.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe2ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6fe4157c70 0x7f6fe4161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe2ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6fe4157c70 0x7f6fe4161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47646/0 (socket says 172.21.15.106:47646) 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe2ffd640 1 -- 172.21.15.106:0/1329343815 learned_addr learned my addr 172.21.15.106:0/1329343815 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe37fe640 1 --2- 172.21.15.106:0/1329343815 >> v2:172.21.15.156:3300/0 conn(0x7f6fe415e390 0x7f6fe40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.215+0000 7f6fe1ffb640 1 -- 172.21.15.106:0/1329343815 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 933149544 0 0) 0x7f6fe40ce0b0 con 0x7f6fe40cf450 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.216+0000 7f6fe1ffb640 1 -- 172.21.15.106:0/1329343815 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fcc003520 con 0x7f6fe40cf450 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.216+0000 7f6fe2ffd640 1 -- 172.21.15.106:0/1329343815 >> v2:172.21.15.156:3300/0 conn(0x7f6fe415e390 msgr2=0x7f6fe40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.216+0000 7f6fe2ffd640 1 --2- 172.21.15.106:0/1329343815 >> v2:172.21.15.156:3300/0 conn(0x7f6fe415e390 0x7f6fe40cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.216+0000 7f6fe2ffd640 1 -- 172.21.15.106:0/1329343815 >> v1:172.21.15.156:6789/0 conn(0x7f6fe40cf450 legacy=0x7f6fe415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.216+0000 7f6fe2ffd640 1 -- 172.21.15.106:0/1329343815 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe415e8d0 con 0x7f6fe4157c70 2023-11-24T08:08:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.217+0000 7f6fe2ffd640 1 --2- 172.21.15.106:0/1329343815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4161df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40d0100 tx=0x7f6fd402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e5d00717e51388 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.217+0000 7f6fe1ffb640 1 -- 172.21.15.106:0/1329343815 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4031870 con 0x7f6fe4157c70 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.217+0000 7f6fe1ffb640 1 -- 172.21.15.106:0/1329343815 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd40319d0 con 0x7f6fe4157c70 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.217+0000 7f6fe1ffb640 1 -- 172.21.15.106:0/1329343815 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4031d60 con 0x7f6fe4157c70 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 -- 172.21.15.106:0/1329343815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 msgr2=0x7f6fe4161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 --2- 172.21.15.106:0/1329343815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4161df0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40d0100 tx=0x7f6fd402b910 comp rx=0 tx=0).stop 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 -- 172.21.15.106:0/1329343815 shutdown_connections 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 --2- 172.21.15.106:0/1329343815 >> v2:172.21.15.156:3300/0 conn(0x7f6fe415e390 0x7f6fe40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 --2- 172.21.15.106:0/1329343815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4161df0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 -- 172.21.15.106:0/1329343815 >> 172.21.15.106:0/1329343815 conn(0x7f6fe406a920 msgr2=0x7f6fe406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 -- 172.21.15.106:0/1329343815 shutdown_connections 2023-11-24T08:08:45.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.218+0000 7f6fe936f640 1 -- 172.21.15.106:0/1329343815 wait complete. 2023-11-24T08:08:45.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.220+0000 7f6fe936f640 1 Processor -- start 2023-11-24T08:08:45.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.220+0000 7f6fe936f640 1 -- start start 2023-11-24T08:08:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe936f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe40cf450 0x7f6fe41426b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe936f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4144560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe936f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe415e390 0x7f6fe4144aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe936f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4141610 con 0x7f6fe4157c70 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe936f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe41418f0 con 0x7f6fe415e390 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe936f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe4141bd0 con 0x7f6fe40cf450 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4144560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4144560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47654/0 (socket says 172.21.15.106:47654) 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe27fc640 1 -- 172.21.15.106:0/2330012088 learned_addr learned my addr 172.21.15.106:0/2330012088 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe37fe640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe415e390 0x7f6fe4144aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.221+0000 7f6fe2ffd640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe40cf450 0x7f6fe41426b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe27fc640 1 -- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe40cf450 msgr2=0x7f6fe41426b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe27fc640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe40cf450 0x7f6fe41426b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe27fc640 1 -- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe415e390 msgr2=0x7f6fe4144aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe27fc640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe415e390 0x7f6fe4144aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:45.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe27fc640 1 -- 172.21.15.106:0/2330012088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40094d0 con 0x7f6fe4157c70 2023-11-24T08:08:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe27fc640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4144560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800c430 tx=0x7f6fd800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80052d0 con 0x7f6fe4157c70 2023-11-24T08:08:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd8005450 con 0x7f6fe4157c70 2023-11-24T08:08:45.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8003830 con 0x7f6fe4157c70 2023-11-24T08:08:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe4141eb0 con 0x7f6fe4157c70 2023-11-24T08:08:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.222+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe4142bf0 con 0x7f6fe4157c70 2023-11-24T08:08:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.223+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe41423d0 con 0x7f6fe4157c70 2023-11-24T08:08:45.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.224+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 4) v1 ==== 89175+0+0 (secure 0 0 0) 0x7f6fd8003990 con 0x7f6fe4157c70 2023-11-24T08:08:45.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.226+0000 7f6fcb7fe640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006adb0 0x7f6fd006d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:45.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.226+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6fd8088820 con 0x7f6fe4157c70 2023-11-24T08:08:45.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.226+0000 7f6fe2ffd640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006adb0 0x7f6fd006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:45.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.227+0000 7f6fe2ffd640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006adb0 0x7f6fd006d270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4036000 tx=0x7f6fd403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:45.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.230+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fe41423d0 con 0x7f6fe4157c70 2023-11-24T08:08:45.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:45.421+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6fe41423d0 con 0x7f6fe4157c70 2023-11-24T08:08:46.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.093+0000 7f6fcb7fe640 1 -- 172.21.15.106:0/2330012088 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6fe41423d0 con 0x7f6fe4157c70 2023-11-24T08:08:46.095 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2023-11-24T08:08:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.096+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006adb0 msgr2=0x7f6fd006d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.096+0000 7f6fe936f640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006adb0 0x7f6fd006d270 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6fd4036000 tx=0x7f6fd403c040 comp rx=0 tx=0).stop 2023-11-24T08:08:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.096+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 msgr2=0x7f6fe4144560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.096+0000 7f6fe936f640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4144560 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800c430 tx=0x7f6fd800c8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 shutdown_connections 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fe415e390 0x7f6fe4144aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe4157c70 0x7f6fe4144560 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006adb0 0x7f6fd006d270 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 --2- 172.21.15.106:0/2330012088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe40cf450 0x7f6fe41426b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 >> 172.21.15.106:0/2330012088 conn(0x7f6fe406a920 msgr2=0x7f6fe40b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 shutdown_connections 2023-11-24T08:08:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.097+0000 7f6fe936f640 1 -- 172.21.15.106:0/2330012088 wait complete. 2023-11-24T08:08:46.128 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:08:46.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.385+0000 7f9ed5b5c640 1 Processor -- start 2023-11-24T08:08:46.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.386+0000 7f9ed5b5c640 1 -- start start 2023-11-24T08:08:46.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.387+0000 7f9ed5b5c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9ed006ac10 0x7f9ed00cf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.387+0000 7f9ed5b5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed0161130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.387+0000 7f9ed5b5c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed00cd400 con 0x7f9ed015d6e0 2023-11-24T08:08:46.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.387+0000 7f9ed5b5c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed00cd890 con 0x7f9ed006ac10 2023-11-24T08:08:46.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.387+0000 7f9ed5b5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed00cfdf0 con 0x7f9ed00cf540 2023-11-24T08:08:46.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.388+0000 7f9ed535b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed0161130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.388+0000 7f9ecffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9ed006ac10 0x7f9ed00cf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.388+0000 7f9ed4b5a640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f9ed015d6e0 0x7f9ed006a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:49062/0 (socket says 172.21.15.106:49062) 2023-11-24T08:08:46.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.388+0000 7f9ed4b5a640 1 -- 172.21.15.106:0/1440529499 learned_addr learned my addr 172.21.15.106:0/1440529499 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:46.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.389+0000 7f9ed535b640 1 -- 172.21.15.106:0/1440529499 >> v2:172.21.15.156:3300/0 conn(0x7f9ed006ac10 msgr2=0x7f9ed00cf000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:46.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.389+0000 7f9ed535b640 1 --2- 172.21.15.106:0/1440529499 >> v2:172.21.15.156:3300/0 conn(0x7f9ed006ac10 0x7f9ed00cf000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.389+0000 7f9ed535b640 1 -- 172.21.15.106:0/1440529499 >> v1:172.21.15.156:6789/0 conn(0x7f9ed015d6e0 legacy=0x7f9ed006a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.389+0000 7f9ed535b640 1 -- 172.21.15.106:0/1440529499 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed00ccad0 con 0x7f9ed00cf540 2023-11-24T08:08:46.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.390+0000 7f9ed535b640 1 --2- 172.21.15.106:0/1440529499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed0161130 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80097a0 tx=0x7f9ec802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f048d2f0b90e22c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.391+0000 7f9ecf7fe640 1 -- 172.21.15.106:0/1440529499 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8031830 con 0x7f9ed00cf540 2023-11-24T08:08:46.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.391+0000 7f9ecf7fe640 1 -- 172.21.15.106:0/1440529499 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec8031990 con 0x7f9ed00cf540 2023-11-24T08:08:46.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.391+0000 7f9ecf7fe640 1 -- 172.21.15.106:0/1440529499 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec8031d20 con 0x7f9ed00cf540 2023-11-24T08:08:46.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.391+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1440529499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 msgr2=0x7f9ed0161130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.391+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1440529499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed0161130 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80097a0 tx=0x7f9ec802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.392+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1440529499 shutdown_connections 2023-11-24T08:08:46.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.392+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1440529499 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed0161130 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.392+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1440529499 >> v2:172.21.15.156:3300/0 conn(0x7f9ed006ac10 0x7f9ed00cf000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.392+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1440529499 >> 172.21.15.106:0/1440529499 conn(0x7f9ed00b1cd0 msgr2=0x7f9ed00b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:46.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.392+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1440529499 shutdown_connections 2023-11-24T08:08:46.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.392+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1440529499 wait complete. 2023-11-24T08:08:46.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.393+0000 7f9ed5b5c640 1 Processor -- start 2023-11-24T08:08:46.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 -- start start 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ed006ac10 0x7f9ed0157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed00acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 0x7f9ed01415b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0158ab0 con 0x7f9ed015d6e0 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0156040 con 0x7f9ed006ac10 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.394+0000 7f9ed5b5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed0141af0 con 0x7f9ed00cf540 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 0x7f9ed01415b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 0x7f9ed01415b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47674/0 (socket says 172.21.15.106:47674) 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 -- 172.21.15.106:0/1384848404 learned_addr learned my addr 172.21.15.106:0/1384848404 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ecffff640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed00acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed4b5a640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ed006ac10 0x7f9ed0157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 -- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 msgr2=0x7f9ed00acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed00acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 -- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ed006ac10 msgr2=0x7f9ed0157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ed006ac10 0x7f9ed0157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed535b640 1 -- 172.21.15.106:0/1384848404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009400 con 0x7f9ed015d6e0 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ecffff640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed00acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.395+0000 7f9ed4b5a640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ed006ac10 0x7f9ed0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:46.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.396+0000 7f9ed535b640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 0x7f9ed01415b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80096f0 tx=0x7f9ec80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.396+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec803a630 con 0x7f9ed015d6e0 2023-11-24T08:08:46.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.396+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec803a790 con 0x7f9ed015d6e0 2023-11-24T08:08:46.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.396+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec803ab00 con 0x7f9ed015d6e0 2023-11-24T08:08:46.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.396+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed015dc40 con 0x7f9ed015d6e0 2023-11-24T08:08:46.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.396+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed0157cf0 con 0x7f9ed015d6e0 2023-11-24T08:08:46.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.397+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed004e840 con 0x7f9ed015d6e0 2023-11-24T08:08:46.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.398+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9ec803ac60 con 0x7f9ed015d6e0 2023-11-24T08:08:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.399+0000 7f9ecd7fa640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9eb006ae40 0x7f9eb006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.399+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f9ec80aacd0 con 0x7f9ed015d6e0 2023-11-24T08:08:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.399+0000 7f9ed4b5a640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9eb006ae40 0x7f9eb006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:46.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.400+0000 7f9ed4b5a640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9eb006ae40 0x7f9eb006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00012e0 tx=0x7f9ec0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:46.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.404+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9ec8076010 con 0x7f9ed015d6e0 2023-11-24T08:08:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:46.599+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9ed00b2890 con 0x7f9ed015d6e0 2023-11-24T08:08:47.112 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.110+0000 7f9ecd7fa640 1 -- 172.21.15.106:0/1384848404 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9ec80480d0 con 0x7f9ed015d6e0 2023-11-24T08:08:47.112 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2023-11-24T08:08:47.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.113+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9eb006ae40 msgr2=0x7f9eb006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.113+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9eb006ae40 0x7f9eb006d300 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ec00012e0 tx=0x7f9ec0009040 comp rx=0 tx=0).stop 2023-11-24T08:08:47.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.113+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 msgr2=0x7f9ed01415b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.113+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 0x7f9ed01415b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80096f0 tx=0x7f9ec80046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.114+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 shutdown_connections 2023-11-24T08:08:47.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.114+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9ed015d6e0 0x7f9ed01415b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.114+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9eb006ae40 0x7f9eb006d300 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.114+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9ed00cf540 0x7f9ed00acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.114+0000 7f9ed5b5c640 1 --2- 172.21.15.106:0/1384848404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9ed006ac10 0x7f9ed0157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.114+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 >> 172.21.15.106:0/1384848404 conn(0x7f9ed00b1cd0 msgr2=0x7f9ed006e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:47.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.115+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 shutdown_connections 2023-11-24T08:08:47.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.116+0000 7f9ed5b5c640 1 -- 172.21.15.106:0/1384848404 wait complete. 2023-11-24T08:08:47.154 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:08:47.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.337+0000 7ff55e3de640 1 Processor -- start 2023-11-24T08:08:47.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.338+0000 7ff55e3de640 1 -- start start 2023-11-24T08:08:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff55e3de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff55815b440 0x7ff5580cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff55e3de640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff5580cbf20 0x7ff5580c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff55e3de640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55815eea0 con 0x7ff5580c57e0 2023-11-24T08:08:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff55e3de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5580cc3d0 con 0x7ff55815b440 2023-11-24T08:08:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff55e3de640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55815b820 con 0x7ff5580cbf20 2023-11-24T08:08:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff557fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff55815b440 0x7ff5580cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff557fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff55815b440 0x7ff5580cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47676/0 (socket says 172.21.15.106:47676) 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.339+0000 7ff557fff640 1 -- 172.21.15.106:0/1926864362 learned_addr learned my addr 172.21.15.106:0/1926864362 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.340+0000 7ff54ffff640 1 --2- 172.21.15.106:0/1926864362 >> v2:172.21.15.156:3300/0 conn(0x7ff5580cbf20 0x7ff5580c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.340+0000 7ff557fff640 1 -- 172.21.15.106:0/1926864362 >> v2:172.21.15.156:3300/0 conn(0x7ff5580cbf20 msgr2=0x7ff5580c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.340+0000 7ff557fff640 1 --2- 172.21.15.106:0/1926864362 >> v2:172.21.15.156:3300/0 conn(0x7ff5580cbf20 0x7ff5580c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.340+0000 7ff557fff640 1 -- 172.21.15.106:0/1926864362 >> v1:172.21.15.156:6789/0 conn(0x7ff5580c57e0 legacy=0x7ff5580c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:47.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.340+0000 7ff557fff640 1 -- 172.21.15.106:0/1926864362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55806d600 con 0x7ff55815b440 2023-11-24T08:08:47.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.341+0000 7ff557fff640 1 --2- 172.21.15.106:0/1926864362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580cb9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff55806e890 tx=0x7ff54802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c4154b2f93db87 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.341+0000 7ff5577fe640 1 -- 172.21.15.106:0/1926864362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5480317e0 con 0x7ff55815b440 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.341+0000 7ff5577fe640 1 -- 172.21.15.106:0/1926864362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548031940 con 0x7ff55815b440 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.341+0000 7ff5577fe640 1 -- 172.21.15.106:0/1926864362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548031cd0 con 0x7ff55815b440 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 -- 172.21.15.106:0/1926864362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 msgr2=0x7ff5580cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1926864362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580cb9e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff55806e890 tx=0x7ff54802b880 comp rx=0 tx=0).stop 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 -- 172.21.15.106:0/1926864362 shutdown_connections 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1926864362 >> v2:172.21.15.156:3300/0 conn(0x7ff5580cbf20 0x7ff5580c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1926864362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580cb9e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 -- 172.21.15.106:0/1926864362 >> 172.21.15.106:0/1926864362 conn(0x7ff5580b0c50 msgr2=0x7ff5580b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 -- 172.21.15.106:0/1926864362 shutdown_connections 2023-11-24T08:08:47.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.342+0000 7ff55e3de640 1 -- 172.21.15.106:0/1926864362 wait complete. 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.344+0000 7ff55e3de640 1 Processor -- start 2023-11-24T08:08:47.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.344+0000 7ff55e3de640 1 -- start start 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55e3de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5580c57e0 0x7ff558142eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55e3de640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5580cbf20 0x7ff5580b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55e3de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580704c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55e3de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581433f0 con 0x7ff55815b440 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55e3de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580999d0 con 0x7ff5580c57e0 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55e3de640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5580b49a0 con 0x7ff5580cbf20 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47688/0 (socket says 172.21.15.106:47688) 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 -- 172.21.15.106:0/1211022931 learned_addr learned my addr 172.21.15.106:0/1211022931 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff557fff640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5580c57e0 0x7ff558142eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff54ffff640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5580cbf20 0x7ff5580b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 -- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5580cbf20 msgr2=0x7ff5580b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5580cbf20 0x7ff5580b3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 -- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5580c57e0 msgr2=0x7ff558142eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5580c57e0 0x7ff558142eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff55c97b640 1 -- 172.21.15.106:0/1211022931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5480094d0 con 0x7ff55815b440 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.345+0000 7ff557fff640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5580c57e0 0x7ff558142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.346+0000 7ff55c97b640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580704c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c780 tx=0x7ff55000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.346+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550003d80 con 0x7ff55815b440 2023-11-24T08:08:47.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.346+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55000ce30 con 0x7ff55815b440 2023-11-24T08:08:47.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.346+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff558099c20 con 0x7ff55815b440 2023-11-24T08:08:47.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.346+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550005290 con 0x7ff55815b440 2023-11-24T08:08:47.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.346+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55809a1e0 con 0x7ff55815b440 2023-11-24T08:08:47.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.347+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55809a400 con 0x7ff55815b440 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.348+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5500053f0 con 0x7ff55815b440 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.349+0000 7ff5557fa640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff53806ae40 0x7ff53806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.349+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5609+0+0 (secure 0 0 0) 0x7ff550088dc0 con 0x7ff55815b440 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.349+0000 7ff557fff640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff53806ae40 0x7ff53806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:47.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.350+0000 7ff557fff640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff53806ae40 0x7ff53806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff548036000 tx=0x7ff54803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:47.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.354+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff550053f60 con 0x7ff55815b440 2023-11-24T08:08:47.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:47.584+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff5580c93c0 con 0x7ff55815b440 2023-11-24T08:08:48.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.114+0000 7ff5557fa640 1 -- 172.21.15.106:0/1211022931 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7ff550017070 con 0x7ff55815b440 2023-11-24T08:08:48.116 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 2 and data pool 3 2023-11-24T08:08:48.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.117+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff53806ae40 msgr2=0x7ff53806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.117+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff53806ae40 0x7ff53806d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff548036000 tx=0x7ff54803d040 comp rx=0 tx=0).stop 2023-11-24T08:08:48.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.117+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 msgr2=0x7ff5580704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.117+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580704c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c780 tx=0x7ff55000cc40 comp rx=0 tx=0).stop 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 shutdown_connections 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff55815b440 0x7ff5580704c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff53806ae40 0x7ff53806d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5580cbf20 0x7ff5580b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 --2- 172.21.15.106:0/1211022931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5580c57e0 0x7ff558142eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 >> 172.21.15.106:0/1211022931 conn(0x7ff5580b0c50 msgr2=0x7ff5580b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 shutdown_connections 2023-11-24T08:08:48.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.118+0000 7ff55e3de640 1 -- 172.21.15.106:0/1211022931 wait complete. 2023-11-24T08:08:48.154 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:48.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.337+0000 7f07acb1d640 1 Processor -- start 2023-11-24T08:08:48.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.338+0000 7f07acb1d640 1 -- start start 2023-11-24T08:08:48.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07acb1d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f07a80cf450 0x7f07a80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07acb1d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f07a80cdfb0 0x7f07a80ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07acb1d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a80cece0 con 0x7f07a815c7f0 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07acb1d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a80cfe40 con 0x7f07a80cdfb0 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07acb1d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a8156010 con 0x7f07a80cf450 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07a659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f07a815c7f0 0x7f07a806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54302/0 (socket says 172.21.15.106:54302) 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07a659c640 1 -- 172.21.15.106:0/3014754354 learned_addr learned my addr 172.21.15.106:0/3014754354 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.339+0000 7f07a6d9d640 1 --2- 172.21.15.106:0/3014754354 >> v2:172.21.15.106:3300/0 conn(0x7f07a80cdfb0 0x7f07a80ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a5d9b640 1 --2- 172.21.15.106:0/3014754354 >> v2:172.21.15.156:3300/0 conn(0x7f07a80cf450 0x7f07a80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a559a640 1 -- 172.21.15.106:0/3014754354 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527666819 0 0) 0x7f07a80cece0 con 0x7f07a815c7f0 2023-11-24T08:08:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a559a640 1 -- 172.21.15.106:0/3014754354 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0794003520 con 0x7f07a815c7f0 2023-11-24T08:08:48.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a6d9d640 1 -- 172.21.15.106:0/3014754354 >> v2:172.21.15.156:3300/0 conn(0x7f07a80cf450 msgr2=0x7f07a80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a6d9d640 1 --2- 172.21.15.106:0/3014754354 >> v2:172.21.15.156:3300/0 conn(0x7f07a80cf450 0x7f07a80cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a6d9d640 1 -- 172.21.15.106:0/3014754354 >> v1:172.21.15.106:6789/0 conn(0x7f07a815c7f0 legacy=0x7f07a806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.340+0000 7f07a6d9d640 1 -- 172.21.15.106:0/3014754354 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a806d040 con 0x7f07a80cdfb0 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.341+0000 7f07a6d9d640 1 --2- 172.21.15.106:0/3014754354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a80ce3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f07a8070410 tx=0x7f07a002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded0920d230a9572 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.341+0000 7f07a559a640 1 -- 172.21.15.106:0/3014754354 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a002cc80 con 0x7f07a80cdfb0 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.341+0000 7f07a559a640 1 -- 172.21.15.106:0/3014754354 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a002cde0 con 0x7f07a80cdfb0 2023-11-24T08:08:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.341+0000 7f07a559a640 1 -- 172.21.15.106:0/3014754354 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0034660 con 0x7f07a80cdfb0 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.341+0000 7f07acb1d640 1 -- 172.21.15.106:0/3014754354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 msgr2=0x7f07a80ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.341+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3014754354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a80ce3e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f07a8070410 tx=0x7f07a002bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.342+0000 7f07acb1d640 1 -- 172.21.15.106:0/3014754354 shutdown_connections 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.342+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3014754354 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a80ce3e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.342+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3014754354 >> v2:172.21.15.156:3300/0 conn(0x7f07a80cf450 0x7f07a80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.342+0000 7f07acb1d640 1 -- 172.21.15.106:0/3014754354 >> 172.21.15.106:0/3014754354 conn(0x7f07a806a920 msgr2=0x7f07a806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.342+0000 7f07acb1d640 1 -- 172.21.15.106:0/3014754354 shutdown_connections 2023-11-24T08:08:48.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.342+0000 7f07acb1d640 1 -- 172.21.15.106:0/3014754354 wait complete. 2023-11-24T08:08:48.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.343+0000 7f07acb1d640 1 Processor -- start 2023-11-24T08:08:48.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.344+0000 7f07acb1d640 1 -- start start 2023-11-24T08:08:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.344+0000 7f07acb1d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a81516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.344+0000 7f07acb1d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07a80cf450 0x7f07a80c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07acb1d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a815c7f0 0x7f07a80c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07acb1d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80b36b0 con 0x7f07a80cdfb0 2023-11-24T08:08:48.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07acb1d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80ac570 con 0x7f07a815c7f0 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07acb1d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a80ac850 con 0x7f07a80cf450 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a81516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a81516f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47706/0 (socket says 172.21.15.106:47706) 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 -- 172.21.15.106:0/3085925498 learned_addr learned my addr 172.21.15.106:0/3085925498 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a5d9b640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07a80cf450 0x7f07a80c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a6d9d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a815c7f0 0x7f07a80c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 -- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07a80cf450 msgr2=0x7f07a80c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07a80cf450 0x7f07a80c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 -- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a815c7f0 msgr2=0x7f07a80c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a815c7f0 0x7f07a80c9220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 -- 172.21.15.106:0/3085925498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a0009a70 con 0x7f07a80cdfb0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.345+0000 7f07a659c640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a81516f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f079800c410 tx=0x7f079800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.346+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0798005ce0 con 0x7f07a80cdfb0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.346+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0798005e40 con 0x7f07a80cdfb0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.346+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07980072f0 con 0x7f07a80cdfb0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.346+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a80acb30 con 0x7f07a80cdfb0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.346+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a8156150 con 0x7f07a80cdfb0 2023-11-24T08:08:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.347+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a80ad050 con 0x7f07a80cdfb0 2023-11-24T08:08:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.348+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0798007510 con 0x7f07a80cdfb0 2023-11-24T08:08:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.349+0000 7f079effd640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f078806ae40 0x7f078806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.349+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0798088cc0 con 0x7f07a80cdfb0 2023-11-24T08:08:48.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.350+0000 7f07a5d9b640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f078806ae40 0x7f078806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.350+0000 7f07a5d9b640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f078806ae40 0x7f078806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.354+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f07a80ad050 con 0x7f07a80cdfb0 2023-11-24T08:08:48.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.538+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07a80ad050 con 0x7f07a80cdfb0 2023-11-24T08:08:48.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.540+0000 7f079effd640 1 -- 172.21.15.106:0/3085925498 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16566 (secure 0 0 0) 0x7f07a80ad050 con 0x7f07a80cdfb0 2023-11-24T08:08:48.542 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:48.542 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":21,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:47.586084+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f078806ae40 msgr2=0x7f078806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f078806ae40 0x7f078806d300 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790008040 comp rx=0 tx=0).stop 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 msgr2=0x7f07a81516f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a81516f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f079800c410 tx=0x7f079800c8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 shutdown_connections 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07a815c7f0 0x7f07a80c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f078806ae40 0x7f078806d300 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07a80cf450 0x7f07a80c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 --2- 172.21.15.106:0/3085925498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07a80cdfb0 0x7f07a81516f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 >> 172.21.15.106:0/3085925498 conn(0x7f07a806a920 msgr2=0x7f07a806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 shutdown_connections 2023-11-24T08:08:48.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.542+0000 7f07acb1d640 1 -- 172.21.15.106:0/3085925498 wait complete. 2023-11-24T08:08:48.560 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.757+0000 7fe5e300e640 1 Processor -- start 2023-11-24T08:08:48.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.758+0000 7fe5e300e640 1 -- start start 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.759+0000 7fe5e300e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe5dc0cf450 0x7fe5dc0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.759+0000 7fe5e300e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 0x7fe5dc0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.759+0000 7fe5e300e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0cece0 con 0x7fe5dc15c7f0 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.759+0000 7fe5e300e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc0cfe40 con 0x7fe5dc0cf450 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.759+0000 7fe5e300e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc156010 con 0x7fe5dc0cdfb0 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5e0daa640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe5dc15c7f0 0x7fe5dc06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54312/0 (socket says 172.21.15.106:54312) 2023-11-24T08:08:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5e0daa640 1 -- 172.21.15.106:0/3150047969 learned_addr learned my addr 172.21.15.106:0/3150047969 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:48.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3150047969 >> v2:172.21.15.156:3300/0 conn(0x7fe5dc0cf450 0x7fe5dc0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5e15ab640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 0x7fe5dc0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033183852 0 0) 0x7fe5dc0cece0 con 0x7fe5dc15c7f0 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5cc003520 con 0x7fe5dc15c7f0 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2317850265 0 0) 0x7fe5cc003520 con 0x7fe5dc15c7f0 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 msgr2=0x7fe5dc0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 0x7fe5dc0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 >> v2:172.21.15.156:3300/0 conn(0x7fe5dc0cf450 msgr2=0x7fe5dc0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3150047969 >> v2:172.21.15.156:3300/0 conn(0x7fe5dc0cf450 0x7fe5dc0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc06d040 con 0x7fe5dc15c7f0 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fe5d0001670 con 0x7fe5dc15c7f0 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.760+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 >> v1:172.21.15.106:6789/0 conn(0x7fe5dc15c7f0 legacy=0x7fe5dc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5db7fe640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 0x7fe5cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0cfe40 con 0x7fe5cc0047c0 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc156010 con 0x7fe5cc003970 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc00b980 con 0x7fe5dc0cdfb0 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5e0daa640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5e15ab640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 0x7fe5cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5dbfff640 1 -- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 msgr2=0x7fe5cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc0cdfb0 0x7fe5cc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5dbfff640 1 -- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5cc003970 msgr2=0x7fe5cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5dbfff640 1 -- 172.21.15.106:0/3150047969 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc003520 con 0x7fe5cc0047c0 2023-11-24T08:08:48.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.761+0000 7fe5e0daa640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5cc003970 0x7fe5cc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:48.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.762+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009ac0 tx=0x7fe5cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eac0713c55ba5ac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.762+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802c710 con 0x7fe5cc0047c0 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c802c870 con 0x7fe5cc0047c0 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5db7fe640 1 -- 172.21.15.106:0/3150047969 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c802cc00 con 0x7fe5cc0047c0 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e300e640 1 -- 172.21.15.106:0/3150047969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 msgr2=0x7fe5cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e300e640 1 --2- 172.21.15.106:0/3150047969 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5cc004c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8009ac0 tx=0x7fe5cc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e15ab640 1 -- 172.21.15.106:0/3150047969 reap_dead start 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e300e640 1 -- 172.21.15.106:0/3150047969 shutdown_connections 2023-11-24T08:08:48.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e300e640 1 -- 172.21.15.106:0/3150047969 >> 172.21.15.106:0/3150047969 conn(0x7fe5dc06a920 msgr2=0x7fe5dc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e300e640 1 -- 172.21.15.106:0/3150047969 shutdown_connections 2023-11-24T08:08:48.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.763+0000 7fe5e300e640 1 -- 172.21.15.106:0/3150047969 wait complete. 2023-11-24T08:08:48.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.765+0000 7fe5e300e640 1 Processor -- start 2023-11-24T08:08:48.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.765+0000 7fe5e300e640 1 -- start start 2023-11-24T08:08:48.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e300e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5dc15e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e300e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5dc15d9e0 0x7fe5dc15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e300e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc1f4760 0x7fe5dc1f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e300e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc099e50 con 0x7fe5cc0047c0 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e300e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc09a0a0 con 0x7fe5dc15d9e0 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e300e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc09a380 con 0x7fe5dc1f4760 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e0daa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5dc15e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e0daa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5dc15e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47736/0 (socket says 172.21.15.106:47736) 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e0daa640 1 -- 172.21.15.106:0/2747144991 learned_addr learned my addr 172.21.15.106:0/2747144991 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5dc15d9e0 0x7fe5dc15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.766+0000 7fe5e0daa640 1 -- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc1f4760 msgr2=0x7fe5dc1f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e0daa640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc1f4760 0x7fe5dc1f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e0daa640 1 -- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5dc15d9e0 msgr2=0x7fe5dc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e0daa640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5dc15d9e0 0x7fe5dc15a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e0daa640 1 -- 172.21.15.106:0/2747144991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc09a660 con 0x7fe5cc0047c0 2023-11-24T08:08:48.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e0daa640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5dc15e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d8b0 tx=0x7fe5d000dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00074e0 con 0x7fe5cc0047c0 2023-11-24T08:08:48.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d0007640 con 0x7fe5cc0047c0 2023-11-24T08:08:48.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc09a940 con 0x7fe5cc0047c0 2023-11-24T08:08:48.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d00049e0 con 0x7fe5cc0047c0 2023-11-24T08:08:48.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.767+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc15aeb0 con 0x7fe5cc0047c0 2023-11-24T08:08:48.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.768+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc067470 con 0x7fe5cc0047c0 2023-11-24T08:08:48.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.769+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe5d0004b40 con 0x7fe5cc0047c0 2023-11-24T08:08:48.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.770+0000 7fe5d97fa640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5c006ae40 0x7fe5c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:48.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.770+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5d00892e0 con 0x7fe5cc0047c0 2023-11-24T08:08:48.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.770+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5c006ae40 0x7fe5c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:48.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.771+0000 7fe5dbfff640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5c006ae40 0x7fe5c006d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8036000 tx=0x7fe5c8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:48.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.774+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5d0054440 con 0x7fe5cc0047c0 2023-11-24T08:08:48.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.950+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5dc0cd730 con 0x7fe5cc0047c0 2023-11-24T08:08:48.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.953+0000 7fe5d97fa640 1 -- 172.21.15.106:0/2747144991 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16566 (secure 0 0 0) 0x7fe5d0057a60 con 0x7fe5cc0047c0 2023-11-24T08:08:48.955 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:48.956 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":21,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:47.586084+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:48.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5c006ae40 msgr2=0x7fe5c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5c006ae40 0x7fe5c006d300 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8036000 tx=0x7fe5c8030000 comp rx=0 tx=0).stop 2023-11-24T08:08:48.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 msgr2=0x7fe5dc15e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:48.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5dc15e2f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe5d000d8b0 tx=0x7fe5d000dd70 comp rx=0 tx=0).stop 2023-11-24T08:08:48.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 shutdown_connections 2023-11-24T08:08:48.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5c006ae40 0x7fe5c006d300 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5dc1f4760 0x7fe5dc1f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5dc15d9e0 0x7fe5dc15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 --2- 172.21.15.106:0/2747144991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5cc0047c0 0x7fe5dc15e2f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:48.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 >> 172.21.15.106:0/2747144991 conn(0x7fe5dc06a920 msgr2=0x7fe5dc0aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:48.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 shutdown_connections 2023-11-24T08:08:48.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:48.956+0000 7fe5e300e640 1 -- 172.21.15.106:0/2747144991 wait complete. 2023-11-24T08:08:48.975 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:08:49.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.191+0000 7f6dea583640 1 Processor -- start 2023-11-24T08:08:49.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.191+0000 7f6dea583640 1 -- start start 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6dea583640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6de40cc0f0 0x7f6de40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6dea583640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6de40c57e0 0x7f6de406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6dea583640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de415eea0 con 0x7f6de415b440 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6dea583640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de415b820 con 0x7f6de40cc0f0 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6dea583640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de40abf30 con 0x7f6de40c57e0 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6de40cc0f0 0x7f6de40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6de40cc0f0 0x7f6de40c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47750/0 (socket says 172.21.15.106:47750) 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 -- 172.21.15.106:0/3532536840 learned_addr learned my addr 172.21.15.106:0/3532536840 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6de8b20640 1 --2- 172.21.15.106:0/3532536840 >> v2:172.21.15.156:3300/0 conn(0x7f6de40c57e0 0x7f6de406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 -- 172.21.15.106:0/3532536840 >> v2:172.21.15.156:3300/0 conn(0x7f6de40c57e0 msgr2=0x7f6de406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 --2- 172.21.15.106:0/3532536840 >> v2:172.21.15.156:3300/0 conn(0x7f6de40c57e0 0x7f6de406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 -- 172.21.15.106:0/3532536840 >> v1:172.21.15.156:6789/0 conn(0x7f6de415b440 legacy=0x7f6de40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6ddbfff640 1 -- 172.21.15.106:0/3532536840 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de415f0f0 con 0x7f6de40cc0f0 2023-11-24T08:08:49.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.193+0000 7f6de8b20640 1 --2- 172.21.15.106:0/3532536840 >> v2:172.21.15.156:3300/0 conn(0x7f6de40c57e0 0x7f6de406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.195+0000 7f6ddbfff640 1 --2- 172.21.15.106:0/3532536840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de40cc0f0 0x7f6de40c52a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6de40c95e0 tx=0x7f6dcc02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=841327cc7ae979a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.195+0000 7f6de37fe640 1 -- 172.21.15.106:0/3532536840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc0317e0 con 0x7f6de40cc0f0 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.195+0000 7f6de37fe640 1 -- 172.21.15.106:0/3532536840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc031940 con 0x7f6de40cc0f0 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.195+0000 7f6de37fe640 1 -- 172.21.15.106:0/3532536840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc031cd0 con 0x7f6de40cc0f0 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.195+0000 7f6dea583640 1 -- 172.21.15.106:0/3532536840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de40cc0f0 msgr2=0x7f6de40c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.195+0000 7f6dea583640 1 --2- 172.21.15.106:0/3532536840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de40cc0f0 0x7f6de40c52a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6de40c95e0 tx=0x7f6dcc02b880 comp rx=0 tx=0).stop 2023-11-24T08:08:49.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.196+0000 7f6dea583640 1 -- 172.21.15.106:0/3532536840 shutdown_connections 2023-11-24T08:08:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.196+0000 7f6dea583640 1 --2- 172.21.15.106:0/3532536840 >> v2:172.21.15.156:3300/0 conn(0x7f6de40c57e0 0x7f6de406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.196+0000 7f6dea583640 1 --2- 172.21.15.106:0/3532536840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de40cc0f0 0x7f6de40c52a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.196+0000 7f6dea583640 1 -- 172.21.15.106:0/3532536840 >> 172.21.15.106:0/3532536840 conn(0x7f6de40b0c50 msgr2=0x7f6de40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.196+0000 7f6dea583640 1 -- 172.21.15.106:0/3532536840 shutdown_connections 2023-11-24T08:08:49.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.196+0000 7f6dea583640 1 -- 172.21.15.106:0/3532536840 wait complete. 2023-11-24T08:08:49.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.197+0000 7f6dea583640 1 Processor -- start 2023-11-24T08:08:49.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.197+0000 7f6dea583640 1 -- start start 2023-11-24T08:08:49.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.198+0000 7f6dea583640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6de40c57e0 0x7f6de4153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.198+0000 7f6dea583640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de40cc0f0 0x7f6de4155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.198+0000 7f6dea583640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 0x7f6de4159800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.198+0000 7f6dea583640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de4157100 con 0x7f6de415b440 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.198+0000 7f6dea583640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41564f0 con 0x7f6de40cc0f0 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.198+0000 7f6dea583640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de41567d0 con 0x7f6de40c57e0 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 0x7f6de4159800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 0x7f6de4159800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47754/0 (socket says 172.21.15.106:47754) 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 -- 172.21.15.106:0/138167779 learned_addr learned my addr 172.21.15.106:0/138167779 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 -- 172.21.15.106:0/138167779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6de40c57e0 msgr2=0x7f6de4153570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6de40c57e0 0x7f6de4153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 -- 172.21.15.106:0/138167779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de40cc0f0 msgr2=0x7f6de4155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:49.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de40cc0f0 0x7f6de4155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 -- 172.21.15.106:0/138167779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc0094d0 con 0x7f6de415b440 2023-11-24T08:08:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.199+0000 7f6de8b20640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 0x7f6de4159800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00cef0 tx=0x7f6ddc00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:49.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.200+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00d840 con 0x7f6de415b440 2023-11-24T08:08:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.200+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ddc00d9c0 con 0x7f6de415b440 2023-11-24T08:08:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.200+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de4156ab0 con 0x7f6de415b440 2023-11-24T08:08:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.200+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ddc00dd70 con 0x7f6de415b440 2023-11-24T08:08:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.200+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de4153ab0 con 0x7f6de415b440 2023-11-24T08:08:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.201+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de404e8b0 con 0x7f6de415b440 2023-11-24T08:08:49.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.202+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6ddc01d360 con 0x7f6de415b440 2023-11-24T08:08:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.203+0000 7f6de17fa640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6dc406ae90 0x7f6dc406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.203+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ddc08a100 con 0x7f6de415b440 2023-11-24T08:08:49.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.203+0000 7f6de3fff640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6dc406ae90 0x7f6dc406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:49.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.203+0000 7f6de3fff640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6dc406ae90 0x7f6dc406d350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006fd0 tx=0x7f6dd4006bc0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:49.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.207+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ddc0552c0 con 0x7f6de415b440 2023-11-24T08:08:49.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:49.424+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6de406dc80 con 0x7f6de415b440 2023-11-24T08:08:50.147 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.145+0000 7f6de17fa640 1 -- 172.21.15.106:0/138167779 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6ddc01a0b0 con 0x7f6de415b440 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6dc406ae90 msgr2=0x7f6dc406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6dc406ae90 0x7f6dc406d350 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4006fd0 tx=0x7f6dd4006bc0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 msgr2=0x7f6de4159800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 0x7f6de4159800 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6ddc00cef0 tx=0x7f6ddc00a710 comp rx=0 tx=0).stop 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 shutdown_connections 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6de415b440 0x7f6de4159800 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6de40cc0f0 0x7f6de4155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6dc406ae90 0x7f6dc406d350 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 --2- 172.21.15.106:0/138167779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6de40c57e0 0x7f6de4153570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 >> 172.21.15.106:0/138167779 conn(0x7f6de40b0c50 msgr2=0x7f6de4154800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:50.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.149+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 shutdown_connections 2023-11-24T08:08:50.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.150+0000 7f6dea583640 1 -- 172.21.15.106:0/138167779 wait complete. 2023-11-24T08:08:50.170 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:08:50.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:08:50.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.381+0000 7f242e8aa640 1 Processor -- start 2023-11-24T08:08:50.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.381+0000 7f242e8aa640 1 -- start start 2023-11-24T08:08:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.382+0000 7f242e8aa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 0x7f24280ca120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242e8aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280ae5d0 con 0x7f24280cce40 2023-11-24T08:08:50.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242e8aa640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24280ae8b0 con 0x7f24280b0b40 2023-11-24T08:08:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242e8aa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f242806cd10 con 0x7f242815d310 2023-11-24T08:08:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242e0a9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f24280cce40 0x7f24280cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54320/0 (socket says 172.21.15.106:54320) 2023-11-24T08:08:50.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242e0a9640 1 -- 172.21.15.106:0/467029216 learned_addr learned my addr 172.21.15.106:0/467029216 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242d8a8640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 0x7f24280ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 553250065 0 0) 0x7f24280ae5d0 con 0x7f24280cce40 2023-11-24T08:08:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.383+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2414003520 con 0x7f24280cce40 2023-11-24T08:08:50.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 132010501 0 0) 0x7f2414003520 con 0x7f24280cce40 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 msgr2=0x7f24280ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 0x7f24280ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 >> v1:172.21.15.156:6789/0 conn(0x7f24280b0b40 legacy=0x7f24280b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242806c650 con 0x7f24280cce40 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2420001670 con 0x7f24280cce40 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 >> v1:172.21.15.106:6789/0 conn(0x7f24280cce40 legacy=0x7f24280cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24140039a0 0x7f2414003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f2414004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 0x7f2414005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f242806cd10 con 0x7f24140047c0 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241400ba30 con 0x7f24140039a0 2023-11-24T08:08:50.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241400bbf0 con 0x7f242815d310 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242d0a7640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f2414004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242d8a8640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24140039a0 0x7f2414003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242d0a7640 1 -- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 msgr2=0x7f2414005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242e0a9640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 0x7f2414005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.384+0000 7f242d0a7640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815d310 0x7f2414005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.385+0000 7f242d0a7640 1 -- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24140039a0 msgr2=0x7f2414003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.385+0000 7f242d0a7640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24140039a0 0x7f2414003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.385+0000 7f242d0a7640 1 -- 172.21.15.106:0/467029216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414003520 con 0x7f24140047c0 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.385+0000 7f242d8a8640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24140039a0 0x7f2414003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242d0a7640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f2414004c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f241c001e20 tx=0x7f24140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb22716a98396fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c003b70 con 0x7f24140047c0 2023-11-24T08:08:50.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241c003cd0 con 0x7f24140047c0 2023-11-24T08:08:50.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242c8a6640 1 -- 172.21.15.106:0/467029216 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241c005310 con 0x7f24140047c0 2023-11-24T08:08:50.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e8aa640 1 -- 172.21.15.106:0/467029216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 msgr2=0x7f2414004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e8aa640 1 --2- 172.21.15.106:0/467029216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f2414004c40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f241c001e20 tx=0x7f24140057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e0a9640 1 -- 172.21.15.106:0/467029216 reap_dead start 2023-11-24T08:08:50.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e8aa640 1 -- 172.21.15.106:0/467029216 shutdown_connections 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e8aa640 1 -- 172.21.15.106:0/467029216 >> 172.21.15.106:0/467029216 conn(0x7f24280cbb40 msgr2=0x7f24280cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e8aa640 1 -- 172.21.15.106:0/467029216 shutdown_connections 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.386+0000 7f242e8aa640 1 -- 172.21.15.106:0/467029216 wait complete. 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.388+0000 7f242e8aa640 1 Processor -- start 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.388+0000 7f242e8aa640 1 -- start start 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242e8aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f24280c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242e8aa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815c250 0x7f242815ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242e8aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24280c3f00 0x7f2428071800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242e8aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24281e8990 con 0x7f24140047c0 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242e8aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c44c0 con 0x7f24280c3f00 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242e8aa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24280c47a0 con 0x7f242815c250 2023-11-24T08:08:50.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242d8a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f24280c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242d8a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f24280c5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47770/0 (socket says 172.21.15.106:47770) 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.389+0000 7f242d8a8640 1 -- 172.21.15.106:0/1399513505 learned_addr learned my addr 172.21.15.106:0/1399513505 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242d8a8640 1 -- 172.21.15.106:0/1399513505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815c250 msgr2=0x7f242815ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242d8a8640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815c250 0x7f242815ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242d8a8640 1 -- 172.21.15.106:0/1399513505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24280c3f00 msgr2=0x7f2428071800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242d8a8640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24280c3f00 0x7f2428071800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242d8a8640 1 -- 172.21.15.106:0/1399513505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24280c4a80 con 0x7f24140047c0 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242d8a8640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f24280c5560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24240108b0 tx=0x7f2424010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242400eb60 con 0x7f24140047c0 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242400ecc0 con 0x7f24140047c0 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24280c4d60 con 0x7f24140047c0 2023-11-24T08:08:50.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242401c400 con 0x7f24140047c0 2023-11-24T08:08:50.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.390+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f242815c9d0 con 0x7f24140047c0 2023-11-24T08:08:50.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.391+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f242804e840 con 0x7f24140047c0 2023-11-24T08:08:50.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.392+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f242401c560 con 0x7f24140047c0 2023-11-24T08:08:50.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.393+0000 7f241a7fc640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240c06ae40 0x7f240c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:50.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.393+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2424089cd0 con 0x7f24140047c0 2023-11-24T08:08:50.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.393+0000 7f242d0a7640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240c06ae40 0x7f240c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:50.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.394+0000 7f242d0a7640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240c06ae40 0x7f240c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:50.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.397+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2424054d90 con 0x7f24140047c0 2023-11-24T08:08:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:50.629+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f242806e870 con 0x7f24140047c0 2023-11-24T08:08:51.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.148+0000 7f241a7fc640 1 -- 172.21.15.106:0/1399513505 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v7) v1 ==== 122+0+0 (secure 0 0 0) 0x7f24240583b0 con 0x7f24140047c0 2023-11-24T08:08:51.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240c06ae40 msgr2=0x7f240c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240c06ae40 0x7f240c06d300 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f241c034000 tx=0x7f241c030000 comp rx=0 tx=0).stop 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 msgr2=0x7f24280c5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f24280c5560 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f24240108b0 tx=0x7f2424010d70 comp rx=0 tx=0).stop 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 shutdown_connections 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24280c3f00 0x7f2428071800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240c06ae40 0x7f240c06d300 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f242815c250 0x7f242815ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 --2- 172.21.15.106:0/1399513505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f24140047c0 0x7f24280c5560 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 >> 172.21.15.106:0/1399513505 conn(0x7f24280cbb40 msgr2=0x7f2428070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 shutdown_connections 2023-11-24T08:08:51.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.153+0000 7f242e8aa640 1 -- 172.21.15.106:0/1399513505 wait complete. 2023-11-24T08:08:51.175 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:08:51.175 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:08:51.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.383+0000 7f6b950b2640 1 Processor -- start 2023-11-24T08:08:51.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.383+0000 7f6b950b2640 1 -- start start 2023-11-24T08:08:51.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.384+0000 7f6b950b2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b900b21d0 0x7f6b900cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.384+0000 7f6b950b2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900b3a40 con 0x7f6b900cbf80 2023-11-24T08:08:51.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.384+0000 7f6b950b2640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b900cc3d0 con 0x7f6b90159ce0 2023-11-24T08:08:51.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.385+0000 7f6b950b2640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b900cdfa0 con 0x7f6b900b21d0 2023-11-24T08:08:51.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.385+0000 7f6b8f59e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6b900cbf80 0x7f6b9006c560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54322/0 (socket says 172.21.15.106:54322) 2023-11-24T08:08:51.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.385+0000 7f6b8f59e640 1 -- 172.21.15.106:0/903377177 learned_addr learned my addr 172.21.15.106:0/903377177 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:51.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.385+0000 7f6b8e59c640 1 --2- 172.21.15.106:0/903377177 >> v2:172.21.15.156:3300/0 conn(0x7f6b900b21d0 0x7f6b900cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.385+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559235943 0 0) 0x7f6b900b3a40 con 0x7f6b900cbf80 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.385+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7c003520 con 0x7f6b900cbf80 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3718463994 0 0) 0x7f6b7c003520 con 0x7f6b900cbf80 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 >> v2:172.21.15.156:3300/0 conn(0x7f6b900b21d0 msgr2=0x7f6b900cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 --2- 172.21.15.106:0/903377177 >> v2:172.21.15.156:3300/0 conn(0x7f6b900b21d0 0x7f6b900cba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 >> v1:172.21.15.156:6789/0 conn(0x7f6b90159ce0 legacy=0x7f6b9006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b900cd9e0 con 0x7f6b900cbf80 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f6b88001670 con 0x7f6b900cbf80 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 >> v1:172.21.15.106:6789/0 conn(0x7f6b900cbf80 legacy=0x7f6b9006c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c003970 0x7f6b7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7c0047c0 0x7f6b7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b7c005180 0x7f6b7c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b900cdfa0 con 0x7f6b7c005180 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c00e230 con 0x7f6b7c0047c0 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c00e3f0 con 0x7f6b7c003970 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8f59e640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b7c005180 0x7f6b7c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8ed9d640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c003970 0x7f6b7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.386+0000 7f6b8e59c640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7c0047c0 0x7f6b7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.387+0000 7f6b8f59e640 1 -- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c003970 msgr2=0x7f6b7c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.387+0000 7f6b8f59e640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c003970 0x7f6b7c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.387+0000 7f6b8f59e640 1 -- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7c0047c0 msgr2=0x7f6b7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.387+0000 7f6b8f59e640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7c0047c0 0x7f6b7c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.387+0000 7f6b8f59e640 1 -- 172.21.15.106:0/903377177 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c003520 con 0x7f6b7c005180 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.388+0000 7f6b8f59e640 1 -- 172.21.15.106:0/903377177 reap_dead start 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.388+0000 7f6b8f59e640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b7c005180 0x7f6b7c00db90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c240 tx=0x7f6b7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59619a8bc84db2c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.388+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b88005ad0 con 0x7f6b7c005180 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.388+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b88005c70 con 0x7f6b7c005180 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.388+0000 7f6b8dd9b640 1 -- 172.21.15.106:0/903377177 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8802c8b0 con 0x7f6b7c005180 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 -- 172.21.15.106:0/903377177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b7c005180 msgr2=0x7f6b7c00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b7c005180 0x7f6b7c00db90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f6b8800c240 tx=0x7f6b7c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 -- 172.21.15.106:0/903377177 shutdown_connections 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 --2- 172.21.15.106:0/903377177 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b7c005180 0x7f6b7c00db90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 -- 172.21.15.106:0/903377177 >> 172.21.15.106:0/903377177 conn(0x7f6b9006a920 msgr2=0x7f6b9006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 -- 172.21.15.106:0/903377177 shutdown_connections 2023-11-24T08:08:51.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.389+0000 7f6b950b2640 1 -- 172.21.15.106:0/903377177 wait complete. 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.391+0000 7f6b950b2640 1 Processor -- start 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.391+0000 7f6b950b2640 1 -- start start 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b950b2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c005180 0x7f6b90153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b950b2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 0x7f6b9015b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b950b2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b9009b790 0x7f6b901540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b950b2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9015d220 con 0x7f6b9009b310 2023-11-24T08:08:51.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b950b2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9006d160 con 0x7f6b9009b790 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b950b2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b9006d440 con 0x7f6b7c005180 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b8e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 0x7f6b9015b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b8e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 0x7f6b9015b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47782/0 (socket says 172.21.15.106:47782) 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b8e59c640 1 -- 172.21.15.106:0/2863009115 learned_addr learned my addr 172.21.15.106:0/2863009115 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b8ed9d640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c005180 0x7f6b90153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.392+0000 7f6b8f59e640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b9009b790 0x7f6b901540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8e59c640 1 -- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c005180 msgr2=0x7f6b90153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8e59c640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c005180 0x7f6b90153730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8e59c640 1 -- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b9009b790 msgr2=0x7f6b901540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8e59c640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b9009b790 0x7f6b901540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8e59c640 1 -- 172.21.15.106:0/2863009115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b9006d720 con 0x7f6b9009b310 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8ed9d640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c005180 0x7f6b90153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b8e59c640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 0x7f6b9015b120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b78011540 tx=0x7f6b78011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78015930 con 0x7f6b9009b310 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b78015a90 con 0x7f6b9009b310 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b78015e60 con 0x7f6b9009b310 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b90153c70 con 0x7f6b9009b310 2023-11-24T08:08:51.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.393+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9006cdc0 con 0x7f6b9009b310 2023-11-24T08:08:51.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.395+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b9015d360 con 0x7f6b9009b310 2023-11-24T08:08:51.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.395+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6b7800f460 con 0x7f6b9009b310 2023-11-24T08:08:51.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.396+0000 7f6b877fe640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b7006ae40 0x7f6b7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.396+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6b78089e30 con 0x7f6b9009b310 2023-11-24T08:08:51.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.396+0000 7f6b8ed9d640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b7006ae40 0x7f6b7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.397+0000 7f6b8ed9d640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b7006ae40 0x7f6b7006d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b80001d40 tx=0x7f6b80009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.401+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b78054ed0 con 0x7f6b9009b310 2023-11-24T08:08:51.534 INFO:tasks.ceph.mds.d.smithi156.stderr:2023-11-24T08:08:51.531+0000 7f3d5eb4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:08:51.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.628+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b9015cfb0 con 0x7f6b9009b310 2023-11-24T08:08:51.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.629+0000 7f6b877fe640 1 -- 172.21.15.106:0/2863009115 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f6b780584f0 con 0x7f6b9009b310 2023-11-24T08:08:51.648 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:51.648 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:08:51.649 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:08:51.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b7006ae40 msgr2=0x7f6b7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b7006ae40 0x7f6b7006d300 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6b80001d40 tx=0x7f6b80009040 comp rx=0 tx=0).stop 2023-11-24T08:08:51.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 msgr2=0x7f6b9015b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 0x7f6b9015b120 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6b78011540 tx=0x7f6b78011a00 comp rx=0 tx=0).stop 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 shutdown_connections 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b9009b790 0x7f6b901540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b9009b310 0x7f6b9015b120 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b7006ae40 0x7f6b7006d300 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 --2- 172.21.15.106:0/2863009115 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7c005180 0x7f6b90153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 >> 172.21.15.106:0/2863009115 conn(0x7f6b9006a920 msgr2=0x7f6b900b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 shutdown_connections 2023-11-24T08:08:51.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.632+0000 7f6b950b2640 1 -- 172.21.15.106:0/2863009115 wait complete. 2023-11-24T08:08:51.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:51.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.857+0000 7faf388bb640 1 Processor -- start 2023-11-24T08:08:51.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.857+0000 7faf388bb640 1 -- start start 2023-11-24T08:08:51.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.858+0000 7faf388bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 0x7faf3415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.858+0000 7faf388bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3415ab80 con 0x7faf340b2630 2023-11-24T08:08:51.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf388bb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf3415e680 con 0x7faf340b2a70 2023-11-24T08:08:51.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf388bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3415a990 con 0x7faf340b2250 2023-11-24T08:08:51.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf2bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7faf340b2630 0x7faf340ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54338/0 (socket says 172.21.15.106:54338) 2023-11-24T08:08:51.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf2bfff640 1 -- 172.21.15.106:0/3150531066 learned_addr learned my addr 172.21.15.106:0/3150531066 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:51.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf3259c640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 0x7faf3415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 32143947 0 0) 0x7faf3415ab80 con 0x7faf340b2630 2023-11-24T08:08:51.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf20003520 con 0x7faf340b2630 2023-11-24T08:08:51.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3565581487 0 0) 0x7faf20003520 con 0x7faf340b2630 2023-11-24T08:08:51.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 msgr2=0x7faf3415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf31d9b640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 0x7faf3415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.859+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 >> v1:172.21.15.156:6789/0 conn(0x7faf340b2a70 legacy=0x7faf3415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3415e960 con 0x7faf340b2630 2023-11-24T08:08:51.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7faf1c001670 con 0x7faf340b2630 2023-11-24T08:08:51.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 >> v1:172.21.15.106:6789/0 conn(0x7faf340b2630 legacy=0x7faf340ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 0x7faf200039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20003fd0 0x7faf200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf20004d00 0x7faf20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2000bb10 con 0x7faf20004d00 2023-11-24T08:08:51.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2000bc80 con 0x7faf20003fd0 2023-11-24T08:08:51.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2000be40 con 0x7faf340b2250 2023-11-24T08:08:51.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf32d9d640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf20004d00 0x7faf20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf3259c640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 0x7faf200039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf2bfff640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20003fd0 0x7faf200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf32d9d640 1 -- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 msgr2=0x7faf200039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf32d9d640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf340b2250 0x7faf200039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf32d9d640 1 -- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20003fd0 msgr2=0x7faf200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:51.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf32d9d640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20003fd0 0x7faf200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf32d9d640 1 -- 172.21.15.106:0/3150531066 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20003520 con 0x7faf20004d00 2023-11-24T08:08:51.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.860+0000 7faf2bfff640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20003fd0 0x7faf200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:51.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.861+0000 7faf32d9d640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf20004d00 0x7faf20005180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf2c001a30 tx=0x7faf200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1fcced7d1018cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c003b70 con 0x7faf20004d00 2023-11-24T08:08:51.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c003cd0 con 0x7faf20004d00 2023-11-24T08:08:51.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf31d9b640 1 -- 172.21.15.106:0/3150531066 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c005310 con 0x7faf20004d00 2023-11-24T08:08:51.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf388bb640 1 -- 172.21.15.106:0/3150531066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf20004d00 msgr2=0x7faf20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf388bb640 1 --2- 172.21.15.106:0/3150531066 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf20004d00 0x7faf20005180 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf2c001a30 tx=0x7faf200057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf32d9d640 1 -- 172.21.15.106:0/3150531066 reap_dead start 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf388bb640 1 -- 172.21.15.106:0/3150531066 shutdown_connections 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf388bb640 1 -- 172.21.15.106:0/3150531066 >> 172.21.15.106:0/3150531066 conn(0x7faf3406a920 msgr2=0x7faf3406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf388bb640 1 -- 172.21.15.106:0/3150531066 shutdown_connections 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.862+0000 7faf388bb640 1 -- 172.21.15.106:0/3150531066 wait complete. 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.864+0000 7faf388bb640 1 Processor -- start 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.864+0000 7faf388bb640 1 -- start start 2023-11-24T08:08:51.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf388bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf20004d00 0x7faf340afe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf388bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf341f2f00 0x7faf34156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf388bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 0x7faf34071610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf388bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf341574d0 con 0x7faf34157050 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf388bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf340aeb50 con 0x7faf341f2f00 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf388bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf340af9b0 con 0x7faf20004d00 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.865+0000 7faf32d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 0x7faf34071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 0x7faf34071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47802/0 (socket says 172.21.15.106:47802) 2023-11-24T08:08:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 -- 172.21.15.106:0/2755152000 learned_addr learned my addr 172.21.15.106:0/2755152000 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 -- 172.21.15.106:0/2755152000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf20004d00 msgr2=0x7faf340afe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf20004d00 0x7faf340afe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 -- 172.21.15.106:0/2755152000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf341f2f00 msgr2=0x7faf34156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf341f2f00 0x7faf34156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 -- 172.21.15.106:0/2755152000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf340cd7c0 con 0x7faf34157050 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf32d9d640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 0x7faf34071610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02e270 tx=0x7faf2c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c00f1d0 con 0x7faf34157050 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c007db0 con 0x7faf34157050 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf341f3380 con 0x7faf34157050 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.866+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c005810 con 0x7faf34157050 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.867+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf340af080 con 0x7faf34157050 2023-11-24T08:08:51.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.868+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf340af080 con 0x7faf34157050 2023-11-24T08:08:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.869+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7faf2c02e600 con 0x7faf34157050 2023-11-24T08:08:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.869+0000 7faf2affd640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faf1406ae70 0x7faf1406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.870+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf2c0aa5f0 con 0x7faf34157050 2023-11-24T08:08:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.870+0000 7faf3259c640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faf1406ae70 0x7faf1406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:51.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.870+0000 7faf3259c640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faf1406ae70 0x7faf1406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf24006fd0 tx=0x7faf2400b2e0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:51.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:51.874+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faf2c075680 con 0x7faf34157050 2023-11-24T08:08:52.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.065+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf340af080 con 0x7faf34157050 2023-11-24T08:08:52.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.068+0000 7faf2affd640 1 -- 172.21.15.106:0/2755152000 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7faf2c078ca0 con 0x7faf34157050 2023-11-24T08:08:52.070 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:52.070 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.070+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faf1406ae70 msgr2=0x7faf1406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.070+0000 7faf388bb640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faf1406ae70 0x7faf1406d330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faf24006fd0 tx=0x7faf2400b2e0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.070+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 msgr2=0x7faf34071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.070+0000 7faf388bb640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 0x7faf34071610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7faf2c02e270 tx=0x7faf2c0014a0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 shutdown_connections 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faf34157050 0x7faf34071610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf341f2f00 0x7faf34156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faf1406ae70 0x7faf1406d330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 --2- 172.21.15.106:0/2755152000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faf20004d00 0x7faf340afe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 >> 172.21.15.106:0/2755152000 conn(0x7faf3406a920 msgr2=0x7faf340734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 shutdown_connections 2023-11-24T08:08:52.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.071+0000 7faf388bb640 1 -- 172.21.15.106:0/2755152000 wait complete. 2023-11-24T08:08:52.092 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:08:52.092 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:52.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.305+0000 7f590759e640 1 Processor -- start 2023-11-24T08:08:52.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.306+0000 7f590759e640 1 -- start start 2023-11-24T08:08:52.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f590759e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f590815d4e0 0x7f59080d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f590759e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d38e0 0x7f59080cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f590759e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f59080d0070 0x7f59080d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f590759e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080d4110 con 0x7f59080d0070 2023-11-24T08:08:52.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f590759e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59080d04e0 con 0x7f590815d4e0 2023-11-24T08:08:52.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f590759e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59080ce990 con 0x7f59080d38e0 2023-11-24T08:08:52.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f5906d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f59080d0070 0x7f59080d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f5906d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f59080d0070 0x7f59080d0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47808/0 (socket says 172.21.15.106:47808) 2023-11-24T08:08:52.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.307+0000 7f5906d9d640 1 -- 172.21.15.106:0/1111608679 learned_addr learned my addr 172.21.15.106:0/1111608679 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f58fffff640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d38e0 0x7f59080cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f590659c640 1 --2- 172.21.15.106:0/1111608679 >> v2:172.21.15.156:3300/0 conn(0x7f590815d4e0 0x7f59080d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f5906d9d640 1 -- 172.21.15.106:0/1111608679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d38e0 msgr2=0x7f59080cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f5906d9d640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d38e0 0x7f59080cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f5906d9d640 1 -- 172.21.15.106:0/1111608679 >> v2:172.21.15.156:3300/0 conn(0x7f590815d4e0 msgr2=0x7f59080d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f5906d9d640 1 --2- 172.21.15.106:0/1111608679 >> v2:172.21.15.156:3300/0 conn(0x7f590815d4e0 0x7f59080d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f5906d9d640 1 -- 172.21.15.106:0/1111608679 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080cec70 con 0x7f59080d0070 2023-11-24T08:08:52.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.308+0000 7f58fffff640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d38e0 0x7f59080cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.309+0000 7f5906d9d640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d0070 0x7f59080d0da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f590808b800 tx=0x7f590002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bb2391894be656b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.309+0000 7f5905d9b640 1 -- 172.21.15.106:0/1111608679 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590002ca70 con 0x7f59080d0070 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.309+0000 7f5905d9b640 1 -- 172.21.15.106:0/1111608679 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f590002cbd0 con 0x7f59080d0070 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.309+0000 7f5905d9b640 1 -- 172.21.15.106:0/1111608679 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59000343f0 con 0x7f59080d0070 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 -- 172.21.15.106:0/1111608679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d0070 msgr2=0x7f59080d0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d0070 0x7f59080d0da0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f590808b800 tx=0x7f590002bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 -- 172.21.15.106:0/1111608679 shutdown_connections 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d0070 0x7f59080d0da0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 --2- 172.21.15.106:0/1111608679 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d38e0 0x7f59080cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 --2- 172.21.15.106:0/1111608679 >> v2:172.21.15.156:3300/0 conn(0x7f590815d4e0 0x7f59080d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 -- 172.21.15.106:0/1111608679 >> 172.21.15.106:0/1111608679 conn(0x7f590809c510 msgr2=0x7f59080d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 -- 172.21.15.106:0/1111608679 shutdown_connections 2023-11-24T08:08:52.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.310+0000 7f590759e640 1 -- 172.21.15.106:0/1111608679 wait complete. 2023-11-24T08:08:52.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.312+0000 7f590759e640 1 Processor -- start 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.312+0000 7f590759e640 1 -- start start 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590759e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d0070 0x7f5908159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 0x7f59080c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590759e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f590815d4e0 0x7f59080b8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590759e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f590815a160 con 0x7f59080d38e0 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590759e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59081f3fc0 con 0x7f590815d4e0 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590759e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59081f33b0 con 0x7f59080d0070 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 0x7f59080c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 0x7f59080c73a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47818/0 (socket says 172.21.15.106:47818) 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 -- 172.21.15.106:0/2780958997 learned_addr learned my addr 172.21.15.106:0/2780958997 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:52.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f590659c640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d0070 0x7f5908159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f5906d9d640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f590815d4e0 0x7f59080b8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 -- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d0070 msgr2=0x7f5908159c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d0070 0x7f5908159c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 -- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f590815d4e0 msgr2=0x7f59080b8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f590815d4e0 0x7f59080b8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f58fffff640 1 -- 172.21.15.106:0/2780958997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900009b30 con 0x7f59080d38e0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.313+0000 7f5906d9d640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f590815d4e0 0x7f59080b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.314+0000 7f58fffff640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 0x7f59080c73a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c7c0 tx=0x7f58f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.314+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007a30 con 0x7f59080d38e0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.314+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0007b90 con 0x7f59080d38e0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.314+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59081f3690 con 0x7f59080d38e0 2023-11-24T08:08:52.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.314+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000ce70 con 0x7f59080d38e0 2023-11-24T08:08:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.314+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59081f3c50 con 0x7f59080d38e0 2023-11-24T08:08:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.315+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f590815ea90 con 0x7f59080d38e0 2023-11-24T08:08:52.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.316+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f58f00113f0 con 0x7f59080d38e0 2023-11-24T08:08:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.317+0000 7f58feffd640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f58e806ae40 0x7f58e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.317+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58f00897a0 con 0x7f59080d38e0 2023-11-24T08:08:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.317+0000 7f590659c640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f58e806ae40 0x7f58e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.318+0000 7f590659c640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f58e806ae40 0x7f58e806d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58f8006fd0 tx=0x7f58f8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.321+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58f00547b0 con 0x7f59080d38e0 2023-11-24T08:08:52.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.511+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f590815d8c0 con 0x7f58e806ae40 2023-11-24T08:08:52.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.526+0000 7f58feffd640 1 -- 172.21.15.106:0/2780958997 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264506 (secure 0 0 0) 0x7f58f800b5e0 con 0x7f58e806ae40 2023-11-24T08:08:52.532 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f58e806ae40 msgr2=0x7f58e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f58e806ae40 0x7f58e806d300 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f58f8006fd0 tx=0x7f58f8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 msgr2=0x7f59080c73a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 0x7f59080c73a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f58f000c7c0 tx=0x7f58f000cc80 comp rx=0 tx=0).stop 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 shutdown_connections 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f590815d4e0 0x7f59080b8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59080d38e0 0x7f59080c73a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f58e806ae40 0x7f58e806d300 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 --2- 172.21.15.106:0/2780958997 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59080d0070 0x7f5908159c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 >> 172.21.15.106:0/2780958997 conn(0x7f590809c510 msgr2=0x7f59080b1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 shutdown_connections 2023-11-24T08:08:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.533+0000 7f590759e640 1 -- 172.21.15.106:0/2780958997 wait complete. 2023-11-24T08:08:52.555 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-24T08:08:51.274961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235912,"kb_used_data":2948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738808,"statfs":{"total":773094113280,"available":772852539392,"internally_reserved":0,"allocated":3018752,"data_stored":1797664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178582"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354084+0000","last_change":"2023-11-24T08:08:48.155148+0000","last_active":"2023-11-24T08:08:49.354084+0000","last_peered":"2023-11-24T08:08:49.354084+0000","last_clean":"2023-11-24T08:08:49.354084+0000","last_became_active":"2023-11-24T08:08:48.148725+0000","last_became_peered":"2023-11-24T08:08:48.148725+0000","last_unstale":"2023-11-24T08:08:49.354084+0000","last_undegraded":"2023-11-24T08:08:49.354084+0000","last_fullsized":"2023-11-24T08:08:49.354084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:46.218050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126326+0000","last_change":"2023-11-24T08:08:48.153462+0000","last_active":"2023-11-24T08:08:49.126326+0000","last_peered":"2023-11-24T08:08:49.126326+0000","last_clean":"2023-11-24T08:08:49.126326+0000","last_became_active":"2023-11-24T08:08:48.153009+0000","last_became_peered":"2023-11-24T08:08:48.153009+0000","last_unstale":"2023-11-24T08:08:49.126326+0000","last_undegraded":"2023-11-24T08:08:49.126326+0000","last_fullsized":"2023-11-24T08:08:49.126326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.154972+0000","last_change":"2023-11-24T08:08:48.147649+0000","last_active":"2023-11-24T08:08:48.154972+0000","last_peered":"2023-11-24T08:08:48.154972+0000","last_clean":"2023-11-24T08:08:48.154972+0000","last_became_active":"2023-11-24T08:08:48.147280+0000","last_became_peered":"2023-11-24T08:08:48.147280+0000","last_unstale":"2023-11-24T08:08:48.154972+0000","last_undegraded":"2023-11-24T08:08:48.154972+0000","last_fullsized":"2023-11-24T08:08:48.154972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:50.529015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126469+0000","last_change":"2023-11-24T08:08:48.158446+0000","last_active":"2023-11-24T08:08:49.126469+0000","last_peered":"2023-11-24T08:08:49.126469+0000","last_clean":"2023-11-24T08:08:49.126469+0000","last_became_active":"2023-11-24T08:08:48.158196+0000","last_became_peered":"2023-11-24T08:08:48.158196+0000","last_unstale":"2023-11-24T08:08:49.126469+0000","last_undegraded":"2023-11-24T08:08:49.126469+0000","last_fullsized":"2023-11-24T08:08:49.126469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:58.752582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353187+0000","last_change":"2023-11-24T08:08:48.148128+0000","last_active":"2023-11-24T08:08:49.353187+0000","last_peered":"2023-11-24T08:08:49.353187+0000","last_clean":"2023-11-24T08:08:49.353187+0000","last_became_active":"2023-11-24T08:08:48.147789+0000","last_became_peered":"2023-11-24T08:08:48.147789+0000","last_unstale":"2023-11-24T08:08:49.353187+0000","last_undegraded":"2023-11-24T08:08:49.353187+0000","last_fullsized":"2023-11-24T08:08:49.353187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:17.852217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126840+0000","last_change":"2023-11-24T08:08:48.160813+0000","last_active":"2023-11-24T08:08:49.126840+0000","last_peered":"2023-11-24T08:08:49.126840+0000","last_clean":"2023-11-24T08:08:49.126840+0000","last_became_active":"2023-11-24T08:08:48.159970+0000","last_became_peered":"2023-11-24T08:08:48.159970+0000","last_unstale":"2023-11-24T08:08:49.126840+0000","last_undegraded":"2023-11-24T08:08:49.126840+0000","last_fullsized":"2023-11-24T08:08:49.126840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:43.997384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352936+0000","last_change":"2023-11-24T08:08:48.147161+0000","last_active":"2023-11-24T08:08:49.352936+0000","last_peered":"2023-11-24T08:08:49.352936+0000","last_clean":"2023-11-24T08:08:49.352936+0000","last_became_active":"2023-11-24T08:08:48.146867+0000","last_became_peered":"2023-11-24T08:08:48.146867+0000","last_unstale":"2023-11-24T08:08:49.352936+0000","last_undegraded":"2023-11-24T08:08:49.352936+0000","last_fullsized":"2023-11-24T08:08:49.352936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:22:50.592771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.161892+0000","last_change":"2023-11-24T08:08:48.151256+0000","last_active":"2023-11-24T08:08:48.161892+0000","last_peered":"2023-11-24T08:08:48.161892+0000","last_clean":"2023-11-24T08:08:48.161892+0000","last_became_active":"2023-11-24T08:08:48.150977+0000","last_became_peered":"2023-11-24T08:08:48.150977+0000","last_unstale":"2023-11-24T08:08:48.161892+0000","last_undegraded":"2023-11-24T08:08:48.161892+0000","last_fullsized":"2023-11-24T08:08:48.161892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:50.529015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354145+0000","last_change":"2023-11-24T08:08:48.165291+0000","last_active":"2023-11-24T08:08:49.354145+0000","last_peered":"2023-11-24T08:08:49.354145+0000","last_clean":"2023-11-24T08:08:49.354145+0000","last_became_active":"2023-11-24T08:08:48.164956+0000","last_became_peered":"2023-11-24T08:08:48.164956+0000","last_unstale":"2023-11-24T08:08:49.354145+0000","last_undegraded":"2023-11-24T08:08:49.354145+0000","last_fullsized":"2023-11-24T08:08:49.354145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:32.727731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125355+0000","last_change":"2023-11-24T08:08:48.159627+0000","last_active":"2023-11-24T08:08:49.125355+0000","last_peered":"2023-11-24T08:08:49.125355+0000","last_clean":"2023-11-24T08:08:49.125355+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:49.125355+0000","last_undegraded":"2023-11-24T08:08:49.125355+0000","last_fullsized":"2023-11-24T08:08:49.125355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125146+0000","last_change":"2023-11-24T08:08:48.157396+0000","last_active":"2023-11-24T08:08:49.125146+0000","last_peered":"2023-11-24T08:08:49.125146+0000","last_clean":"2023-11-24T08:08:49.125146+0000","last_became_active":"2023-11-24T08:08:48.155190+0000","last_became_peered":"2023-11-24T08:08:48.155190+0000","last_unstale":"2023-11-24T08:08:49.125146+0000","last_undegraded":"2023-11-24T08:08:49.125146+0000","last_fullsized":"2023-11-24T08:08:49.125146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107513+0000","last_change":"2023-11-24T08:08:48.139798+0000","last_active":"2023-11-24T08:08:50.107513+0000","last_peered":"2023-11-24T08:08:50.107513+0000","last_clean":"2023-11-24T08:08:50.107513+0000","last_became_active":"2023-11-24T08:08:48.139186+0000","last_became_peered":"2023-11-24T08:08:48.139186+0000","last_unstale":"2023-11-24T08:08:50.107513+0000","last_undegraded":"2023-11-24T08:08:50.107513+0000","last_fullsized":"2023-11-24T08:08:50.107513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.146745+0000","last_change":"2023-11-24T08:08:48.141667+0000","last_active":"2023-11-24T08:08:48.146745+0000","last_peered":"2023-11-24T08:08:48.146745+0000","last_clean":"2023-11-24T08:08:48.146745+0000","last_became_active":"2023-11-24T08:08:48.141410+0000","last_became_peered":"2023-11-24T08:08:48.141410+0000","last_unstale":"2023-11-24T08:08:48.146745+0000","last_undegraded":"2023-11-24T08:08:48.146745+0000","last_fullsized":"2023-11-24T08:08:48.146745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164172+0000","last_change":"2023-11-24T08:08:48.154158+0000","last_active":"2023-11-24T08:08:48.164172+0000","last_peered":"2023-11-24T08:08:48.164172+0000","last_clean":"2023-11-24T08:08:48.164172+0000","last_became_active":"2023-11-24T08:08:48.153833+0000","last_became_peered":"2023-11-24T08:08:48.153833+0000","last_unstale":"2023-11-24T08:08:48.164172+0000","last_undegraded":"2023-11-24T08:08:48.164172+0000","last_fullsized":"2023-11-24T08:08:48.164172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125503+0000","last_change":"2023-11-24T08:08:48.130056+0000","last_active":"2023-11-24T08:08:49.125503+0000","last_peered":"2023-11-24T08:08:49.125503+0000","last_clean":"2023-11-24T08:08:49.125503+0000","last_became_active":"2023-11-24T08:08:48.129739+0000","last_became_peered":"2023-11-24T08:08:48.129739+0000","last_unstale":"2023-11-24T08:08:49.125503+0000","last_undegraded":"2023-11-24T08:08:49.125503+0000","last_fullsized":"2023-11-24T08:08:49.125503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125661+0000","last_change":"2023-11-24T08:08:48.157064+0000","last_active":"2023-11-24T08:08:49.125661+0000","last_peered":"2023-11-24T08:08:49.125661+0000","last_clean":"2023-11-24T08:08:49.125661+0000","last_became_active":"2023-11-24T08:08:48.153014+0000","last_became_peered":"2023-11-24T08:08:48.153014+0000","last_unstale":"2023-11-24T08:08:49.125661+0000","last_undegraded":"2023-11-24T08:08:49.125661+0000","last_fullsized":"2023-11-24T08:08:49.125661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:53.228655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128174+0000","last_change":"2023-11-24T08:08:48.168538+0000","last_active":"2023-11-24T08:08:49.128174+0000","last_peered":"2023-11-24T08:08:49.128174+0000","last_clean":"2023-11-24T08:08:49.128174+0000","last_became_active":"2023-11-24T08:08:48.168251+0000","last_became_peered":"2023-11-24T08:08:48.168251+0000","last_unstale":"2023-11-24T08:08:49.128174+0000","last_undegraded":"2023-11-24T08:08:49.128174+0000","last_fullsized":"2023-11-24T08:08:49.128174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:34.330436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.136935+0000","last_change":"2023-11-24T08:08:48.135832+0000","last_active":"2023-11-24T08:08:48.136935+0000","last_peered":"2023-11-24T08:08:48.136935+0000","last_clean":"2023-11-24T08:08:48.136935+0000","last_became_active":"2023-11-24T08:08:48.135419+0000","last_became_peered":"2023-11-24T08:08:48.135419+0000","last_unstale":"2023-11-24T08:08:48.136935+0000","last_undegraded":"2023-11-24T08:08:48.136935+0000","last_fullsized":"2023-11-24T08:08:48.136935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:22.547643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353897+0000","last_change":"2023-11-24T08:08:48.148026+0000","last_active":"2023-11-24T08:08:49.353897+0000","last_peered":"2023-11-24T08:08:49.353897+0000","last_clean":"2023-11-24T08:08:49.353897+0000","last_became_active":"2023-11-24T08:08:48.147722+0000","last_became_peered":"2023-11-24T08:08:48.147722+0000","last_unstale":"2023-11-24T08:08:49.353897+0000","last_undegraded":"2023-11-24T08:08:49.353897+0000","last_fullsized":"2023-11-24T08:08:49.353897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:15:57.275173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128304+0000","last_change":"2023-11-24T08:08:48.145897+0000","last_active":"2023-11-24T08:08:49.128304+0000","last_peered":"2023-11-24T08:08:49.128304+0000","last_clean":"2023-11-24T08:08:49.128304+0000","last_became_active":"2023-11-24T08:08:48.145620+0000","last_became_peered":"2023-11-24T08:08:48.145620+0000","last_unstale":"2023-11-24T08:08:49.128304+0000","last_undegraded":"2023-11-24T08:08:49.128304+0000","last_fullsized":"2023-11-24T08:08:49.128304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.167909+0000","last_change":"2023-11-24T08:08:48.164711+0000","last_active":"2023-11-24T08:08:48.167909+0000","last_peered":"2023-11-24T08:08:48.167909+0000","last_clean":"2023-11-24T08:08:48.167909+0000","last_became_active":"2023-11-24T08:08:48.164331+0000","last_became_peered":"2023-11-24T08:08:48.164331+0000","last_unstale":"2023-11-24T08:08:48.167909+0000","last_undegraded":"2023-11-24T08:08:48.167909+0000","last_fullsized":"2023-11-24T08:08:48.167909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127224+0000","last_change":"2023-11-24T08:08:48.148716+0000","last_active":"2023-11-24T08:08:49.127224+0000","last_peered":"2023-11-24T08:08:49.127224+0000","last_clean":"2023-11-24T08:08:49.127224+0000","last_became_active":"2023-11-24T08:08:48.148271+0000","last_became_peered":"2023-11-24T08:08:48.148271+0000","last_unstale":"2023-11-24T08:08:49.127224+0000","last_undegraded":"2023-11-24T08:08:49.127224+0000","last_fullsized":"2023-11-24T08:08:49.127224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:14.407973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164097+0000","last_change":"2023-11-24T08:08:48.156383+0000","last_active":"2023-11-24T08:08:48.164097+0000","last_peered":"2023-11-24T08:08:48.164097+0000","last_clean":"2023-11-24T08:08:48.164097+0000","last_became_active":"2023-11-24T08:08:48.156083+0000","last_became_peered":"2023-11-24T08:08:48.156083+0000","last_unstale":"2023-11-24T08:08:48.164097+0000","last_undegraded":"2023-11-24T08:08:48.164097+0000","last_fullsized":"2023-11-24T08:08:48.164097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:38.196670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126030+0000","last_change":"2023-11-24T08:08:48.157282+0000","last_active":"2023-11-24T08:08:49.126030+0000","last_peered":"2023-11-24T08:08:49.126030+0000","last_clean":"2023-11-24T08:08:49.126030+0000","last_became_active":"2023-11-24T08:08:48.155003+0000","last_became_peered":"2023-11-24T08:08:48.155003+0000","last_unstale":"2023-11-24T08:08:49.126030+0000","last_undegraded":"2023-11-24T08:08:49.126030+0000","last_fullsized":"2023-11-24T08:08:49.126030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:02.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.150367+0000","last_change":"2023-11-24T08:08:48.146014+0000","last_active":"2023-11-24T08:08:48.150367+0000","last_peered":"2023-11-24T08:08:48.150367+0000","last_clean":"2023-11-24T08:08:48.150367+0000","last_became_active":"2023-11-24T08:08:48.144834+0000","last_became_peered":"2023-11-24T08:08:48.144834+0000","last_unstale":"2023-11-24T08:08:48.150367+0000","last_undegraded":"2023-11-24T08:08:48.150367+0000","last_fullsized":"2023-11-24T08:08:48.150367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:19.982732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125567+0000","last_change":"2023-11-24T08:08:48.142247+0000","last_active":"2023-11-24T08:08:49.125567+0000","last_peered":"2023-11-24T08:08:49.125567+0000","last_clean":"2023-11-24T08:08:49.125567+0000","last_became_active":"2023-11-24T08:08:48.141969+0000","last_became_peered":"2023-11-24T08:08:48.141969+0000","last_unstale":"2023-11-24T08:08:49.125567+0000","last_undegraded":"2023-11-24T08:08:49.125567+0000","last_fullsized":"2023-11-24T08:08:49.125567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126146+0000","last_change":"2023-11-24T08:08:48.134993+0000","last_active":"2023-11-24T08:08:49.126146+0000","last_peered":"2023-11-24T08:08:49.126146+0000","last_clean":"2023-11-24T08:08:49.126146+0000","last_became_active":"2023-11-24T08:08:48.134752+0000","last_became_peered":"2023-11-24T08:08:48.134752+0000","last_unstale":"2023-11-24T08:08:49.126146+0000","last_undegraded":"2023-11-24T08:08:49.126146+0000","last_fullsized":"2023-11-24T08:08:49.126146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.155541+0000","last_change":"2023-11-24T08:08:48.147386+0000","last_active":"2023-11-24T08:08:48.155541+0000","last_peered":"2023-11-24T08:08:48.155541+0000","last_clean":"2023-11-24T08:08:48.155541+0000","last_became_active":"2023-11-24T08:08:48.144941+0000","last_became_peered":"2023-11-24T08:08:48.144941+0000","last_unstale":"2023-11-24T08:08:48.155541+0000","last_undegraded":"2023-11-24T08:08:48.155541+0000","last_fullsized":"2023-11-24T08:08:48.155541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.161304+0000","last_change":"2023-11-24T08:08:48.153360+0000","last_active":"2023-11-24T08:08:48.161304+0000","last_peered":"2023-11-24T08:08:48.161304+0000","last_clean":"2023-11-24T08:08:48.161304+0000","last_became_active":"2023-11-24T08:08:48.153099+0000","last_became_peered":"2023-11-24T08:08:48.153099+0000","last_unstale":"2023-11-24T08:08:48.161304+0000","last_undegraded":"2023-11-24T08:08:48.161304+0000","last_fullsized":"2023-11-24T08:08:48.161304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:38.196670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127574+0000","last_change":"2023-11-24T08:08:48.156712+0000","last_active":"2023-11-24T08:08:49.127574+0000","last_peered":"2023-11-24T08:08:49.127574+0000","last_clean":"2023-11-24T08:08:49.127574+0000","last_became_active":"2023-11-24T08:08:48.151074+0000","last_became_peered":"2023-11-24T08:08:48.151074+0000","last_unstale":"2023-11-24T08:08:49.127574+0000","last_undegraded":"2023-11-24T08:08:49.127574+0000","last_fullsized":"2023-11-24T08:08:49.127574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.079373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.156491+0000","last_change":"2023-11-24T08:08:48.149394+0000","last_active":"2023-11-24T08:08:48.156491+0000","last_peered":"2023-11-24T08:08:48.156491+0000","last_clean":"2023-11-24T08:08:48.156491+0000","last_became_active":"2023-11-24T08:08:48.149158+0000","last_became_peered":"2023-11-24T08:08:48.149158+0000","last_unstale":"2023-11-24T08:08:48.156491+0000","last_undegraded":"2023-11-24T08:08:48.156491+0000","last_fullsized":"2023-11-24T08:08:48.156491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:09.396602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124630+0000","last_change":"2023-11-24T08:08:48.155627+0000","last_active":"2023-11-24T08:08:49.124630+0000","last_peered":"2023-11-24T08:08:49.124630+0000","last_clean":"2023-11-24T08:08:49.124630+0000","last_became_active":"2023-11-24T08:08:48.155343+0000","last_became_peered":"2023-11-24T08:08:48.155343+0000","last_unstale":"2023-11-24T08:08:49.124630+0000","last_undegraded":"2023-11-24T08:08:49.124630+0000","last_fullsized":"2023-11-24T08:08:49.124630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:00.726435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107026+0000","last_change":"2023-11-24T08:08:48.141457+0000","last_active":"2023-11-24T08:08:50.107026+0000","last_peered":"2023-11-24T08:08:50.107026+0000","last_clean":"2023-11-24T08:08:50.107026+0000","last_became_active":"2023-11-24T08:08:48.139546+0000","last_became_peered":"2023-11-24T08:08:48.139546+0000","last_unstale":"2023-11-24T08:08:50.107026+0000","last_undegraded":"2023-11-24T08:08:50.107026+0000","last_fullsized":"2023-11-24T08:08:50.107026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.167290+0000","last_change":"2023-11-24T08:08:47.144970+0000","last_active":"2023-11-24T08:08:48.167290+0000","last_peered":"2023-11-24T08:08:48.167290+0000","last_clean":"2023-11-24T08:08:48.167290+0000","last_became_active":"2023-11-24T08:08:47.144667+0000","last_became_peered":"2023-11-24T08:08:47.144667+0000","last_unstale":"2023-11-24T08:08:48.167290+0000","last_undegraded":"2023-11-24T08:08:48.167290+0000","last_fullsized":"2023-11-24T08:08:48.167290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:59.483267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162425+0000","last_change":"2023-11-24T08:08:48.158059+0000","last_active":"2023-11-24T08:08:48.162425+0000","last_peered":"2023-11-24T08:08:48.162425+0000","last_clean":"2023-11-24T08:08:48.162425+0000","last_became_active":"2023-11-24T08:08:48.157743+0000","last_became_peered":"2023-11-24T08:08:48.157743+0000","last_unstale":"2023-11-24T08:08:48.162425+0000","last_undegraded":"2023-11-24T08:08:48.162425+0000","last_fullsized":"2023-11-24T08:08:48.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125078+0000","last_change":"2023-11-24T08:08:47.135247+0000","last_active":"2023-11-24T08:08:49.125078+0000","last_peered":"2023-11-24T08:08:49.125078+0000","last_clean":"2023-11-24T08:08:49.125078+0000","last_became_active":"2023-11-24T08:08:47.134923+0000","last_became_peered":"2023-11-24T08:08:47.134923+0000","last_unstale":"2023-11-24T08:08:49.125078+0000","last_undegraded":"2023-11-24T08:08:49.125078+0000","last_fullsized":"2023-11-24T08:08:49.125078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.131152+0000","last_change":"2023-11-24T08:08:47.151833+0000","last_active":"2023-11-24T08:08:48.131152+0000","last_peered":"2023-11-24T08:08:48.131152+0000","last_clean":"2023-11-24T08:08:48.131152+0000","last_became_active":"2023-11-24T08:08:47.151517+0000","last_became_peered":"2023-11-24T08:08:47.151517+0000","last_unstale":"2023-11-24T08:08:48.131152+0000","last_undegraded":"2023-11-24T08:08:48.131152+0000","last_fullsized":"2023-11-24T08:08:48.131152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.155308+0000","last_change":"2023-11-24T08:08:48.148262+0000","last_active":"2023-11-24T08:08:48.155308+0000","last_peered":"2023-11-24T08:08:48.155308+0000","last_clean":"2023-11-24T08:08:48.155308+0000","last_became_active":"2023-11-24T08:08:48.147874+0000","last_became_peered":"2023-11-24T08:08:48.147874+0000","last_unstale":"2023-11-24T08:08:48.155308+0000","last_undegraded":"2023-11-24T08:08:48.155308+0000","last_fullsized":"2023-11-24T08:08:48.155308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:09.396602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126262+0000","last_change":"2023-11-24T08:08:47.144280+0000","last_active":"2023-11-24T08:08:49.126262+0000","last_peered":"2023-11-24T08:08:49.126262+0000","last_clean":"2023-11-24T08:08:49.126262+0000","last_became_active":"2023-11-24T08:08:47.143933+0000","last_became_peered":"2023-11-24T08:08:47.143933+0000","last_unstale":"2023-11-24T08:08:49.126262+0000","last_undegraded":"2023-11-24T08:08:49.126262+0000","last_fullsized":"2023-11-24T08:08:49.126262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:28.820828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125379+0000","last_change":"2023-11-24T08:08:48.151024+0000","last_active":"2023-11-24T08:08:49.125379+0000","last_peered":"2023-11-24T08:08:49.125379+0000","last_clean":"2023-11-24T08:08:49.125379+0000","last_became_active":"2023-11-24T08:08:48.150727+0000","last_became_peered":"2023-11-24T08:08:48.150727+0000","last_unstale":"2023-11-24T08:08:49.125379+0000","last_undegraded":"2023-11-24T08:08:49.125379+0000","last_fullsized":"2023-11-24T08:08:49.125379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127936+0000","last_change":"2023-11-24T08:08:47.158190+0000","last_active":"2023-11-24T08:08:49.127936+0000","last_peered":"2023-11-24T08:08:49.127936+0000","last_clean":"2023-11-24T08:08:49.127936+0000","last_became_active":"2023-11-24T08:08:47.157894+0000","last_became_peered":"2023-11-24T08:08:47.157894+0000","last_unstale":"2023-11-24T08:08:49.127936+0000","last_undegraded":"2023-11-24T08:08:49.127936+0000","last_fullsized":"2023-11-24T08:08:49.127936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:18.739564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127729+0000","last_change":"2023-11-24T08:08:48.152122+0000","last_active":"2023-11-24T08:08:49.127729+0000","last_peered":"2023-11-24T08:08:49.127729+0000","last_clean":"2023-11-24T08:08:49.127729+0000","last_became_active":"2023-11-24T08:08:48.151509+0000","last_became_peered":"2023-11-24T08:08:48.151509+0000","last_unstale":"2023-11-24T08:08:49.127729+0000","last_undegraded":"2023-11-24T08:08:49.127729+0000","last_fullsized":"2023-11-24T08:08:49.127729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:17.382456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127836+0000","last_change":"2023-11-24T08:08:47.154457+0000","last_active":"2023-11-24T08:08:49.127836+0000","last_peered":"2023-11-24T08:08:49.127836+0000","last_clean":"2023-11-24T08:08:49.127836+0000","last_became_active":"2023-11-24T08:08:47.154137+0000","last_became_peered":"2023-11-24T08:08:47.154137+0000","last_unstale":"2023-11-24T08:08:49.127836+0000","last_undegraded":"2023-11-24T08:08:49.127836+0000","last_fullsized":"2023-11-24T08:08:49.127836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:19.309577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162515+0000","last_change":"2023-11-24T08:08:48.154269+0000","last_active":"2023-11-24T08:08:48.162515+0000","last_peered":"2023-11-24T08:08:48.162515+0000","last_clean":"2023-11-24T08:08:48.162515+0000","last_became_active":"2023-11-24T08:08:48.153873+0000","last_became_peered":"2023-11-24T08:08:48.153873+0000","last_unstale":"2023-11-24T08:08:48.162515+0000","last_undegraded":"2023-11-24T08:08:48.162515+0000","last_fullsized":"2023-11-24T08:08:48.162515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:09.396602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126169+0000","last_change":"2023-11-24T08:08:47.156467+0000","last_active":"2023-11-24T08:08:49.126169+0000","last_peered":"2023-11-24T08:08:49.126169+0000","last_clean":"2023-11-24T08:08:49.126169+0000","last_became_active":"2023-11-24T08:08:47.155999+0000","last_became_peered":"2023-11-24T08:08:47.155999+0000","last_unstale":"2023-11-24T08:08:49.126169+0000","last_undegraded":"2023-11-24T08:08:49.126169+0000","last_fullsized":"2023-11-24T08:08:49.126169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107764+0000","last_change":"2023-11-24T08:08:48.142662+0000","last_active":"2023-11-24T08:08:50.107764+0000","last_peered":"2023-11-24T08:08:50.107764+0000","last_clean":"2023-11-24T08:08:50.107764+0000","last_became_active":"2023-11-24T08:08:48.139505+0000","last_became_peered":"2023-11-24T08:08:48.139505+0000","last_unstale":"2023-11-24T08:08:50.107764+0000","last_undegraded":"2023-11-24T08:08:50.107764+0000","last_fullsized":"2023-11-24T08:08:50.107764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:01.498497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125359+0000","last_change":"2023-11-24T08:08:47.148708+0000","last_active":"2023-11-24T08:08:49.125359+0000","last_peered":"2023-11-24T08:08:49.125359+0000","last_clean":"2023-11-24T08:08:49.125359+0000","last_became_active":"2023-11-24T08:08:47.148386+0000","last_became_peered":"2023-11-24T08:08:47.148386+0000","last_unstale":"2023-11-24T08:08:49.125359+0000","last_undegraded":"2023-11-24T08:08:49.125359+0000","last_fullsized":"2023-11-24T08:08:49.125359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353438+0000","last_change":"2023-11-24T08:08:48.147343+0000","last_active":"2023-11-24T08:08:49.353438+0000","last_peered":"2023-11-24T08:08:49.353438+0000","last_clean":"2023-11-24T08:08:49.353438+0000","last_became_active":"2023-11-24T08:08:48.146625+0000","last_became_peered":"2023-11-24T08:08:48.146625+0000","last_unstale":"2023-11-24T08:08:49.353438+0000","last_undegraded":"2023-11-24T08:08:49.353438+0000","last_fullsized":"2023-11-24T08:08:49.353438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.269500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.161250+0000","last_change":"2023-11-24T08:08:47.141638+0000","last_active":"2023-11-24T08:08:48.161250+0000","last_peered":"2023-11-24T08:08:48.161250+0000","last_clean":"2023-11-24T08:08:48.161250+0000","last_became_active":"2023-11-24T08:08:47.141294+0000","last_became_peered":"2023-11-24T08:08:47.141294+0000","last_unstale":"2023-11-24T08:08:48.161250+0000","last_undegraded":"2023-11-24T08:08:48.161250+0000","last_fullsized":"2023-11-24T08:08:48.161250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164226+0000","last_change":"2023-11-24T08:08:48.159790+0000","last_active":"2023-11-24T08:08:48.164226+0000","last_peered":"2023-11-24T08:08:48.164226+0000","last_clean":"2023-11-24T08:08:48.164226+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:48.164226+0000","last_undegraded":"2023-11-24T08:08:48.164226+0000","last_fullsized":"2023-11-24T08:08:48.164226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127841+0000","last_change":"2023-11-24T08:08:47.157623+0000","last_active":"2023-11-24T08:08:49.127841+0000","last_peered":"2023-11-24T08:08:49.127841+0000","last_clean":"2023-11-24T08:08:49.127841+0000","last_became_active":"2023-11-24T08:08:47.156256+0000","last_became_peered":"2023-11-24T08:08:47.156256+0000","last_unstale":"2023-11-24T08:08:49.127841+0000","last_undegraded":"2023-11-24T08:08:49.127841+0000","last_fullsized":"2023-11-24T08:08:49.127841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:59.304396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353649+0000","last_change":"2023-11-24T08:08:48.152586+0000","last_active":"2023-11-24T08:08:49.353649+0000","last_peered":"2023-11-24T08:08:49.353649+0000","last_clean":"2023-11-24T08:08:49.353649+0000","last_became_active":"2023-11-24T08:08:48.152296+0000","last_became_peered":"2023-11-24T08:08:48.152296+0000","last_unstale":"2023-11-24T08:08:49.353649+0000","last_undegraded":"2023-11-24T08:08:49.353649+0000","last_fullsized":"2023-11-24T08:08:49.353649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:23.916532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124531+0000","last_change":"2023-11-24T08:08:47.136838+0000","last_active":"2023-11-24T08:08:49.124531+0000","last_peered":"2023-11-24T08:08:49.124531+0000","last_clean":"2023-11-24T08:08:49.124531+0000","last_became_active":"2023-11-24T08:08:47.136397+0000","last_became_peered":"2023-11-24T08:08:47.136397+0000","last_unstale":"2023-11-24T08:08:49.124531+0000","last_undegraded":"2023-11-24T08:08:49.124531+0000","last_fullsized":"2023-11-24T08:08:49.124531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125520+0000","last_change":"2023-11-24T08:08:48.153870+0000","last_active":"2023-11-24T08:08:49.125520+0000","last_peered":"2023-11-24T08:08:49.125520+0000","last_clean":"2023-11-24T08:08:49.125520+0000","last_became_active":"2023-11-24T08:08:48.153566+0000","last_became_peered":"2023-11-24T08:08:48.153566+0000","last_unstale":"2023-11-24T08:08:49.125520+0000","last_undegraded":"2023-11-24T08:08:49.125520+0000","last_fullsized":"2023-11-24T08:08:49.125520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.169106+0000","last_change":"2023-11-24T08:08:47.133661+0000","last_active":"2023-11-24T08:08:48.169106+0000","last_peered":"2023-11-24T08:08:48.169106+0000","last_clean":"2023-11-24T08:08:48.169106+0000","last_became_active":"2023-11-24T08:08:47.133327+0000","last_became_peered":"2023-11-24T08:08:47.133327+0000","last_unstale":"2023-11-24T08:08:48.169106+0000","last_undegraded":"2023-11-24T08:08:48.169106+0000","last_fullsized":"2023-11-24T08:08:48.169106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:36.361920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352992+0000","last_change":"2023-11-24T08:08:48.159918+0000","last_active":"2023-11-24T08:08:49.352992+0000","last_peered":"2023-11-24T08:08:49.352992+0000","last_clean":"2023-11-24T08:08:49.352992+0000","last_became_active":"2023-11-24T08:08:48.159604+0000","last_became_peered":"2023-11-24T08:08:48.159604+0000","last_unstale":"2023-11-24T08:08:49.352992+0000","last_undegraded":"2023-11-24T08:08:49.352992+0000","last_fullsized":"2023-11-24T08:08:49.352992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:19.645688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107838+0000","last_change":"2023-11-24T08:08:47.139479+0000","last_active":"2023-11-24T08:08:50.107838+0000","last_peered":"2023-11-24T08:08:50.107838+0000","last_clean":"2023-11-24T08:08:50.107838+0000","last_became_active":"2023-11-24T08:08:47.139162+0000","last_became_peered":"2023-11-24T08:08:47.139162+0000","last_unstale":"2023-11-24T08:08:50.107838+0000","last_undegraded":"2023-11-24T08:08:50.107838+0000","last_fullsized":"2023-11-24T08:08:50.107838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:51.916232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353918+0000","last_change":"2023-11-24T08:08:48.160119+0000","last_active":"2023-11-24T08:08:49.353918+0000","last_peered":"2023-11-24T08:08:49.353918+0000","last_clean":"2023-11-24T08:08:49.353918+0000","last_became_active":"2023-11-24T08:08:48.159806+0000","last_became_peered":"2023-11-24T08:08:48.159806+0000","last_unstale":"2023-11-24T08:08:49.353918+0000","last_undegraded":"2023-11-24T08:08:49.353918+0000","last_fullsized":"2023-11-24T08:08:49.353918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:19.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.169765+0000","last_change":"2023-11-24T08:08:47.139000+0000","last_active":"2023-11-24T08:08:48.169765+0000","last_peered":"2023-11-24T08:08:48.169765+0000","last_clean":"2023-11-24T08:08:48.169765+0000","last_became_active":"2023-11-24T08:08:47.138477+0000","last_became_peered":"2023-11-24T08:08:47.138477+0000","last_unstale":"2023-11-24T08:08:48.169765+0000","last_undegraded":"2023-11-24T08:08:48.169765+0000","last_fullsized":"2023-11-24T08:08:48.169765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:49.285847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354329+0000","last_change":"2023-11-24T08:08:48.162632+0000","last_active":"2023-11-24T08:08:49.354329+0000","last_peered":"2023-11-24T08:08:49.354329+0000","last_clean":"2023-11-24T08:08:49.354329+0000","last_became_active":"2023-11-24T08:08:48.160274+0000","last_became_peered":"2023-11-24T08:08:48.160274+0000","last_unstale":"2023-11-24T08:08:49.354329+0000","last_undegraded":"2023-11-24T08:08:49.354329+0000","last_fullsized":"2023-11-24T08:08:49.354329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:23.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126343+0000","last_change":"2023-11-24T08:08:47.133083+0000","last_active":"2023-11-24T08:08:49.126343+0000","last_peered":"2023-11-24T08:08:49.126343+0000","last_clean":"2023-11-24T08:08:49.126343+0000","last_became_active":"2023-11-24T08:08:47.132746+0000","last_became_peered":"2023-11-24T08:08:47.132746+0000","last_unstale":"2023-11-24T08:08:49.126343+0000","last_undegraded":"2023-11-24T08:08:49.126343+0000","last_fullsized":"2023-11-24T08:08:49.126343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:25.146390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126883+0000","last_change":"2023-11-24T08:08:48.151332+0000","last_active":"2023-11-24T08:08:49.126883+0000","last_peered":"2023-11-24T08:08:49.126883+0000","last_clean":"2023-11-24T08:08:49.126883+0000","last_became_active":"2023-11-24T08:08:48.151045+0000","last_became_peered":"2023-11-24T08:08:48.151045+0000","last_unstale":"2023-11-24T08:08:49.126883+0000","last_undegraded":"2023-11-24T08:08:49.126883+0000","last_fullsized":"2023-11-24T08:08:49.126883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:15.646317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.132022+0000","last_change":"2023-11-24T08:08:47.150523+0000","last_active":"2023-11-24T08:08:48.132022+0000","last_peered":"2023-11-24T08:08:48.132022+0000","last_clean":"2023-11-24T08:08:48.132022+0000","last_became_active":"2023-11-24T08:08:47.150208+0000","last_became_peered":"2023-11-24T08:08:47.150208+0000","last_unstale":"2023-11-24T08:08:48.132022+0000","last_undegraded":"2023-11-24T08:08:48.132022+0000","last_fullsized":"2023-11-24T08:08:48.132022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353456+0000","last_change":"2023-11-24T08:08:48.156319+0000","last_active":"2023-11-24T08:08:49.353456+0000","last_peered":"2023-11-24T08:08:49.353456+0000","last_clean":"2023-11-24T08:08:49.353456+0000","last_became_active":"2023-11-24T08:08:48.154550+0000","last_became_peered":"2023-11-24T08:08:48.154550+0000","last_unstale":"2023-11-24T08:08:49.353456+0000","last_undegraded":"2023-11-24T08:08:49.353456+0000","last_fullsized":"2023-11-24T08:08:49.353456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:17.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.135471+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:08:48.135471+0000","last_peered":"2023-11-24T08:08:48.135471+0000","last_clean":"2023-11-24T08:08:48.135471+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:08:48.135471+0000","last_undegraded":"2023-11-24T08:08:48.135471+0000","last_fullsized":"2023-11-24T08:08:48.135471+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125945+0000","last_change":"2023-11-24T08:08:47.157035+0000","last_active":"2023-11-24T08:08:49.125945+0000","last_peered":"2023-11-24T08:08:49.125945+0000","last_clean":"2023-11-24T08:08:49.125945+0000","last_became_active":"2023-11-24T08:08:47.155189+0000","last_became_peered":"2023-11-24T08:08:47.155189+0000","last_unstale":"2023-11-24T08:08:49.125945+0000","last_undegraded":"2023-11-24T08:08:49.125945+0000","last_fullsized":"2023-11-24T08:08:49.125945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:54.597005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.165805+0000","last_change":"2023-11-24T08:08:48.161239+0000","last_active":"2023-11-24T08:08:48.165805+0000","last_peered":"2023-11-24T08:08:48.165805+0000","last_clean":"2023-11-24T08:08:48.165805+0000","last_became_active":"2023-11-24T08:08:48.160530+0000","last_became_peered":"2023-11-24T08:08:48.160530+0000","last_unstale":"2023-11-24T08:08:48.165805+0000","last_undegraded":"2023-11-24T08:08:48.165805+0000","last_fullsized":"2023-11-24T08:08:48.165805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353366+0000","last_change":"2023-11-24T08:08:47.153886+0000","last_active":"2023-11-24T08:08:49.353366+0000","last_peered":"2023-11-24T08:08:49.353366+0000","last_clean":"2023-11-24T08:08:49.353366+0000","last_became_active":"2023-11-24T08:08:47.153576+0000","last_became_peered":"2023-11-24T08:08:47.153576+0000","last_unstale":"2023-11-24T08:08:49.353366+0000","last_undegraded":"2023-11-24T08:08:49.353366+0000","last_fullsized":"2023-11-24T08:08:49.353366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162413+0000","last_change":"2023-11-24T08:08:48.157955+0000","last_active":"2023-11-24T08:08:48.162413+0000","last_peered":"2023-11-24T08:08:48.162413+0000","last_clean":"2023-11-24T08:08:48.162413+0000","last_became_active":"2023-11-24T08:08:48.157636+0000","last_became_peered":"2023-11-24T08:08:48.157636+0000","last_unstale":"2023-11-24T08:08:48.162413+0000","last_undegraded":"2023-11-24T08:08:48.162413+0000","last_fullsized":"2023-11-24T08:08:48.162413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:48.939925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.132244+0000","last_change":"2023-11-24T08:08:47.143826+0000","last_active":"2023-11-24T08:08:48.132244+0000","last_peered":"2023-11-24T08:08:48.132244+0000","last_clean":"2023-11-24T08:08:48.132244+0000","last_became_active":"2023-11-24T08:08:47.142226+0000","last_became_peered":"2023-11-24T08:08:47.142226+0000","last_unstale":"2023-11-24T08:08:48.132244+0000","last_undegraded":"2023-11-24T08:08:48.132244+0000","last_fullsized":"2023-11-24T08:08:48.132244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125232+0000","last_change":"2023-11-24T08:08:48.133248+0000","last_active":"2023-11-24T08:08:49.125232+0000","last_peered":"2023-11-24T08:08:49.125232+0000","last_clean":"2023-11-24T08:08:49.125232+0000","last_became_active":"2023-11-24T08:08:48.133014+0000","last_became_peered":"2023-11-24T08:08:48.133014+0000","last_unstale":"2023-11-24T08:08:49.125232+0000","last_undegraded":"2023-11-24T08:08:49.125232+0000","last_fullsized":"2023-11-24T08:08:49.125232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127822+0000","last_change":"2023-11-24T08:08:47.154380+0000","last_active":"2023-11-24T08:08:49.127822+0000","last_peered":"2023-11-24T08:08:49.127822+0000","last_clean":"2023-11-24T08:08:49.127822+0000","last_became_active":"2023-11-24T08:08:47.154066+0000","last_became_peered":"2023-11-24T08:08:47.154066+0000","last_unstale":"2023-11-24T08:08:49.127822+0000","last_undegraded":"2023-11-24T08:08:49.127822+0000","last_fullsized":"2023-11-24T08:08:49.127822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:48:29.123228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107253+0000","last_change":"2023-11-24T08:08:48.155559+0000","last_active":"2023-11-24T08:08:50.107253+0000","last_peered":"2023-11-24T08:08:50.107253+0000","last_clean":"2023-11-24T08:08:50.107253+0000","last_became_active":"2023-11-24T08:08:48.155311+0000","last_became_peered":"2023-11-24T08:08:48.155311+0000","last_unstale":"2023-11-24T08:08:50.107253+0000","last_undegraded":"2023-11-24T08:08:50.107253+0000","last_fullsized":"2023-11-24T08:08:50.107253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125803+0000","last_change":"2023-11-24T08:08:47.137292+0000","last_active":"2023-11-24T08:08:49.125803+0000","last_peered":"2023-11-24T08:08:49.125803+0000","last_clean":"2023-11-24T08:08:49.125803+0000","last_became_active":"2023-11-24T08:08:47.136872+0000","last_became_peered":"2023-11-24T08:08:47.136872+0000","last_unstale":"2023-11-24T08:08:49.125803+0000","last_undegraded":"2023-11-24T08:08:49.125803+0000","last_fullsized":"2023-11-24T08:08:49.125803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127383+0000","last_change":"2023-11-24T08:08:48.161415+0000","last_active":"2023-11-24T08:08:49.127383+0000","last_peered":"2023-11-24T08:08:49.127383+0000","last_clean":"2023-11-24T08:08:49.127383+0000","last_became_active":"2023-11-24T08:08:48.158160+0000","last_became_peered":"2023-11-24T08:08:48.158160+0000","last_unstale":"2023-11-24T08:08:49.127383+0000","last_undegraded":"2023-11-24T08:08:49.127383+0000","last_fullsized":"2023-11-24T08:08:49.127383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:24.063260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354005+0000","last_change":"2023-11-24T08:08:47.153377+0000","last_active":"2023-11-24T08:08:49.354005+0000","last_peered":"2023-11-24T08:08:49.354005+0000","last_clean":"2023-11-24T08:08:49.354005+0000","last_became_active":"2023-11-24T08:08:47.153112+0000","last_became_peered":"2023-11-24T08:08:47.153112+0000","last_unstale":"2023-11-24T08:08:49.354005+0000","last_undegraded":"2023-11-24T08:08:49.354005+0000","last_fullsized":"2023-11-24T08:08:49.354005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:39.232399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125805+0000","last_change":"2023-11-24T08:08:48.134378+0000","last_active":"2023-11-24T08:08:49.125805+0000","last_peered":"2023-11-24T08:08:49.125805+0000","last_clean":"2023-11-24T08:08:49.125805+0000","last_became_active":"2023-11-24T08:08:48.134043+0000","last_became_peered":"2023-11-24T08:08:48.134043+0000","last_unstale":"2023-11-24T08:08:49.125805+0000","last_undegraded":"2023-11-24T08:08:49.125805+0000","last_fullsized":"2023-11-24T08:08:49.125805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126267+0000","last_change":"2023-11-24T08:08:47.148275+0000","last_active":"2023-11-24T08:08:49.126267+0000","last_peered":"2023-11-24T08:08:49.126267+0000","last_clean":"2023-11-24T08:08:49.126267+0000","last_became_active":"2023-11-24T08:08:47.148025+0000","last_became_peered":"2023-11-24T08:08:47.148025+0000","last_unstale":"2023-11-24T08:08:49.126267+0000","last_undegraded":"2023-11-24T08:08:49.126267+0000","last_fullsized":"2023-11-24T08:08:49.126267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.157434+0000","last_change":"2023-11-24T08:08:48.146544+0000","last_active":"2023-11-24T08:08:48.157434+0000","last_peered":"2023-11-24T08:08:48.157434+0000","last_clean":"2023-11-24T08:08:48.157434+0000","last_became_active":"2023-11-24T08:08:48.146269+0000","last_became_peered":"2023-11-24T08:08:48.146269+0000","last_unstale":"2023-11-24T08:08:48.157434+0000","last_undegraded":"2023-11-24T08:08:48.157434+0000","last_fullsized":"2023-11-24T08:08:48.157434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:21.412283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128129+0000","last_change":"2023-11-24T08:08:47.152442+0000","last_active":"2023-11-24T08:08:49.128129+0000","last_peered":"2023-11-24T08:08:49.128129+0000","last_clean":"2023-11-24T08:08:49.128129+0000","last_became_active":"2023-11-24T08:08:47.151763+0000","last_became_peered":"2023-11-24T08:08:47.151763+0000","last_unstale":"2023-11-24T08:08:49.128129+0000","last_undegraded":"2023-11-24T08:08:49.128129+0000","last_fullsized":"2023-11-24T08:08:49.128129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.156246+0000","last_change":"2023-11-24T08:08:48.152228+0000","last_active":"2023-11-24T08:08:48.156246+0000","last_peered":"2023-11-24T08:08:48.156246+0000","last_clean":"2023-11-24T08:08:48.156246+0000","last_became_active":"2023-11-24T08:08:48.151904+0000","last_became_peered":"2023-11-24T08:08:48.151904+0000","last_unstale":"2023-11-24T08:08:48.156246+0000","last_undegraded":"2023-11-24T08:08:48.156246+0000","last_fullsized":"2023-11-24T08:08:48.156246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:37.605088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352672+0000","last_change":"2023-11-24T08:08:47.150060+0000","last_active":"2023-11-24T08:08:49.352672+0000","last_peered":"2023-11-24T08:08:49.352672+0000","last_clean":"2023-11-24T08:08:49.352672+0000","last_became_active":"2023-11-24T08:08:47.149756+0000","last_became_peered":"2023-11-24T08:08:47.149756+0000","last_unstale":"2023-11-24T08:08:49.352672+0000","last_undegraded":"2023-11-24T08:08:49.352672+0000","last_fullsized":"2023-11-24T08:08:49.352672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107214+0000","last_change":"2023-11-24T08:08:48.150454+0000","last_active":"2023-11-24T08:08:50.107214+0000","last_peered":"2023-11-24T08:08:50.107214+0000","last_clean":"2023-11-24T08:08:50.107214+0000","last_became_active":"2023-11-24T08:08:48.139831+0000","last_became_peered":"2023-11-24T08:08:48.139831+0000","last_unstale":"2023-11-24T08:08:50.107214+0000","last_undegraded":"2023-11-24T08:08:50.107214+0000","last_fullsized":"2023-11-24T08:08:50.107214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125621+0000","last_change":"2023-11-24T08:08:47.141901+0000","last_active":"2023-11-24T08:08:49.125621+0000","last_peered":"2023-11-24T08:08:49.125621+0000","last_clean":"2023-11-24T08:08:49.125621+0000","last_became_active":"2023-11-24T08:08:47.140760+0000","last_became_peered":"2023-11-24T08:08:47.140760+0000","last_unstale":"2023-11-24T08:08:49.125621+0000","last_undegraded":"2023-11-24T08:08:49.125621+0000","last_fullsized":"2023-11-24T08:08:49.125621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:12.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.145683+0000","last_change":"2023-11-24T08:08:48.141100+0000","last_active":"2023-11-24T08:08:48.145683+0000","last_peered":"2023-11-24T08:08:48.145683+0000","last_clean":"2023-11-24T08:08:48.145683+0000","last_became_active":"2023-11-24T08:08:48.138053+0000","last_became_peered":"2023-11-24T08:08:48.138053+0000","last_unstale":"2023-11-24T08:08:48.145683+0000","last_undegraded":"2023-11-24T08:08:48.145683+0000","last_fullsized":"2023-11-24T08:08:48.145683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:22.547643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125743+0000","last_change":"2023-11-24T08:08:47.138812+0000","last_active":"2023-11-24T08:08:49.125743+0000","last_peered":"2023-11-24T08:08:49.125743+0000","last_clean":"2023-11-24T08:08:49.125743+0000","last_became_active":"2023-11-24T08:08:47.138512+0000","last_became_peered":"2023-11-24T08:08:47.138512+0000","last_unstale":"2023-11-24T08:08:49.125743+0000","last_undegraded":"2023-11-24T08:08:49.125743+0000","last_fullsized":"2023-11-24T08:08:49.125743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.163845+0000","last_change":"2023-11-24T08:08:48.155403+0000","last_active":"2023-11-24T08:08:48.163845+0000","last_peered":"2023-11-24T08:08:48.163845+0000","last_clean":"2023-11-24T08:08:48.163845+0000","last_became_active":"2023-11-24T08:08:48.154517+0000","last_became_peered":"2023-11-24T08:08:48.154517+0000","last_unstale":"2023-11-24T08:08:48.163845+0000","last_undegraded":"2023-11-24T08:08:48.163845+0000","last_fullsized":"2023-11-24T08:08:48.163845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:37.605088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353934+0000","last_change":"2023-11-24T08:08:47.152677+0000","last_active":"2023-11-24T08:08:49.353934+0000","last_peered":"2023-11-24T08:08:49.353934+0000","last_clean":"2023-11-24T08:08:49.353934+0000","last_became_active":"2023-11-24T08:08:47.151375+0000","last_became_peered":"2023-11-24T08:08:47.151375+0000","last_unstale":"2023-11-24T08:08:49.353934+0000","last_undegraded":"2023-11-24T08:08:49.353934+0000","last_fullsized":"2023-11-24T08:08:49.353934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:56.046702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125623+0000","last_change":"2023-11-24T08:08:48.162261+0000","last_active":"2023-11-24T08:08:49.125623+0000","last_peered":"2023-11-24T08:08:49.125623+0000","last_clean":"2023-11-24T08:08:49.125623+0000","last_became_active":"2023-11-24T08:08:48.156627+0000","last_became_peered":"2023-11-24T08:08:48.156627+0000","last_unstale":"2023-11-24T08:08:49.125623+0000","last_undegraded":"2023-11-24T08:08:49.125623+0000","last_fullsized":"2023-11-24T08:08:49.125623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127263+0000","last_change":"2023-11-24T08:08:48.162034+0000","last_active":"2023-11-24T08:08:49.127263+0000","last_peered":"2023-11-24T08:08:49.127263+0000","last_clean":"2023-11-24T08:08:49.127263+0000","last_became_active":"2023-11-24T08:08:48.161663+0000","last_became_peered":"2023-11-24T08:08:48.161663+0000","last_unstale":"2023-11-24T08:08:49.127263+0000","last_undegraded":"2023-11-24T08:08:49.127263+0000","last_fullsized":"2023-11-24T08:08:49.127263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:56.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353243+0000","last_change":"2023-11-24T08:08:47.156960+0000","last_active":"2023-11-24T08:08:49.353243+0000","last_peered":"2023-11-24T08:08:49.353243+0000","last_clean":"2023-11-24T08:08:49.353243+0000","last_became_active":"2023-11-24T08:08:47.156401+0000","last_became_peered":"2023-11-24T08:08:47.156401+0000","last_unstale":"2023-11-24T08:08:49.353243+0000","last_undegraded":"2023-11-24T08:08:49.353243+0000","last_fullsized":"2023-11-24T08:08:49.353243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:04.035911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126566+0000","last_change":"2023-11-24T08:08:48.159506+0000","last_active":"2023-11-24T08:08:49.126566+0000","last_peered":"2023-11-24T08:08:49.126566+0000","last_clean":"2023-11-24T08:08:49.126566+0000","last_became_active":"2023-11-24T08:08:48.159220+0000","last_became_peered":"2023-11-24T08:08:48.159220+0000","last_unstale":"2023-11-24T08:08:49.126566+0000","last_undegraded":"2023-11-24T08:08:49.126566+0000","last_fullsized":"2023-11-24T08:08:49.126566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:14.900691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353690+0000","last_change":"2023-11-24T08:08:47.155268+0000","last_active":"2023-11-24T08:08:49.353690+0000","last_peered":"2023-11-24T08:08:49.353690+0000","last_clean":"2023-11-24T08:08:49.353690+0000","last_became_active":"2023-11-24T08:08:47.154139+0000","last_became_peered":"2023-11-24T08:08:47.154139+0000","last_unstale":"2023-11-24T08:08:49.353690+0000","last_undegraded":"2023-11-24T08:08:49.353690+0000","last_fullsized":"2023-11-24T08:08:49.353690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162564+0000","last_change":"2023-11-24T08:08:48.157139+0000","last_active":"2023-11-24T08:08:48.162564+0000","last_peered":"2023-11-24T08:08:48.162564+0000","last_clean":"2023-11-24T08:08:48.162564+0000","last_became_active":"2023-11-24T08:08:48.156875+0000","last_became_peered":"2023-11-24T08:08:48.156875+0000","last_unstale":"2023-11-24T08:08:48.162564+0000","last_undegraded":"2023-11-24T08:08:48.162564+0000","last_fullsized":"2023-11-24T08:08:48.162564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:22.547643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164239+0000","last_change":"2023-11-24T08:08:47.143443+0000","last_active":"2023-11-24T08:08:48.164239+0000","last_peered":"2023-11-24T08:08:48.164239+0000","last_clean":"2023-11-24T08:08:48.164239+0000","last_became_active":"2023-11-24T08:08:47.143156+0000","last_became_peered":"2023-11-24T08:08:47.143156+0000","last_unstale":"2023-11-24T08:08:48.164239+0000","last_undegraded":"2023-11-24T08:08:48.164239+0000","last_fullsized":"2023-11-24T08:08:48.164239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.145461+0000","last_change":"2023-11-24T08:08:48.141063+0000","last_active":"2023-11-24T08:08:48.145461+0000","last_peered":"2023-11-24T08:08:48.145461+0000","last_clean":"2023-11-24T08:08:48.145461+0000","last_became_active":"2023-11-24T08:08:48.140809+0000","last_became_peered":"2023-11-24T08:08:48.140809+0000","last_unstale":"2023-11-24T08:08:48.145461+0000","last_undegraded":"2023-11-24T08:08:48.145461+0000","last_fullsized":"2023-11-24T08:08:48.145461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.167730+0000","last_change":"2023-11-24T08:08:47.150905+0000","last_active":"2023-11-24T08:08:48.167730+0000","last_peered":"2023-11-24T08:08:48.167730+0000","last_clean":"2023-11-24T08:08:48.167730+0000","last_became_active":"2023-11-24T08:08:47.150670+0000","last_became_peered":"2023-11-24T08:08:47.150670+0000","last_unstale":"2023-11-24T08:08:48.167730+0000","last_undegraded":"2023-11-24T08:08:48.167730+0000","last_fullsized":"2023-11-24T08:08:48.167730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2930000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7090000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9119999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.887}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6879999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7309999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.8409999999999993}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.2729999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0579999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1399999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1379999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1400000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":663582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9610000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:52.566 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:52.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.810+0000 7f83282c9640 1 Processor -- start 2023-11-24T08:08:52.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.810+0000 7f83282c9640 1 -- start start 2023-11-24T08:08:52.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.811+0000 7f83282c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f832015c7f0 0x7f832006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.811+0000 7f83282c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83200cf450 0x7f83200cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.811+0000 7f83282c9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83200cece0 con 0x7f83200cdfb0 2023-11-24T08:08:52.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.811+0000 7f83282c9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8320156010 con 0x7f83200cf450 2023-11-24T08:08:52.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.811+0000 7f83282c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832006ce30 con 0x7f832015c7f0 2023-11-24T08:08:52.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83200cf450 0x7f83200cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83200cf450 0x7f83200cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47822/0 (socket says 172.21.15.106:47822) 2023-11-24T08:08:52.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 -- 172.21.15.106:0/2504578883 learned_addr learned my addr 172.21.15.106:0/2504578883 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:52.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8326065640 1 --2- 172.21.15.106:0/2504578883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f832015c7f0 0x7f832006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 -- 172.21.15.106:0/2504578883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f832015c7f0 msgr2=0x7f832006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 --2- 172.21.15.106:0/2504578883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f832015c7f0 0x7f832006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 -- 172.21.15.106:0/2504578883 >> v1:172.21.15.156:6789/0 conn(0x7f83200cdfb0 legacy=0x7f83200ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:52.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.812+0000 7f8325864640 1 -- 172.21.15.106:0/2504578883 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832006d040 con 0x7f83200cf450 2023-11-24T08:08:52.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.814+0000 7f8325864640 1 --2- 172.21.15.106:0/2504578883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f83200cf8d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8310009870 tx=0x7f831002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2da75f1e580ecc72 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.814+0000 7f8325063640 1 -- 172.21.15.106:0/2504578883 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310031640 con 0x7f83200cf450 2023-11-24T08:08:52.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.814+0000 7f8325063640 1 -- 172.21.15.106:0/2504578883 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83100317a0 con 0x7f83200cf450 2023-11-24T08:08:52.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.814+0000 7f8325063640 1 -- 172.21.15.106:0/2504578883 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310031b30 con 0x7f83200cf450 2023-11-24T08:08:52.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.814+0000 7f83282c9640 1 -- 172.21.15.106:0/2504578883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 msgr2=0x7f83200cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.814+0000 7f83282c9640 1 --2- 172.21.15.106:0/2504578883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f83200cf8d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8310009870 tx=0x7f831002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.815+0000 7f83282c9640 1 -- 172.21.15.106:0/2504578883 shutdown_connections 2023-11-24T08:08:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.815+0000 7f83282c9640 1 --2- 172.21.15.106:0/2504578883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f83200cf8d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.815+0000 7f83282c9640 1 --2- 172.21.15.106:0/2504578883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f832015c7f0 0x7f832006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.815+0000 7f83282c9640 1 -- 172.21.15.106:0/2504578883 >> 172.21.15.106:0/2504578883 conn(0x7f832006a920 msgr2=0x7f832006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.815+0000 7f83282c9640 1 -- 172.21.15.106:0/2504578883 shutdown_connections 2023-11-24T08:08:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.815+0000 7f83282c9640 1 -- 172.21.15.106:0/2504578883 wait complete. 2023-11-24T08:08:52.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.816+0000 7f83282c9640 1 Processor -- start 2023-11-24T08:08:52.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 -- start start 2023-11-24T08:08:52.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83200cdfb0 0x7f8320158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f8320157a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f832015c7f0 0x7f83201533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320152890 con 0x7f83200cf450 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8320159020 con 0x7f832015c7f0 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.817+0000 7f83282c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83201f0cf0 con 0x7f83200cdfb0 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f8320157a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f8320157a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47830/0 (socket says 172.21.15.106:47830) 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 -- 172.21.15.106:0/1208720142 learned_addr learned my addr 172.21.15.106:0/1208720142 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8326065640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83200cdfb0 0x7f8320158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8326866640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f832015c7f0 0x7f83201533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 -- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83200cdfb0 msgr2=0x7f8320158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83200cdfb0 0x7f8320158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 -- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f832015c7f0 msgr2=0x7f83201533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f832015c7f0 0x7f83201533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 -- 172.21.15.106:0/1208720142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83100094d0 con 0x7f83200cf450 2023-11-24T08:08:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.818+0000 7f8325864640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f8320157a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8310000c00 tx=0x7f83100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.819+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831003ac60 con 0x7f83200cf450 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.819+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f831003adc0 con 0x7f83200cf450 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.819+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310044630 con 0x7f83200cf450 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.819+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8320151d10 con 0x7f83200cf450 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.819+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83201522d0 con 0x7f83200cf450 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.820+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83201f0e60 con 0x7f83200cf450 2023-11-24T08:08:52.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.821+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f831002f050 con 0x7f83200cf450 2023-11-24T08:08:52.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.822+0000 7f8316ffd640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f830406ae70 0x7f830406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:52.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.822+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f83100ab510 con 0x7f83200cf450 2023-11-24T08:08:52.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.823+0000 7f8326065640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f830406ae70 0x7f830406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:52.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.823+0000 7f8326065640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f830406ae70 0x7f830406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f831c006fd0 tx=0x7f831c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:52.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:52.827+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83100765b0 con 0x7f83200cf450 2023-11-24T08:08:53.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.013+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f832006e060 con 0x7f830406ae70 2023-11-24T08:08:53.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.021+0000 7f8316ffd640 1 -- 172.21.15.106:0/1208720142 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264506 (secure 0 0 0) 0x7f831c00e4c0 con 0x7f830406ae70 2023-11-24T08:08:53.026 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:53.027 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:08:53.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f830406ae70 msgr2=0x7f830406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f830406ae70 0x7f830406d330 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f831c006fd0 tx=0x7f831c008040 comp rx=0 tx=0).stop 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 msgr2=0x7f8320157a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f8320157a20 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8310000c00 tx=0x7f83100046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 shutdown_connections 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f832015c7f0 0x7f83201533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83200cf450 0x7f8320157a20 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f830406ae70 0x7f830406d330 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 --2- 172.21.15.106:0/1208720142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83200cdfb0 0x7f8320158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 >> 172.21.15.106:0/1208720142 conn(0x7f832006a920 msgr2=0x7f83200735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 shutdown_connections 2023-11-24T08:08:53.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.026+0000 7f83282c9640 1 -- 172.21.15.106:0/1208720142 wait complete. 2023-11-24T08:08:53.047 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-24T08:08:51.274961+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":166,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235912,"kb_used_data":2948,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738808,"statfs":{"total":773094113280,"available":772852539392,"internally_reserved":0,"allocated":3018752,"data_stored":1797664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":58,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.178582"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354084+0000","last_change":"2023-11-24T08:08:48.155148+0000","last_active":"2023-11-24T08:08:49.354084+0000","last_peered":"2023-11-24T08:08:49.354084+0000","last_clean":"2023-11-24T08:08:49.354084+0000","last_became_active":"2023-11-24T08:08:48.148725+0000","last_became_peered":"2023-11-24T08:08:48.148725+0000","last_unstale":"2023-11-24T08:08:49.354084+0000","last_undegraded":"2023-11-24T08:08:49.354084+0000","last_fullsized":"2023-11-24T08:08:49.354084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:46.218050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126326+0000","last_change":"2023-11-24T08:08:48.153462+0000","last_active":"2023-11-24T08:08:49.126326+0000","last_peered":"2023-11-24T08:08:49.126326+0000","last_clean":"2023-11-24T08:08:49.126326+0000","last_became_active":"2023-11-24T08:08:48.153009+0000","last_became_peered":"2023-11-24T08:08:48.153009+0000","last_unstale":"2023-11-24T08:08:49.126326+0000","last_undegraded":"2023-11-24T08:08:49.126326+0000","last_fullsized":"2023-11-24T08:08:49.126326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.154972+0000","last_change":"2023-11-24T08:08:48.147649+0000","last_active":"2023-11-24T08:08:48.154972+0000","last_peered":"2023-11-24T08:08:48.154972+0000","last_clean":"2023-11-24T08:08:48.154972+0000","last_became_active":"2023-11-24T08:08:48.147280+0000","last_became_peered":"2023-11-24T08:08:48.147280+0000","last_unstale":"2023-11-24T08:08:48.154972+0000","last_undegraded":"2023-11-24T08:08:48.154972+0000","last_fullsized":"2023-11-24T08:08:48.154972+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:50.529015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126469+0000","last_change":"2023-11-24T08:08:48.158446+0000","last_active":"2023-11-24T08:08:49.126469+0000","last_peered":"2023-11-24T08:08:49.126469+0000","last_clean":"2023-11-24T08:08:49.126469+0000","last_became_active":"2023-11-24T08:08:48.158196+0000","last_became_peered":"2023-11-24T08:08:48.158196+0000","last_unstale":"2023-11-24T08:08:49.126469+0000","last_undegraded":"2023-11-24T08:08:49.126469+0000","last_fullsized":"2023-11-24T08:08:49.126469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:58.752582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353187+0000","last_change":"2023-11-24T08:08:48.148128+0000","last_active":"2023-11-24T08:08:49.353187+0000","last_peered":"2023-11-24T08:08:49.353187+0000","last_clean":"2023-11-24T08:08:49.353187+0000","last_became_active":"2023-11-24T08:08:48.147789+0000","last_became_peered":"2023-11-24T08:08:48.147789+0000","last_unstale":"2023-11-24T08:08:49.353187+0000","last_undegraded":"2023-11-24T08:08:49.353187+0000","last_fullsized":"2023-11-24T08:08:49.353187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:17.852217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126840+0000","last_change":"2023-11-24T08:08:48.160813+0000","last_active":"2023-11-24T08:08:49.126840+0000","last_peered":"2023-11-24T08:08:49.126840+0000","last_clean":"2023-11-24T08:08:49.126840+0000","last_became_active":"2023-11-24T08:08:48.159970+0000","last_became_peered":"2023-11-24T08:08:48.159970+0000","last_unstale":"2023-11-24T08:08:49.126840+0000","last_undegraded":"2023-11-24T08:08:49.126840+0000","last_fullsized":"2023-11-24T08:08:49.126840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:43.997384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352936+0000","last_change":"2023-11-24T08:08:48.147161+0000","last_active":"2023-11-24T08:08:49.352936+0000","last_peered":"2023-11-24T08:08:49.352936+0000","last_clean":"2023-11-24T08:08:49.352936+0000","last_became_active":"2023-11-24T08:08:48.146867+0000","last_became_peered":"2023-11-24T08:08:48.146867+0000","last_unstale":"2023-11-24T08:08:49.352936+0000","last_undegraded":"2023-11-24T08:08:49.352936+0000","last_fullsized":"2023-11-24T08:08:49.352936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:22:50.592771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.161892+0000","last_change":"2023-11-24T08:08:48.151256+0000","last_active":"2023-11-24T08:08:48.161892+0000","last_peered":"2023-11-24T08:08:48.161892+0000","last_clean":"2023-11-24T08:08:48.161892+0000","last_became_active":"2023-11-24T08:08:48.150977+0000","last_became_peered":"2023-11-24T08:08:48.150977+0000","last_unstale":"2023-11-24T08:08:48.161892+0000","last_undegraded":"2023-11-24T08:08:48.161892+0000","last_fullsized":"2023-11-24T08:08:48.161892+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:50.529015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354145+0000","last_change":"2023-11-24T08:08:48.165291+0000","last_active":"2023-11-24T08:08:49.354145+0000","last_peered":"2023-11-24T08:08:49.354145+0000","last_clean":"2023-11-24T08:08:49.354145+0000","last_became_active":"2023-11-24T08:08:48.164956+0000","last_became_peered":"2023-11-24T08:08:48.164956+0000","last_unstale":"2023-11-24T08:08:49.354145+0000","last_undegraded":"2023-11-24T08:08:49.354145+0000","last_fullsized":"2023-11-24T08:08:49.354145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:32.727731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125355+0000","last_change":"2023-11-24T08:08:48.159627+0000","last_active":"2023-11-24T08:08:49.125355+0000","last_peered":"2023-11-24T08:08:49.125355+0000","last_clean":"2023-11-24T08:08:49.125355+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:49.125355+0000","last_undegraded":"2023-11-24T08:08:49.125355+0000","last_fullsized":"2023-11-24T08:08:49.125355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125146+0000","last_change":"2023-11-24T08:08:48.157396+0000","last_active":"2023-11-24T08:08:49.125146+0000","last_peered":"2023-11-24T08:08:49.125146+0000","last_clean":"2023-11-24T08:08:49.125146+0000","last_became_active":"2023-11-24T08:08:48.155190+0000","last_became_peered":"2023-11-24T08:08:48.155190+0000","last_unstale":"2023-11-24T08:08:49.125146+0000","last_undegraded":"2023-11-24T08:08:49.125146+0000","last_fullsized":"2023-11-24T08:08:49.125146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107513+0000","last_change":"2023-11-24T08:08:48.139798+0000","last_active":"2023-11-24T08:08:50.107513+0000","last_peered":"2023-11-24T08:08:50.107513+0000","last_clean":"2023-11-24T08:08:50.107513+0000","last_became_active":"2023-11-24T08:08:48.139186+0000","last_became_peered":"2023-11-24T08:08:48.139186+0000","last_unstale":"2023-11-24T08:08:50.107513+0000","last_undegraded":"2023-11-24T08:08:50.107513+0000","last_fullsized":"2023-11-24T08:08:50.107513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.146745+0000","last_change":"2023-11-24T08:08:48.141667+0000","last_active":"2023-11-24T08:08:48.146745+0000","last_peered":"2023-11-24T08:08:48.146745+0000","last_clean":"2023-11-24T08:08:48.146745+0000","last_became_active":"2023-11-24T08:08:48.141410+0000","last_became_peered":"2023-11-24T08:08:48.141410+0000","last_unstale":"2023-11-24T08:08:48.146745+0000","last_undegraded":"2023-11-24T08:08:48.146745+0000","last_fullsized":"2023-11-24T08:08:48.146745+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164172+0000","last_change":"2023-11-24T08:08:48.154158+0000","last_active":"2023-11-24T08:08:48.164172+0000","last_peered":"2023-11-24T08:08:48.164172+0000","last_clean":"2023-11-24T08:08:48.164172+0000","last_became_active":"2023-11-24T08:08:48.153833+0000","last_became_peered":"2023-11-24T08:08:48.153833+0000","last_unstale":"2023-11-24T08:08:48.164172+0000","last_undegraded":"2023-11-24T08:08:48.164172+0000","last_fullsized":"2023-11-24T08:08:48.164172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125503+0000","last_change":"2023-11-24T08:08:48.130056+0000","last_active":"2023-11-24T08:08:49.125503+0000","last_peered":"2023-11-24T08:08:49.125503+0000","last_clean":"2023-11-24T08:08:49.125503+0000","last_became_active":"2023-11-24T08:08:48.129739+0000","last_became_peered":"2023-11-24T08:08:48.129739+0000","last_unstale":"2023-11-24T08:08:49.125503+0000","last_undegraded":"2023-11-24T08:08:49.125503+0000","last_fullsized":"2023-11-24T08:08:49.125503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125661+0000","last_change":"2023-11-24T08:08:48.157064+0000","last_active":"2023-11-24T08:08:49.125661+0000","last_peered":"2023-11-24T08:08:49.125661+0000","last_clean":"2023-11-24T08:08:49.125661+0000","last_became_active":"2023-11-24T08:08:48.153014+0000","last_became_peered":"2023-11-24T08:08:48.153014+0000","last_unstale":"2023-11-24T08:08:49.125661+0000","last_undegraded":"2023-11-24T08:08:49.125661+0000","last_fullsized":"2023-11-24T08:08:49.125661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:53.228655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128174+0000","last_change":"2023-11-24T08:08:48.168538+0000","last_active":"2023-11-24T08:08:49.128174+0000","last_peered":"2023-11-24T08:08:49.128174+0000","last_clean":"2023-11-24T08:08:49.128174+0000","last_became_active":"2023-11-24T08:08:48.168251+0000","last_became_peered":"2023-11-24T08:08:48.168251+0000","last_unstale":"2023-11-24T08:08:49.128174+0000","last_undegraded":"2023-11-24T08:08:49.128174+0000","last_fullsized":"2023-11-24T08:08:49.128174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:34.330436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.136935+0000","last_change":"2023-11-24T08:08:48.135832+0000","last_active":"2023-11-24T08:08:48.136935+0000","last_peered":"2023-11-24T08:08:48.136935+0000","last_clean":"2023-11-24T08:08:48.136935+0000","last_became_active":"2023-11-24T08:08:48.135419+0000","last_became_peered":"2023-11-24T08:08:48.135419+0000","last_unstale":"2023-11-24T08:08:48.136935+0000","last_undegraded":"2023-11-24T08:08:48.136935+0000","last_fullsized":"2023-11-24T08:08:48.136935+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:22.547643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353897+0000","last_change":"2023-11-24T08:08:48.148026+0000","last_active":"2023-11-24T08:08:49.353897+0000","last_peered":"2023-11-24T08:08:49.353897+0000","last_clean":"2023-11-24T08:08:49.353897+0000","last_became_active":"2023-11-24T08:08:48.147722+0000","last_became_peered":"2023-11-24T08:08:48.147722+0000","last_unstale":"2023-11-24T08:08:49.353897+0000","last_undegraded":"2023-11-24T08:08:49.353897+0000","last_fullsized":"2023-11-24T08:08:49.353897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:15:57.275173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128304+0000","last_change":"2023-11-24T08:08:48.145897+0000","last_active":"2023-11-24T08:08:49.128304+0000","last_peered":"2023-11-24T08:08:49.128304+0000","last_clean":"2023-11-24T08:08:49.128304+0000","last_became_active":"2023-11-24T08:08:48.145620+0000","last_became_peered":"2023-11-24T08:08:48.145620+0000","last_unstale":"2023-11-24T08:08:49.128304+0000","last_undegraded":"2023-11-24T08:08:49.128304+0000","last_fullsized":"2023-11-24T08:08:49.128304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.167909+0000","last_change":"2023-11-24T08:08:48.164711+0000","last_active":"2023-11-24T08:08:48.167909+0000","last_peered":"2023-11-24T08:08:48.167909+0000","last_clean":"2023-11-24T08:08:48.167909+0000","last_became_active":"2023-11-24T08:08:48.164331+0000","last_became_peered":"2023-11-24T08:08:48.164331+0000","last_unstale":"2023-11-24T08:08:48.167909+0000","last_undegraded":"2023-11-24T08:08:48.167909+0000","last_fullsized":"2023-11-24T08:08:48.167909+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127224+0000","last_change":"2023-11-24T08:08:48.148716+0000","last_active":"2023-11-24T08:08:49.127224+0000","last_peered":"2023-11-24T08:08:49.127224+0000","last_clean":"2023-11-24T08:08:49.127224+0000","last_became_active":"2023-11-24T08:08:48.148271+0000","last_became_peered":"2023-11-24T08:08:48.148271+0000","last_unstale":"2023-11-24T08:08:49.127224+0000","last_undegraded":"2023-11-24T08:08:49.127224+0000","last_fullsized":"2023-11-24T08:08:49.127224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:14.407973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164097+0000","last_change":"2023-11-24T08:08:48.156383+0000","last_active":"2023-11-24T08:08:48.164097+0000","last_peered":"2023-11-24T08:08:48.164097+0000","last_clean":"2023-11-24T08:08:48.164097+0000","last_became_active":"2023-11-24T08:08:48.156083+0000","last_became_peered":"2023-11-24T08:08:48.156083+0000","last_unstale":"2023-11-24T08:08:48.164097+0000","last_undegraded":"2023-11-24T08:08:48.164097+0000","last_fullsized":"2023-11-24T08:08:48.164097+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:38.196670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126030+0000","last_change":"2023-11-24T08:08:48.157282+0000","last_active":"2023-11-24T08:08:49.126030+0000","last_peered":"2023-11-24T08:08:49.126030+0000","last_clean":"2023-11-24T08:08:49.126030+0000","last_became_active":"2023-11-24T08:08:48.155003+0000","last_became_peered":"2023-11-24T08:08:48.155003+0000","last_unstale":"2023-11-24T08:08:49.126030+0000","last_undegraded":"2023-11-24T08:08:49.126030+0000","last_fullsized":"2023-11-24T08:08:49.126030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:02.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.150367+0000","last_change":"2023-11-24T08:08:48.146014+0000","last_active":"2023-11-24T08:08:48.150367+0000","last_peered":"2023-11-24T08:08:48.150367+0000","last_clean":"2023-11-24T08:08:48.150367+0000","last_became_active":"2023-11-24T08:08:48.144834+0000","last_became_peered":"2023-11-24T08:08:48.144834+0000","last_unstale":"2023-11-24T08:08:48.150367+0000","last_undegraded":"2023-11-24T08:08:48.150367+0000","last_fullsized":"2023-11-24T08:08:48.150367+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:19.982732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125567+0000","last_change":"2023-11-24T08:08:48.142247+0000","last_active":"2023-11-24T08:08:49.125567+0000","last_peered":"2023-11-24T08:08:49.125567+0000","last_clean":"2023-11-24T08:08:49.125567+0000","last_became_active":"2023-11-24T08:08:48.141969+0000","last_became_peered":"2023-11-24T08:08:48.141969+0000","last_unstale":"2023-11-24T08:08:49.125567+0000","last_undegraded":"2023-11-24T08:08:49.125567+0000","last_fullsized":"2023-11-24T08:08:49.125567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126146+0000","last_change":"2023-11-24T08:08:48.134993+0000","last_active":"2023-11-24T08:08:49.126146+0000","last_peered":"2023-11-24T08:08:49.126146+0000","last_clean":"2023-11-24T08:08:49.126146+0000","last_became_active":"2023-11-24T08:08:48.134752+0000","last_became_peered":"2023-11-24T08:08:48.134752+0000","last_unstale":"2023-11-24T08:08:49.126146+0000","last_undegraded":"2023-11-24T08:08:49.126146+0000","last_fullsized":"2023-11-24T08:08:49.126146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.155541+0000","last_change":"2023-11-24T08:08:48.147386+0000","last_active":"2023-11-24T08:08:48.155541+0000","last_peered":"2023-11-24T08:08:48.155541+0000","last_clean":"2023-11-24T08:08:48.155541+0000","last_became_active":"2023-11-24T08:08:48.144941+0000","last_became_peered":"2023-11-24T08:08:48.144941+0000","last_unstale":"2023-11-24T08:08:48.155541+0000","last_undegraded":"2023-11-24T08:08:48.155541+0000","last_fullsized":"2023-11-24T08:08:48.155541+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.161304+0000","last_change":"2023-11-24T08:08:48.153360+0000","last_active":"2023-11-24T08:08:48.161304+0000","last_peered":"2023-11-24T08:08:48.161304+0000","last_clean":"2023-11-24T08:08:48.161304+0000","last_became_active":"2023-11-24T08:08:48.153099+0000","last_became_peered":"2023-11-24T08:08:48.153099+0000","last_unstale":"2023-11-24T08:08:48.161304+0000","last_undegraded":"2023-11-24T08:08:48.161304+0000","last_fullsized":"2023-11-24T08:08:48.161304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:38.196670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127574+0000","last_change":"2023-11-24T08:08:48.156712+0000","last_active":"2023-11-24T08:08:49.127574+0000","last_peered":"2023-11-24T08:08:49.127574+0000","last_clean":"2023-11-24T08:08:49.127574+0000","last_became_active":"2023-11-24T08:08:48.151074+0000","last_became_peered":"2023-11-24T08:08:48.151074+0000","last_unstale":"2023-11-24T08:08:49.127574+0000","last_undegraded":"2023-11-24T08:08:49.127574+0000","last_fullsized":"2023-11-24T08:08:49.127574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.079373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.156491+0000","last_change":"2023-11-24T08:08:48.149394+0000","last_active":"2023-11-24T08:08:48.156491+0000","last_peered":"2023-11-24T08:08:48.156491+0000","last_clean":"2023-11-24T08:08:48.156491+0000","last_became_active":"2023-11-24T08:08:48.149158+0000","last_became_peered":"2023-11-24T08:08:48.149158+0000","last_unstale":"2023-11-24T08:08:48.156491+0000","last_undegraded":"2023-11-24T08:08:48.156491+0000","last_fullsized":"2023-11-24T08:08:48.156491+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:09.396602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124630+0000","last_change":"2023-11-24T08:08:48.155627+0000","last_active":"2023-11-24T08:08:49.124630+0000","last_peered":"2023-11-24T08:08:49.124630+0000","last_clean":"2023-11-24T08:08:49.124630+0000","last_became_active":"2023-11-24T08:08:48.155343+0000","last_became_peered":"2023-11-24T08:08:48.155343+0000","last_unstale":"2023-11-24T08:08:49.124630+0000","last_undegraded":"2023-11-24T08:08:49.124630+0000","last_fullsized":"2023-11-24T08:08:49.124630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:00.726435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107026+0000","last_change":"2023-11-24T08:08:48.141457+0000","last_active":"2023-11-24T08:08:50.107026+0000","last_peered":"2023-11-24T08:08:50.107026+0000","last_clean":"2023-11-24T08:08:50.107026+0000","last_became_active":"2023-11-24T08:08:48.139546+0000","last_became_peered":"2023-11-24T08:08:48.139546+0000","last_unstale":"2023-11-24T08:08:50.107026+0000","last_undegraded":"2023-11-24T08:08:50.107026+0000","last_fullsized":"2023-11-24T08:08:50.107026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.167290+0000","last_change":"2023-11-24T08:08:47.144970+0000","last_active":"2023-11-24T08:08:48.167290+0000","last_peered":"2023-11-24T08:08:48.167290+0000","last_clean":"2023-11-24T08:08:48.167290+0000","last_became_active":"2023-11-24T08:08:47.144667+0000","last_became_peered":"2023-11-24T08:08:47.144667+0000","last_unstale":"2023-11-24T08:08:48.167290+0000","last_undegraded":"2023-11-24T08:08:48.167290+0000","last_fullsized":"2023-11-24T08:08:48.167290+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:59.483267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162425+0000","last_change":"2023-11-24T08:08:48.158059+0000","last_active":"2023-11-24T08:08:48.162425+0000","last_peered":"2023-11-24T08:08:48.162425+0000","last_clean":"2023-11-24T08:08:48.162425+0000","last_became_active":"2023-11-24T08:08:48.157743+0000","last_became_peered":"2023-11-24T08:08:48.157743+0000","last_unstale":"2023-11-24T08:08:48.162425+0000","last_undegraded":"2023-11-24T08:08:48.162425+0000","last_fullsized":"2023-11-24T08:08:48.162425+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125078+0000","last_change":"2023-11-24T08:08:47.135247+0000","last_active":"2023-11-24T08:08:49.125078+0000","last_peered":"2023-11-24T08:08:49.125078+0000","last_clean":"2023-11-24T08:08:49.125078+0000","last_became_active":"2023-11-24T08:08:47.134923+0000","last_became_peered":"2023-11-24T08:08:47.134923+0000","last_unstale":"2023-11-24T08:08:49.125078+0000","last_undegraded":"2023-11-24T08:08:49.125078+0000","last_fullsized":"2023-11-24T08:08:49.125078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.131152+0000","last_change":"2023-11-24T08:08:47.151833+0000","last_active":"2023-11-24T08:08:48.131152+0000","last_peered":"2023-11-24T08:08:48.131152+0000","last_clean":"2023-11-24T08:08:48.131152+0000","last_became_active":"2023-11-24T08:08:47.151517+0000","last_became_peered":"2023-11-24T08:08:47.151517+0000","last_unstale":"2023-11-24T08:08:48.131152+0000","last_undegraded":"2023-11-24T08:08:48.131152+0000","last_fullsized":"2023-11-24T08:08:48.131152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.155308+0000","last_change":"2023-11-24T08:08:48.148262+0000","last_active":"2023-11-24T08:08:48.155308+0000","last_peered":"2023-11-24T08:08:48.155308+0000","last_clean":"2023-11-24T08:08:48.155308+0000","last_became_active":"2023-11-24T08:08:48.147874+0000","last_became_peered":"2023-11-24T08:08:48.147874+0000","last_unstale":"2023-11-24T08:08:48.155308+0000","last_undegraded":"2023-11-24T08:08:48.155308+0000","last_fullsized":"2023-11-24T08:08:48.155308+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:09.396602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126262+0000","last_change":"2023-11-24T08:08:47.144280+0000","last_active":"2023-11-24T08:08:49.126262+0000","last_peered":"2023-11-24T08:08:49.126262+0000","last_clean":"2023-11-24T08:08:49.126262+0000","last_became_active":"2023-11-24T08:08:47.143933+0000","last_became_peered":"2023-11-24T08:08:47.143933+0000","last_unstale":"2023-11-24T08:08:49.126262+0000","last_undegraded":"2023-11-24T08:08:49.126262+0000","last_fullsized":"2023-11-24T08:08:49.126262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:28.820828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125379+0000","last_change":"2023-11-24T08:08:48.151024+0000","last_active":"2023-11-24T08:08:49.125379+0000","last_peered":"2023-11-24T08:08:49.125379+0000","last_clean":"2023-11-24T08:08:49.125379+0000","last_became_active":"2023-11-24T08:08:48.150727+0000","last_became_peered":"2023-11-24T08:08:48.150727+0000","last_unstale":"2023-11-24T08:08:49.125379+0000","last_undegraded":"2023-11-24T08:08:49.125379+0000","last_fullsized":"2023-11-24T08:08:49.125379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127936+0000","last_change":"2023-11-24T08:08:47.158190+0000","last_active":"2023-11-24T08:08:49.127936+0000","last_peered":"2023-11-24T08:08:49.127936+0000","last_clean":"2023-11-24T08:08:49.127936+0000","last_became_active":"2023-11-24T08:08:47.157894+0000","last_became_peered":"2023-11-24T08:08:47.157894+0000","last_unstale":"2023-11-24T08:08:49.127936+0000","last_undegraded":"2023-11-24T08:08:49.127936+0000","last_fullsized":"2023-11-24T08:08:49.127936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:18.739564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127729+0000","last_change":"2023-11-24T08:08:48.152122+0000","last_active":"2023-11-24T08:08:49.127729+0000","last_peered":"2023-11-24T08:08:49.127729+0000","last_clean":"2023-11-24T08:08:49.127729+0000","last_became_active":"2023-11-24T08:08:48.151509+0000","last_became_peered":"2023-11-24T08:08:48.151509+0000","last_unstale":"2023-11-24T08:08:49.127729+0000","last_undegraded":"2023-11-24T08:08:49.127729+0000","last_fullsized":"2023-11-24T08:08:49.127729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:17.382456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127836+0000","last_change":"2023-11-24T08:08:47.154457+0000","last_active":"2023-11-24T08:08:49.127836+0000","last_peered":"2023-11-24T08:08:49.127836+0000","last_clean":"2023-11-24T08:08:49.127836+0000","last_became_active":"2023-11-24T08:08:47.154137+0000","last_became_peered":"2023-11-24T08:08:47.154137+0000","last_unstale":"2023-11-24T08:08:49.127836+0000","last_undegraded":"2023-11-24T08:08:49.127836+0000","last_fullsized":"2023-11-24T08:08:49.127836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:19.309577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162515+0000","last_change":"2023-11-24T08:08:48.154269+0000","last_active":"2023-11-24T08:08:48.162515+0000","last_peered":"2023-11-24T08:08:48.162515+0000","last_clean":"2023-11-24T08:08:48.162515+0000","last_became_active":"2023-11-24T08:08:48.153873+0000","last_became_peered":"2023-11-24T08:08:48.153873+0000","last_unstale":"2023-11-24T08:08:48.162515+0000","last_undegraded":"2023-11-24T08:08:48.162515+0000","last_fullsized":"2023-11-24T08:08:48.162515+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:47:09.396602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126169+0000","last_change":"2023-11-24T08:08:47.156467+0000","last_active":"2023-11-24T08:08:49.126169+0000","last_peered":"2023-11-24T08:08:49.126169+0000","last_clean":"2023-11-24T08:08:49.126169+0000","last_became_active":"2023-11-24T08:08:47.155999+0000","last_became_peered":"2023-11-24T08:08:47.155999+0000","last_unstale":"2023-11-24T08:08:49.126169+0000","last_undegraded":"2023-11-24T08:08:49.126169+0000","last_fullsized":"2023-11-24T08:08:49.126169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107764+0000","last_change":"2023-11-24T08:08:48.142662+0000","last_active":"2023-11-24T08:08:50.107764+0000","last_peered":"2023-11-24T08:08:50.107764+0000","last_clean":"2023-11-24T08:08:50.107764+0000","last_became_active":"2023-11-24T08:08:48.139505+0000","last_became_peered":"2023-11-24T08:08:48.139505+0000","last_unstale":"2023-11-24T08:08:50.107764+0000","last_undegraded":"2023-11-24T08:08:50.107764+0000","last_fullsized":"2023-11-24T08:08:50.107764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:01.498497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125359+0000","last_change":"2023-11-24T08:08:47.148708+0000","last_active":"2023-11-24T08:08:49.125359+0000","last_peered":"2023-11-24T08:08:49.125359+0000","last_clean":"2023-11-24T08:08:49.125359+0000","last_became_active":"2023-11-24T08:08:47.148386+0000","last_became_peered":"2023-11-24T08:08:47.148386+0000","last_unstale":"2023-11-24T08:08:49.125359+0000","last_undegraded":"2023-11-24T08:08:49.125359+0000","last_fullsized":"2023-11-24T08:08:49.125359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353438+0000","last_change":"2023-11-24T08:08:48.147343+0000","last_active":"2023-11-24T08:08:49.353438+0000","last_peered":"2023-11-24T08:08:49.353438+0000","last_clean":"2023-11-24T08:08:49.353438+0000","last_became_active":"2023-11-24T08:08:48.146625+0000","last_became_peered":"2023-11-24T08:08:48.146625+0000","last_unstale":"2023-11-24T08:08:49.353438+0000","last_undegraded":"2023-11-24T08:08:49.353438+0000","last_fullsized":"2023-11-24T08:08:49.353438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.269500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.161250+0000","last_change":"2023-11-24T08:08:47.141638+0000","last_active":"2023-11-24T08:08:48.161250+0000","last_peered":"2023-11-24T08:08:48.161250+0000","last_clean":"2023-11-24T08:08:48.161250+0000","last_became_active":"2023-11-24T08:08:47.141294+0000","last_became_peered":"2023-11-24T08:08:47.141294+0000","last_unstale":"2023-11-24T08:08:48.161250+0000","last_undegraded":"2023-11-24T08:08:48.161250+0000","last_fullsized":"2023-11-24T08:08:48.161250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164226+0000","last_change":"2023-11-24T08:08:48.159790+0000","last_active":"2023-11-24T08:08:48.164226+0000","last_peered":"2023-11-24T08:08:48.164226+0000","last_clean":"2023-11-24T08:08:48.164226+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:48.164226+0000","last_undegraded":"2023-11-24T08:08:48.164226+0000","last_fullsized":"2023-11-24T08:08:48.164226+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127841+0000","last_change":"2023-11-24T08:08:47.157623+0000","last_active":"2023-11-24T08:08:49.127841+0000","last_peered":"2023-11-24T08:08:49.127841+0000","last_clean":"2023-11-24T08:08:49.127841+0000","last_became_active":"2023-11-24T08:08:47.156256+0000","last_became_peered":"2023-11-24T08:08:47.156256+0000","last_unstale":"2023-11-24T08:08:49.127841+0000","last_undegraded":"2023-11-24T08:08:49.127841+0000","last_fullsized":"2023-11-24T08:08:49.127841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:59.304396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353649+0000","last_change":"2023-11-24T08:08:48.152586+0000","last_active":"2023-11-24T08:08:49.353649+0000","last_peered":"2023-11-24T08:08:49.353649+0000","last_clean":"2023-11-24T08:08:49.353649+0000","last_became_active":"2023-11-24T08:08:48.152296+0000","last_became_peered":"2023-11-24T08:08:48.152296+0000","last_unstale":"2023-11-24T08:08:49.353649+0000","last_undegraded":"2023-11-24T08:08:49.353649+0000","last_fullsized":"2023-11-24T08:08:49.353649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:23.916532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124531+0000","last_change":"2023-11-24T08:08:47.136838+0000","last_active":"2023-11-24T08:08:49.124531+0000","last_peered":"2023-11-24T08:08:49.124531+0000","last_clean":"2023-11-24T08:08:49.124531+0000","last_became_active":"2023-11-24T08:08:47.136397+0000","last_became_peered":"2023-11-24T08:08:47.136397+0000","last_unstale":"2023-11-24T08:08:49.124531+0000","last_undegraded":"2023-11-24T08:08:49.124531+0000","last_fullsized":"2023-11-24T08:08:49.124531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125520+0000","last_change":"2023-11-24T08:08:48.153870+0000","last_active":"2023-11-24T08:08:49.125520+0000","last_peered":"2023-11-24T08:08:49.125520+0000","last_clean":"2023-11-24T08:08:49.125520+0000","last_became_active":"2023-11-24T08:08:48.153566+0000","last_became_peered":"2023-11-24T08:08:48.153566+0000","last_unstale":"2023-11-24T08:08:49.125520+0000","last_undegraded":"2023-11-24T08:08:49.125520+0000","last_fullsized":"2023-11-24T08:08:49.125520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.169106+0000","last_change":"2023-11-24T08:08:47.133661+0000","last_active":"2023-11-24T08:08:48.169106+0000","last_peered":"2023-11-24T08:08:48.169106+0000","last_clean":"2023-11-24T08:08:48.169106+0000","last_became_active":"2023-11-24T08:08:47.133327+0000","last_became_peered":"2023-11-24T08:08:47.133327+0000","last_unstale":"2023-11-24T08:08:48.169106+0000","last_undegraded":"2023-11-24T08:08:48.169106+0000","last_fullsized":"2023-11-24T08:08:48.169106+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:36.361920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352992+0000","last_change":"2023-11-24T08:08:48.159918+0000","last_active":"2023-11-24T08:08:49.352992+0000","last_peered":"2023-11-24T08:08:49.352992+0000","last_clean":"2023-11-24T08:08:49.352992+0000","last_became_active":"2023-11-24T08:08:48.159604+0000","last_became_peered":"2023-11-24T08:08:48.159604+0000","last_unstale":"2023-11-24T08:08:49.352992+0000","last_undegraded":"2023-11-24T08:08:49.352992+0000","last_fullsized":"2023-11-24T08:08:49.352992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:19.645688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107838+0000","last_change":"2023-11-24T08:08:47.139479+0000","last_active":"2023-11-24T08:08:50.107838+0000","last_peered":"2023-11-24T08:08:50.107838+0000","last_clean":"2023-11-24T08:08:50.107838+0000","last_became_active":"2023-11-24T08:08:47.139162+0000","last_became_peered":"2023-11-24T08:08:47.139162+0000","last_unstale":"2023-11-24T08:08:50.107838+0000","last_undegraded":"2023-11-24T08:08:50.107838+0000","last_fullsized":"2023-11-24T08:08:50.107838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:51.916232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353918+0000","last_change":"2023-11-24T08:08:48.160119+0000","last_active":"2023-11-24T08:08:49.353918+0000","last_peered":"2023-11-24T08:08:49.353918+0000","last_clean":"2023-11-24T08:08:49.353918+0000","last_became_active":"2023-11-24T08:08:48.159806+0000","last_became_peered":"2023-11-24T08:08:48.159806+0000","last_unstale":"2023-11-24T08:08:49.353918+0000","last_undegraded":"2023-11-24T08:08:49.353918+0000","last_fullsized":"2023-11-24T08:08:49.353918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:19.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.169765+0000","last_change":"2023-11-24T08:08:47.139000+0000","last_active":"2023-11-24T08:08:48.169765+0000","last_peered":"2023-11-24T08:08:48.169765+0000","last_clean":"2023-11-24T08:08:48.169765+0000","last_became_active":"2023-11-24T08:08:47.138477+0000","last_became_peered":"2023-11-24T08:08:47.138477+0000","last_unstale":"2023-11-24T08:08:48.169765+0000","last_undegraded":"2023-11-24T08:08:48.169765+0000","last_fullsized":"2023-11-24T08:08:48.169765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:49.285847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354329+0000","last_change":"2023-11-24T08:08:48.162632+0000","last_active":"2023-11-24T08:08:49.354329+0000","last_peered":"2023-11-24T08:08:49.354329+0000","last_clean":"2023-11-24T08:08:49.354329+0000","last_became_active":"2023-11-24T08:08:48.160274+0000","last_became_peered":"2023-11-24T08:08:48.160274+0000","last_unstale":"2023-11-24T08:08:49.354329+0000","last_undegraded":"2023-11-24T08:08:49.354329+0000","last_fullsized":"2023-11-24T08:08:49.354329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:23.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126343+0000","last_change":"2023-11-24T08:08:47.133083+0000","last_active":"2023-11-24T08:08:49.126343+0000","last_peered":"2023-11-24T08:08:49.126343+0000","last_clean":"2023-11-24T08:08:49.126343+0000","last_became_active":"2023-11-24T08:08:47.132746+0000","last_became_peered":"2023-11-24T08:08:47.132746+0000","last_unstale":"2023-11-24T08:08:49.126343+0000","last_undegraded":"2023-11-24T08:08:49.126343+0000","last_fullsized":"2023-11-24T08:08:49.126343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:25.146390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126883+0000","last_change":"2023-11-24T08:08:48.151332+0000","last_active":"2023-11-24T08:08:49.126883+0000","last_peered":"2023-11-24T08:08:49.126883+0000","last_clean":"2023-11-24T08:08:49.126883+0000","last_became_active":"2023-11-24T08:08:48.151045+0000","last_became_peered":"2023-11-24T08:08:48.151045+0000","last_unstale":"2023-11-24T08:08:49.126883+0000","last_undegraded":"2023-11-24T08:08:49.126883+0000","last_fullsized":"2023-11-24T08:08:49.126883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:15.646317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.132022+0000","last_change":"2023-11-24T08:08:47.150523+0000","last_active":"2023-11-24T08:08:48.132022+0000","last_peered":"2023-11-24T08:08:48.132022+0000","last_clean":"2023-11-24T08:08:48.132022+0000","last_became_active":"2023-11-24T08:08:47.150208+0000","last_became_peered":"2023-11-24T08:08:47.150208+0000","last_unstale":"2023-11-24T08:08:48.132022+0000","last_undegraded":"2023-11-24T08:08:48.132022+0000","last_fullsized":"2023-11-24T08:08:48.132022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353456+0000","last_change":"2023-11-24T08:08:48.156319+0000","last_active":"2023-11-24T08:08:49.353456+0000","last_peered":"2023-11-24T08:08:49.353456+0000","last_clean":"2023-11-24T08:08:49.353456+0000","last_became_active":"2023-11-24T08:08:48.154550+0000","last_became_peered":"2023-11-24T08:08:48.154550+0000","last_unstale":"2023-11-24T08:08:49.353456+0000","last_undegraded":"2023-11-24T08:08:49.353456+0000","last_fullsized":"2023-11-24T08:08:49.353456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:17.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.135471+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:08:48.135471+0000","last_peered":"2023-11-24T08:08:48.135471+0000","last_clean":"2023-11-24T08:08:48.135471+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:08:48.135471+0000","last_undegraded":"2023-11-24T08:08:48.135471+0000","last_fullsized":"2023-11-24T08:08:48.135471+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125945+0000","last_change":"2023-11-24T08:08:47.157035+0000","last_active":"2023-11-24T08:08:49.125945+0000","last_peered":"2023-11-24T08:08:49.125945+0000","last_clean":"2023-11-24T08:08:49.125945+0000","last_became_active":"2023-11-24T08:08:47.155189+0000","last_became_peered":"2023-11-24T08:08:47.155189+0000","last_unstale":"2023-11-24T08:08:49.125945+0000","last_undegraded":"2023-11-24T08:08:49.125945+0000","last_fullsized":"2023-11-24T08:08:49.125945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:54.597005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.165805+0000","last_change":"2023-11-24T08:08:48.161239+0000","last_active":"2023-11-24T08:08:48.165805+0000","last_peered":"2023-11-24T08:08:48.165805+0000","last_clean":"2023-11-24T08:08:48.165805+0000","last_became_active":"2023-11-24T08:08:48.160530+0000","last_became_peered":"2023-11-24T08:08:48.160530+0000","last_unstale":"2023-11-24T08:08:48.165805+0000","last_undegraded":"2023-11-24T08:08:48.165805+0000","last_fullsized":"2023-11-24T08:08:48.165805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353366+0000","last_change":"2023-11-24T08:08:47.153886+0000","last_active":"2023-11-24T08:08:49.353366+0000","last_peered":"2023-11-24T08:08:49.353366+0000","last_clean":"2023-11-24T08:08:49.353366+0000","last_became_active":"2023-11-24T08:08:47.153576+0000","last_became_peered":"2023-11-24T08:08:47.153576+0000","last_unstale":"2023-11-24T08:08:49.353366+0000","last_undegraded":"2023-11-24T08:08:49.353366+0000","last_fullsized":"2023-11-24T08:08:49.353366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162413+0000","last_change":"2023-11-24T08:08:48.157955+0000","last_active":"2023-11-24T08:08:48.162413+0000","last_peered":"2023-11-24T08:08:48.162413+0000","last_clean":"2023-11-24T08:08:48.162413+0000","last_became_active":"2023-11-24T08:08:48.157636+0000","last_became_peered":"2023-11-24T08:08:48.157636+0000","last_unstale":"2023-11-24T08:08:48.162413+0000","last_undegraded":"2023-11-24T08:08:48.162413+0000","last_fullsized":"2023-11-24T08:08:48.162413+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:59:48.939925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.132244+0000","last_change":"2023-11-24T08:08:47.143826+0000","last_active":"2023-11-24T08:08:48.132244+0000","last_peered":"2023-11-24T08:08:48.132244+0000","last_clean":"2023-11-24T08:08:48.132244+0000","last_became_active":"2023-11-24T08:08:47.142226+0000","last_became_peered":"2023-11-24T08:08:47.142226+0000","last_unstale":"2023-11-24T08:08:48.132244+0000","last_undegraded":"2023-11-24T08:08:48.132244+0000","last_fullsized":"2023-11-24T08:08:48.132244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125232+0000","last_change":"2023-11-24T08:08:48.133248+0000","last_active":"2023-11-24T08:08:49.125232+0000","last_peered":"2023-11-24T08:08:49.125232+0000","last_clean":"2023-11-24T08:08:49.125232+0000","last_became_active":"2023-11-24T08:08:48.133014+0000","last_became_peered":"2023-11-24T08:08:48.133014+0000","last_unstale":"2023-11-24T08:08:49.125232+0000","last_undegraded":"2023-11-24T08:08:49.125232+0000","last_fullsized":"2023-11-24T08:08:49.125232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127822+0000","last_change":"2023-11-24T08:08:47.154380+0000","last_active":"2023-11-24T08:08:49.127822+0000","last_peered":"2023-11-24T08:08:49.127822+0000","last_clean":"2023-11-24T08:08:49.127822+0000","last_became_active":"2023-11-24T08:08:47.154066+0000","last_became_peered":"2023-11-24T08:08:47.154066+0000","last_unstale":"2023-11-24T08:08:49.127822+0000","last_undegraded":"2023-11-24T08:08:49.127822+0000","last_fullsized":"2023-11-24T08:08:49.127822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:48:29.123228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107253+0000","last_change":"2023-11-24T08:08:48.155559+0000","last_active":"2023-11-24T08:08:50.107253+0000","last_peered":"2023-11-24T08:08:50.107253+0000","last_clean":"2023-11-24T08:08:50.107253+0000","last_became_active":"2023-11-24T08:08:48.155311+0000","last_became_peered":"2023-11-24T08:08:48.155311+0000","last_unstale":"2023-11-24T08:08:50.107253+0000","last_undegraded":"2023-11-24T08:08:50.107253+0000","last_fullsized":"2023-11-24T08:08:50.107253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125803+0000","last_change":"2023-11-24T08:08:47.137292+0000","last_active":"2023-11-24T08:08:49.125803+0000","last_peered":"2023-11-24T08:08:49.125803+0000","last_clean":"2023-11-24T08:08:49.125803+0000","last_became_active":"2023-11-24T08:08:47.136872+0000","last_became_peered":"2023-11-24T08:08:47.136872+0000","last_unstale":"2023-11-24T08:08:49.125803+0000","last_undegraded":"2023-11-24T08:08:49.125803+0000","last_fullsized":"2023-11-24T08:08:49.125803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127383+0000","last_change":"2023-11-24T08:08:48.161415+0000","last_active":"2023-11-24T08:08:49.127383+0000","last_peered":"2023-11-24T08:08:49.127383+0000","last_clean":"2023-11-24T08:08:49.127383+0000","last_became_active":"2023-11-24T08:08:48.158160+0000","last_became_peered":"2023-11-24T08:08:48.158160+0000","last_unstale":"2023-11-24T08:08:49.127383+0000","last_undegraded":"2023-11-24T08:08:49.127383+0000","last_fullsized":"2023-11-24T08:08:49.127383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:24.063260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354005+0000","last_change":"2023-11-24T08:08:47.153377+0000","last_active":"2023-11-24T08:08:49.354005+0000","last_peered":"2023-11-24T08:08:49.354005+0000","last_clean":"2023-11-24T08:08:49.354005+0000","last_became_active":"2023-11-24T08:08:47.153112+0000","last_became_peered":"2023-11-24T08:08:47.153112+0000","last_unstale":"2023-11-24T08:08:49.354005+0000","last_undegraded":"2023-11-24T08:08:49.354005+0000","last_fullsized":"2023-11-24T08:08:49.354005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:39.232399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125805+0000","last_change":"2023-11-24T08:08:48.134378+0000","last_active":"2023-11-24T08:08:49.125805+0000","last_peered":"2023-11-24T08:08:49.125805+0000","last_clean":"2023-11-24T08:08:49.125805+0000","last_became_active":"2023-11-24T08:08:48.134043+0000","last_became_peered":"2023-11-24T08:08:48.134043+0000","last_unstale":"2023-11-24T08:08:49.125805+0000","last_undegraded":"2023-11-24T08:08:49.125805+0000","last_fullsized":"2023-11-24T08:08:49.125805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126267+0000","last_change":"2023-11-24T08:08:47.148275+0000","last_active":"2023-11-24T08:08:49.126267+0000","last_peered":"2023-11-24T08:08:49.126267+0000","last_clean":"2023-11-24T08:08:49.126267+0000","last_became_active":"2023-11-24T08:08:47.148025+0000","last_became_peered":"2023-11-24T08:08:47.148025+0000","last_unstale":"2023-11-24T08:08:49.126267+0000","last_undegraded":"2023-11-24T08:08:49.126267+0000","last_fullsized":"2023-11-24T08:08:49.126267+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.157434+0000","last_change":"2023-11-24T08:08:48.146544+0000","last_active":"2023-11-24T08:08:48.157434+0000","last_peered":"2023-11-24T08:08:48.157434+0000","last_clean":"2023-11-24T08:08:48.157434+0000","last_became_active":"2023-11-24T08:08:48.146269+0000","last_became_peered":"2023-11-24T08:08:48.146269+0000","last_unstale":"2023-11-24T08:08:48.157434+0000","last_undegraded":"2023-11-24T08:08:48.157434+0000","last_fullsized":"2023-11-24T08:08:48.157434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:21.412283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128129+0000","last_change":"2023-11-24T08:08:47.152442+0000","last_active":"2023-11-24T08:08:49.128129+0000","last_peered":"2023-11-24T08:08:49.128129+0000","last_clean":"2023-11-24T08:08:49.128129+0000","last_became_active":"2023-11-24T08:08:47.151763+0000","last_became_peered":"2023-11-24T08:08:47.151763+0000","last_unstale":"2023-11-24T08:08:49.128129+0000","last_undegraded":"2023-11-24T08:08:49.128129+0000","last_fullsized":"2023-11-24T08:08:49.128129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.156246+0000","last_change":"2023-11-24T08:08:48.152228+0000","last_active":"2023-11-24T08:08:48.156246+0000","last_peered":"2023-11-24T08:08:48.156246+0000","last_clean":"2023-11-24T08:08:48.156246+0000","last_became_active":"2023-11-24T08:08:48.151904+0000","last_became_peered":"2023-11-24T08:08:48.151904+0000","last_unstale":"2023-11-24T08:08:48.156246+0000","last_undegraded":"2023-11-24T08:08:48.156246+0000","last_fullsized":"2023-11-24T08:08:48.156246+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:37.605088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352672+0000","last_change":"2023-11-24T08:08:47.150060+0000","last_active":"2023-11-24T08:08:49.352672+0000","last_peered":"2023-11-24T08:08:49.352672+0000","last_clean":"2023-11-24T08:08:49.352672+0000","last_became_active":"2023-11-24T08:08:47.149756+0000","last_became_peered":"2023-11-24T08:08:47.149756+0000","last_unstale":"2023-11-24T08:08:49.352672+0000","last_undegraded":"2023-11-24T08:08:49.352672+0000","last_fullsized":"2023-11-24T08:08:49.352672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107214+0000","last_change":"2023-11-24T08:08:48.150454+0000","last_active":"2023-11-24T08:08:50.107214+0000","last_peered":"2023-11-24T08:08:50.107214+0000","last_clean":"2023-11-24T08:08:50.107214+0000","last_became_active":"2023-11-24T08:08:48.139831+0000","last_became_peered":"2023-11-24T08:08:48.139831+0000","last_unstale":"2023-11-24T08:08:50.107214+0000","last_undegraded":"2023-11-24T08:08:50.107214+0000","last_fullsized":"2023-11-24T08:08:50.107214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125621+0000","last_change":"2023-11-24T08:08:47.141901+0000","last_active":"2023-11-24T08:08:49.125621+0000","last_peered":"2023-11-24T08:08:49.125621+0000","last_clean":"2023-11-24T08:08:49.125621+0000","last_became_active":"2023-11-24T08:08:47.140760+0000","last_became_peered":"2023-11-24T08:08:47.140760+0000","last_unstale":"2023-11-24T08:08:49.125621+0000","last_undegraded":"2023-11-24T08:08:49.125621+0000","last_fullsized":"2023-11-24T08:08:49.125621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:12.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.145683+0000","last_change":"2023-11-24T08:08:48.141100+0000","last_active":"2023-11-24T08:08:48.145683+0000","last_peered":"2023-11-24T08:08:48.145683+0000","last_clean":"2023-11-24T08:08:48.145683+0000","last_became_active":"2023-11-24T08:08:48.138053+0000","last_became_peered":"2023-11-24T08:08:48.138053+0000","last_unstale":"2023-11-24T08:08:48.145683+0000","last_undegraded":"2023-11-24T08:08:48.145683+0000","last_fullsized":"2023-11-24T08:08:48.145683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:22.547643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125743+0000","last_change":"2023-11-24T08:08:47.138812+0000","last_active":"2023-11-24T08:08:49.125743+0000","last_peered":"2023-11-24T08:08:49.125743+0000","last_clean":"2023-11-24T08:08:49.125743+0000","last_became_active":"2023-11-24T08:08:47.138512+0000","last_became_peered":"2023-11-24T08:08:47.138512+0000","last_unstale":"2023-11-24T08:08:49.125743+0000","last_undegraded":"2023-11-24T08:08:49.125743+0000","last_fullsized":"2023-11-24T08:08:49.125743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.163845+0000","last_change":"2023-11-24T08:08:48.155403+0000","last_active":"2023-11-24T08:08:48.163845+0000","last_peered":"2023-11-24T08:08:48.163845+0000","last_clean":"2023-11-24T08:08:48.163845+0000","last_became_active":"2023-11-24T08:08:48.154517+0000","last_became_peered":"2023-11-24T08:08:48.154517+0000","last_unstale":"2023-11-24T08:08:48.163845+0000","last_undegraded":"2023-11-24T08:08:48.163845+0000","last_fullsized":"2023-11-24T08:08:48.163845+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:37.605088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353934+0000","last_change":"2023-11-24T08:08:47.152677+0000","last_active":"2023-11-24T08:08:49.353934+0000","last_peered":"2023-11-24T08:08:49.353934+0000","last_clean":"2023-11-24T08:08:49.353934+0000","last_became_active":"2023-11-24T08:08:47.151375+0000","last_became_peered":"2023-11-24T08:08:47.151375+0000","last_unstale":"2023-11-24T08:08:49.353934+0000","last_undegraded":"2023-11-24T08:08:49.353934+0000","last_fullsized":"2023-11-24T08:08:49.353934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:56.046702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125623+0000","last_change":"2023-11-24T08:08:48.162261+0000","last_active":"2023-11-24T08:08:49.125623+0000","last_peered":"2023-11-24T08:08:49.125623+0000","last_clean":"2023-11-24T08:08:49.125623+0000","last_became_active":"2023-11-24T08:08:48.156627+0000","last_became_peered":"2023-11-24T08:08:48.156627+0000","last_unstale":"2023-11-24T08:08:49.125623+0000","last_undegraded":"2023-11-24T08:08:49.125623+0000","last_fullsized":"2023-11-24T08:08:49.125623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127263+0000","last_change":"2023-11-24T08:08:48.162034+0000","last_active":"2023-11-24T08:08:49.127263+0000","last_peered":"2023-11-24T08:08:49.127263+0000","last_clean":"2023-11-24T08:08:49.127263+0000","last_became_active":"2023-11-24T08:08:48.161663+0000","last_became_peered":"2023-11-24T08:08:48.161663+0000","last_unstale":"2023-11-24T08:08:49.127263+0000","last_undegraded":"2023-11-24T08:08:49.127263+0000","last_fullsized":"2023-11-24T08:08:49.127263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:56.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353243+0000","last_change":"2023-11-24T08:08:47.156960+0000","last_active":"2023-11-24T08:08:49.353243+0000","last_peered":"2023-11-24T08:08:49.353243+0000","last_clean":"2023-11-24T08:08:49.353243+0000","last_became_active":"2023-11-24T08:08:47.156401+0000","last_became_peered":"2023-11-24T08:08:47.156401+0000","last_unstale":"2023-11-24T08:08:49.353243+0000","last_undegraded":"2023-11-24T08:08:49.353243+0000","last_fullsized":"2023-11-24T08:08:49.353243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:04.035911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126566+0000","last_change":"2023-11-24T08:08:48.159506+0000","last_active":"2023-11-24T08:08:49.126566+0000","last_peered":"2023-11-24T08:08:49.126566+0000","last_clean":"2023-11-24T08:08:49.126566+0000","last_became_active":"2023-11-24T08:08:48.159220+0000","last_became_peered":"2023-11-24T08:08:48.159220+0000","last_unstale":"2023-11-24T08:08:49.126566+0000","last_undegraded":"2023-11-24T08:08:49.126566+0000","last_fullsized":"2023-11-24T08:08:49.126566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:14.900691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353690+0000","last_change":"2023-11-24T08:08:47.155268+0000","last_active":"2023-11-24T08:08:49.353690+0000","last_peered":"2023-11-24T08:08:49.353690+0000","last_clean":"2023-11-24T08:08:49.353690+0000","last_became_active":"2023-11-24T08:08:47.154139+0000","last_became_peered":"2023-11-24T08:08:47.154139+0000","last_unstale":"2023-11-24T08:08:49.353690+0000","last_undegraded":"2023-11-24T08:08:49.353690+0000","last_fullsized":"2023-11-24T08:08:49.353690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.162564+0000","last_change":"2023-11-24T08:08:48.157139+0000","last_active":"2023-11-24T08:08:48.162564+0000","last_peered":"2023-11-24T08:08:48.162564+0000","last_clean":"2023-11-24T08:08:48.162564+0000","last_became_active":"2023-11-24T08:08:48.156875+0000","last_became_peered":"2023-11-24T08:08:48.156875+0000","last_unstale":"2023-11-24T08:08:48.162564+0000","last_undegraded":"2023-11-24T08:08:48.162564+0000","last_fullsized":"2023-11-24T08:08:48.162564+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:22.547643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.164239+0000","last_change":"2023-11-24T08:08:47.143443+0000","last_active":"2023-11-24T08:08:48.164239+0000","last_peered":"2023-11-24T08:08:48.164239+0000","last_clean":"2023-11-24T08:08:48.164239+0000","last_became_active":"2023-11-24T08:08:47.143156+0000","last_became_peered":"2023-11-24T08:08:47.143156+0000","last_unstale":"2023-11-24T08:08:48.164239+0000","last_undegraded":"2023-11-24T08:08:48.164239+0000","last_fullsized":"2023-11-24T08:08:48.164239+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.145461+0000","last_change":"2023-11-24T08:08:48.141063+0000","last_active":"2023-11-24T08:08:48.145461+0000","last_peered":"2023-11-24T08:08:48.145461+0000","last_clean":"2023-11-24T08:08:48.145461+0000","last_became_active":"2023-11-24T08:08:48.140809+0000","last_became_peered":"2023-11-24T08:08:48.140809+0000","last_unstale":"2023-11-24T08:08:48.145461+0000","last_undegraded":"2023-11-24T08:08:48.145461+0000","last_fullsized":"2023-11-24T08:08:48.145461+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.167730+0000","last_change":"2023-11-24T08:08:47.150905+0000","last_active":"2023-11-24T08:08:48.167730+0000","last_peered":"2023-11-24T08:08:48.167730+0000","last_clean":"2023-11-24T08:08:48.167730+0000","last_became_active":"2023-11-24T08:08:47.150670+0000","last_became_peered":"2023-11-24T08:08:47.150670+0000","last_unstale":"2023-11-24T08:08:48.167730+0000","last_undegraded":"2023-11-24T08:08:48.167730+0000","last_fullsized":"2023-11-24T08:08:48.167730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3380000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0049999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.127}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.635}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0819999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2930000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542148,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0649999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1379999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.587}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6279999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7090000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9119999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542147,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.877}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.595}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.232}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6579999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.835}]}]},{"osd":4,"up_from":14,"seq":60129542148,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.226}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.609}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.298}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3979999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.746}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.887}]}]},{"osd":3,"up_from":14,"seq":60129542148,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":225280,"data_stored":74188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.6879999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7309999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.8409999999999993}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.2729999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.0579999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1399999999999997}]}]},{"osd":2,"up_from":14,"seq":60129542148,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":72576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2529999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6479999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7869999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0129999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7509999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1379999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542148,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1850000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.329}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6879999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8280000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1400000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":819200,"data_stored":663582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0109999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.133}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9610000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3320000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:53.052 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:08:53.052 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-24T08:08:53.052 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-24T08:08:53.053 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-24T08:08:53.053 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-24T08:08:53.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.300+0000 7f4575c75640 1 Processor -- start 2023-11-24T08:08:53.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.300+0000 7f4575c75640 1 -- start start 2023-11-24T08:08:53.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.301+0000 7f4575c75640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 0x7f457015b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.301+0000 7f4575c75640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457015e3f0 con 0x7f45700b2250 2023-11-24T08:08:53.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.301+0000 7f4575c75640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457015ab80 con 0x7f457015df80 2023-11-24T08:08:53.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.301+0000 7f4575c75640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457015a990 con 0x7f457015b080 2023-11-24T08:08:53.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.301+0000 7f456f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f45700b2250 0x7f45700b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54352/0 (socket says 172.21.15.106:54352) 2023-11-24T08:08:53.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.301+0000 7f456f7fe640 1 -- 172.21.15.106:0/2743063677 learned_addr learned my addr 172.21.15.106:0/2743063677 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456ffff640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 0x7f457015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096105817 0 0) 0x7f457015e3f0 con 0x7f45700b2250 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4558005730 con 0x7f45700b2250 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4152205795 0 0) 0x7f4558005730 con 0x7f45700b2250 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 msgr2=0x7f457015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 0x7f457015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 >> v1:172.21.15.156:6789/0 conn(0x7f457015df80 legacy=0x7f45700ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45700b2d40 con 0x7f45700b2250 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f4560001670 con 0x7f45700b2250 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 >> v1:172.21.15.106:6789/0 conn(0x7f45700b2250 legacy=0x7f45700b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558005bb0 0x7f4558005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558006510 0x7f45580069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 0x7f45580071c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4558007890 con 0x7f4558006510 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455800ffc0 con 0x7f4558005bb0 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.302+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4558010180 con 0x7f457015b080 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456effd640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558006510 0x7f45580069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456f7fe640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558005bb0 0x7f4558005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456effd640 1 -- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 msgr2=0x7f45580071c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456effd640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f457015b080 0x7f45580071c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456effd640 1 -- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558005bb0 msgr2=0x7f4558005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456effd640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558005bb0 0x7f4558005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.303+0000 7f456effd640 1 -- 172.21.15.106:0/2743063677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558005730 con 0x7f4558006510 2023-11-24T08:08:53.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.304+0000 7f456effd640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558006510 0x7f45580069d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4564001750 tx=0x7f4558007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b98da36d1d9e8eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.304+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45640052d0 con 0x7f4558006510 2023-11-24T08:08:53.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.304+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456402e050 con 0x7f4558006510 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.304+0000 7f456e7fc640 1 -- 172.21.15.106:0/2743063677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45640035f0 con 0x7f4558006510 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f4575c75640 1 -- 172.21.15.106:0/2743063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558006510 msgr2=0x7f45580069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f4575c75640 1 --2- 172.21.15.106:0/2743063677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4558006510 0x7f45580069d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4564001750 tx=0x7f4558007a00 comp rx=0 tx=0).stop 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f456ffff640 1 -- 172.21.15.106:0/2743063677 reap_dead start 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f4575c75640 1 -- 172.21.15.106:0/2743063677 shutdown_connections 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f4575c75640 1 -- 172.21.15.106:0/2743063677 >> 172.21.15.106:0/2743063677 conn(0x7f457006a920 msgr2=0x7f457006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f4575c75640 1 -- 172.21.15.106:0/2743063677 shutdown_connections 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.305+0000 7f4575c75640 1 -- 172.21.15.106:0/2743063677 wait complete. 2023-11-24T08:08:53.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.307+0000 7f4575c75640 1 Processor -- start 2023-11-24T08:08:53.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.307+0000 7f4575c75640 1 -- start start 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f4575c75640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558006510 0x7f45701485d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f4575c75640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45700abfc0 0x7f45700af220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f4575c75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 0x7f45700cf420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f4575c75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457015c600 con 0x7f45700af760 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f4575c75640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457015c8e0 con 0x7f4558006510 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f4575c75640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457015cbc0 con 0x7f45700abfc0 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 0x7f45700cf420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 0x7f45700cf420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47854/0 (socket says 172.21.15.106:47854) 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 -- 172.21.15.106:0/417173587 learned_addr learned my addr 172.21.15.106:0/417173587 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:53.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456f7fe640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558006510 0x7f45701485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456effd640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45700abfc0 0x7f45700af220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 -- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45700abfc0 msgr2=0x7f45700af220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45700abfc0 0x7f45700af220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 -- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558006510 msgr2=0x7f45701485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558006510 0x7f45701485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456ffff640 1 -- 172.21.15.106:0/417173587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457015cea0 con 0x7f45700af760 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.308+0000 7f456effd640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45700abfc0 0x7f45700af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.309+0000 7f456ffff640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 0x7f45700cf420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4570158560 tx=0x7f456800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.309+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568010040 con 0x7f45700af760 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.309+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800bb70 con 0x7f45700af760 2023-11-24T08:08:53.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.309+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4568014400 con 0x7f45700af760 2023-11-24T08:08:53.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.309+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457015d180 con 0x7f45700af760 2023-11-24T08:08:53.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.309+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45700cfb40 con 0x7f45700af760 2023-11-24T08:08:53.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.310+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45700cfd60 con 0x7f45700af760 2023-11-24T08:08:53.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.311+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4568014560 con 0x7f45700af760 2023-11-24T08:08:53.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.312+0000 7f4557fff640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f455c06ae40 0x7f455c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.312+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4568089ac0 con 0x7f45700af760 2023-11-24T08:08:53.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.312+0000 7f456f7fe640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f455c06ae40 0x7f455c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.313+0000 7f456f7fe640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f455c06ae40 0x7f455c06d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45600059b0 tx=0x7f4560009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.317+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4568054a40 con 0x7f45700af760 2023-11-24T08:08:53.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.553+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f4570067470 con 0x7f45700af760 2023-11-24T08:08:53.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.562+0000 7f4557fff640 1 -- 172.21.15.106:0/417173587 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171039 (secure 0 0 0) 0x7f4568058060 con 0x7f45700af760 2023-11-24T08:08:53.565 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:53.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.566+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f455c06ae40 msgr2=0x7f455c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.566+0000 7f4575c75640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f455c06ae40 0x7f455c06d300 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f45600059b0 tx=0x7f4560009040 comp rx=0 tx=0).stop 2023-11-24T08:08:53.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.566+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 msgr2=0x7f45700cf420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.566+0000 7f4575c75640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 0x7f45700cf420 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4570158560 tx=0x7f456800ef30 comp rx=0 tx=0).stop 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 shutdown_connections 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45700af760 0x7f45700cf420 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f455c06ae40 0x7f455c06d300 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45700abfc0 0x7f45700af220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 --2- 172.21.15.106:0/417173587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4558006510 0x7f45701485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 >> 172.21.15.106:0/417173587 conn(0x7f457006a920 msgr2=0x7f4570158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 shutdown_connections 2023-11-24T08:08:53.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.567+0000 7f4575c75640 1 -- 172.21.15.106:0/417173587 wait complete. 2023-11-24T08:08:53.587 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":5,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6832","nonce":2073981325},{"type":"v1","addr":"172.21.15.106:6833","nonce":2073981325}]},"active_addr":"172.21.15.106:6833/2073981325","active_change":"2023-11-24T08:08:33.257433+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4114,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2558092332}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":1349857883}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":3287038811}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.106:0","nonce":2579183190}]}]} 2023-11-24T08:08:53.590 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-24T08:08:53.590 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-24T08:08:53.590 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:53.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.778+0000 7fc224b2c640 1 Processor -- start 2023-11-24T08:08:53.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.778+0000 7fc224b2c640 1 -- start start 2023-11-24T08:08:53.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.779+0000 7fc224b2c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc22015b080 0x7fc22015b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.779+0000 7fc224b2c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22015e3f0 con 0x7fc2200b2250 2023-11-24T08:08:53.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.779+0000 7fc224b2c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc22015ab80 con 0x7fc22015df80 2023-11-24T08:08:53.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.779+0000 7fc224b2c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc22015a990 con 0x7fc22015b080 2023-11-24T08:08:53.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21e59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc2200b2250 0x7fc2200b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54360/0 (socket says 172.21.15.106:54360) 2023-11-24T08:08:53.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21e59c640 1 -- 172.21.15.106:0/1561559825 learned_addr learned my addr 172.21.15.106:0/1561559825 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:53.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/1561559825 >> v2:172.21.15.156:3300/0 conn(0x7fc22015b080 0x7fc22015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2121512876 0 0) 0x7fc22015e3f0 con 0x7fc2200b2250 2023-11-24T08:08:53.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc20c005730 con 0x7fc2200b2250 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2682573755 0 0) 0x7fc20c005730 con 0x7fc2200b2250 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 >> v2:172.21.15.156:3300/0 conn(0x7fc22015b080 msgr2=0x7fc22015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 --2- 172.21.15.106:0/1561559825 >> v2:172.21.15.156:3300/0 conn(0x7fc22015b080 0x7fc22015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 >> v1:172.21.15.156:6789/0 conn(0x7fc22015df80 legacy=0x7fc2200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200b2d40 con 0x7fc2200b2250 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc210001670 con 0x7fc2200b2250 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 >> v1:172.21.15.106:6789/0 conn(0x7fc2200b2250 legacy=0x7fc2200b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.780+0000 7fc21dd9b640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c005bd0 0x7fc20c005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21dd9b640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc20c0069d0 0x7fc20c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21dd9b640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c007390 0x7fc20c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c00e260 con 0x7fc20c0069d0 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c00e3d0 con 0x7fc20c005bd0 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20c00e590 con 0x7fc20c007390 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc217fff640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc20c0069d0 0x7fc20c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21e59c640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c005bd0 0x7fc20c005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c007390 0x7fc20c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc217fff640 1 -- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c007390 msgr2=0x7fc20c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc217fff640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c007390 0x7fc20c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc217fff640 1 -- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c005bd0 msgr2=0x7fc20c005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc217fff640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c005bd0 0x7fc20c005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc217fff640 1 -- 172.21.15.106:0/1561559825 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20c005730 con 0x7fc20c0069d0 2023-11-24T08:08:53.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc20c007390 0x7fc20c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:53.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.781+0000 7fc21ed9d640 1 -- 172.21.15.106:0/1561559825 reap_dead start 2023-11-24T08:08:53.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.782+0000 7fc217fff640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc20c0069d0 0x7fc20c006e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc208001750 tx=0x7fc20c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f885e252599f0d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080052d0 con 0x7fc20c0069d0 2023-11-24T08:08:53.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20802e050 con 0x7fc20c0069d0 2023-11-24T08:08:53.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc21dd9b640 1 -- 172.21.15.106:0/1561559825 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208003830 con 0x7fc20c0069d0 2023-11-24T08:08:53.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 -- 172.21.15.106:0/1561559825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc20c0069d0 msgr2=0x7fc20c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc20c0069d0 0x7fc20c006e50 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc208001750 tx=0x7fc20c007a00 comp rx=0 tx=0).stop 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 -- 172.21.15.106:0/1561559825 shutdown_connections 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 --2- 172.21.15.106:0/1561559825 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc20c0069d0 0x7fc20c006e50 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 -- 172.21.15.106:0/1561559825 >> 172.21.15.106:0/1561559825 conn(0x7fc22006a920 msgr2=0x7fc22006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 -- 172.21.15.106:0/1561559825 shutdown_connections 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.783+0000 7fc224b2c640 1 -- 172.21.15.106:0/1561559825 wait complete. 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.785+0000 7fc224b2c640 1 Processor -- start 2023-11-24T08:08:53.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.785+0000 7fc224b2c640 1 -- start start 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc224b2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0069d0 0x7fc2201512a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc224b2c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 0x7fc2200b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc224b2c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc220071610 0x7fc220071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc224b2c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2201517e0 con 0x7fc2200af6d0 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc224b2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220071fd0 con 0x7fc20c0069d0 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc224b2c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200c9b80 con 0x7fc220071610 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.786+0000 7fc217fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 0x7fc2200b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 0x7fc2200b1480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47872/0 (socket says 172.21.15.106:47872) 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 -- 172.21.15.106:0/825248246 learned_addr learned my addr 172.21.15.106:0/825248246 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc21e59c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0069d0 0x7fc2201512a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc220071610 0x7fc220071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 -- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc220071610 msgr2=0x7fc220071a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc220071610 0x7fc220071a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 -- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0069d0 msgr2=0x7fc2201512a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0069d0 0x7fc2201512a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 -- 172.21.15.106:0/825248246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2200c9e60 con 0x7fc2200af6d0 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc217fff640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 0x7fc2200b1480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc208000c00 tx=0x7fc208001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2080076c0 con 0x7fc2200af6d0 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc208007a30 con 0x7fc2200af6d0 2023-11-24T08:08:53.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc208006be0 con 0x7fc2200af6d0 2023-11-24T08:08:53.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.787+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2200ca140 con 0x7fc2200af6d0 2023-11-24T08:08:53.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.788+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2200ca700 con 0x7fc2200af6d0 2023-11-24T08:08:53.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.789+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc220072810 con 0x7fc2200af6d0 2023-11-24T08:08:53.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.789+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc208006d40 con 0x7fc2200af6d0 2023-11-24T08:08:53.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.790+0000 7fc216ffd640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006ae70 0x7fc20006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:53.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.790+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2080ab270 con 0x7fc2200af6d0 2023-11-24T08:08:53.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.790+0000 7fc21e59c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006ae70 0x7fc20006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:53.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.791+0000 7fc21e59c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006ae70 0x7fc20006d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc210001050 tx=0x7fc210009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:53.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.795+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc208076310 con 0x7fc2200af6d0 2023-11-24T08:08:53.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.973+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc220067470 con 0x7fc2200af6d0 2023-11-24T08:08:53.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.975+0000 7fc216ffd640 1 -- 172.21.15.106:0/825248246 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fc20807ee60 con 0x7fc2200af6d0 2023-11-24T08:08:53.976 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:53.977 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:53.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.977+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006ae70 msgr2=0x7fc20006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.977+0000 7fc224b2c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006ae70 0x7fc20006d330 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fc210001050 tx=0x7fc210009040 comp rx=0 tx=0).stop 2023-11-24T08:08:53.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.977+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 msgr2=0x7fc2200b1480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:53.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.977+0000 7fc224b2c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 0x7fc2200b1480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc208000c00 tx=0x7fc208001120 comp rx=0 tx=0).stop 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 shutdown_connections 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006ae70 0x7fc20006d330 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc220071610 0x7fc220071a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2200af6d0 0x7fc2200b1480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 --2- 172.21.15.106:0/825248246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc20c0069d0 0x7fc2201512a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 >> 172.21.15.106:0/825248246 conn(0x7fc22006a920 msgr2=0x7fc2200ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 shutdown_connections 2023-11-24T08:08:53.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:53.978+0000 7fc224b2c640 1 -- 172.21.15.106:0/825248246 wait complete. 2023-11-24T08:08:53.998 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-24T08:08:53.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:54.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.194+0000 7f2dfd9a9640 1 Processor -- start 2023-11-24T08:08:54.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.194+0000 7f2dfd9a9640 1 -- start start 2023-11-24T08:08:54.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dfd9a9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2df8159ce0 0x7f2df806de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dfd9a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2df80b21d0 0x7f2df80cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dfd9a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df80b3a40 con 0x7f2df80cbf80 2023-11-24T08:08:54.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dfd9a9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df80cc3d0 con 0x7f2df80b21d0 2023-11-24T08:08:54.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dfd9a9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df80cdfa0 con 0x7f2df8159ce0 2023-11-24T08:08:54.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dee7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2df80b21d0 0x7f2df80cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2df77fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2df80cbf80 0x7f2df815d760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60938/0 (socket says 172.21.15.106:60938) 2023-11-24T08:08:54.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dee7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2df80b21d0 0x7f2df80cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34146/0 (socket says 172.21.15.106:34146) 2023-11-24T08:08:54.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.196+0000 7f2dee7fc640 1 -- 172.21.15.106:0/2544099181 learned_addr learned my addr 172.21.15.106:0/2544099181 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.197+0000 7f2df6ffd640 1 --2- 172.21.15.106:0/2544099181 >> v2:172.21.15.156:3300/0 conn(0x7f2df8159ce0 0x7f2df806de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.197+0000 7f2dee7fc640 1 -- 172.21.15.106:0/2544099181 >> v2:172.21.15.156:3300/0 conn(0x7f2df8159ce0 msgr2=0x7f2df806de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.197+0000 7f2dee7fc640 1 --2- 172.21.15.106:0/2544099181 >> v2:172.21.15.156:3300/0 conn(0x7f2df8159ce0 0x7f2df806de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.197+0000 7f2dee7fc640 1 -- 172.21.15.106:0/2544099181 >> v1:172.21.15.106:6789/0 conn(0x7f2df80cbf80 legacy=0x7f2df815d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.197+0000 7f2df67fc640 1 -- 172.21.15.106:0/2544099181 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401483974 0 0) 0x7f2df80b3a40 con 0x7f2df80cbf80 2023-11-24T08:08:54.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.197+0000 7f2dee7fc640 1 -- 172.21.15.106:0/2544099181 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df80cd9e0 con 0x7f2df80b21d0 2023-11-24T08:08:54.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.198+0000 7f2dee7fc640 1 --2- 172.21.15.106:0/2544099181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df80b21d0 0x7f2df80cba40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2df806e770 tx=0x7f2de002e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f229523025041c68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.198+0000 7f2df67fc640 1 -- 172.21.15.106:0/2544099181 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0036440 con 0x7f2df80b21d0 2023-11-24T08:08:54.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.198+0000 7f2df67fc640 1 -- 172.21.15.106:0/2544099181 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2de00365a0 con 0x7f2df80b21d0 2023-11-24T08:08:54.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.198+0000 7f2df67fc640 1 -- 172.21.15.106:0/2544099181 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0036930 con 0x7f2df80b21d0 2023-11-24T08:08:54.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/2544099181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df80b21d0 msgr2=0x7f2df80cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/2544099181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df80b21d0 0x7f2df80cba40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2df806e770 tx=0x7f2de002e170 comp rx=0 tx=0).stop 2023-11-24T08:08:54.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/2544099181 shutdown_connections 2023-11-24T08:08:54.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/2544099181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df80b21d0 0x7f2df80cba40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/2544099181 >> v2:172.21.15.156:3300/0 conn(0x7f2df8159ce0 0x7f2df806de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/2544099181 >> 172.21.15.106:0/2544099181 conn(0x7f2df806a920 msgr2=0x7f2df806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/2544099181 shutdown_connections 2023-11-24T08:08:54.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.199+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/2544099181 wait complete. 2023-11-24T08:08:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.201+0000 7f2dfd9a9640 1 Processor -- start 2023-11-24T08:08:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.201+0000 7f2dfd9a9640 1 -- start start 2023-11-24T08:08:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2dfd9a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df80b21d0 0x7f2df8152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2dfd9a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df80cbf80 0x7f2df809a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2dfd9a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 0x7f2df809c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2dfd9a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80af440 con 0x7f2df8159ce0 2023-11-24T08:08:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2dfd9a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8152dd0 con 0x7f2df80b21d0 2023-11-24T08:08:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2dfd9a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df809aa40 con 0x7f2df80cbf80 2023-11-24T08:08:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 0x7f2df809c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 0x7f2df809c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34154/0 (socket says 172.21.15.106:34154) 2023-11-24T08:08:54.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2df77fe640 1 -- 172.21.15.106:0/1717067666 learned_addr learned my addr 172.21.15.106:0/1717067666 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.202+0000 7f2df6ffd640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df80b21d0 0x7f2df8152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2dee7fc640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df80cbf80 0x7f2df809a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df77fe640 1 -- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df80cbf80 msgr2=0x7f2df809a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df77fe640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df80cbf80 0x7f2df809a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df77fe640 1 -- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df80b21d0 msgr2=0x7f2df8152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df77fe640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df80b21d0 0x7f2df8152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df77fe640 1 -- 172.21.15.106:0/1717067666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de000b770 con 0x7f2df8159ce0 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df77fe640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 0x7f2df809c250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2df000c780 tx=0x7f2df000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df000ce30 con 0x7f2df8159ce0 2023-11-24T08:08:54.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df000a640 con 0x7f2df8159ce0 2023-11-24T08:08:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df80aed80 con 0x7f2df8159ce0 2023-11-24T08:08:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df000a9f0 con 0x7f2df8159ce0 2023-11-24T08:08:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.203+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df809ad20 con 0x7f2df8159ce0 2023-11-24T08:08:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.205+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df80af580 con 0x7f2df8159ce0 2023-11-24T08:08:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.205+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2df000ab50 con 0x7f2df8159ce0 2023-11-24T08:08:54.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.206+0000 7f2df4ff9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2dd806ae40 0x7f2dd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.206+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2df0088990 con 0x7f2df8159ce0 2023-11-24T08:08:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.207+0000 7f2df6ffd640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2dd806ae40 0x7f2dd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.207+0000 7f2df6ffd640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2dd806ae40 0x7f2dd806d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2de8006fd0 tx=0x7f2de8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.211+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2df80af580 con 0x7f2df8159ce0 2023-11-24T08:08:54.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.393+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2df80af580 con 0x7f2df8159ce0 2023-11-24T08:08:54.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.396+0000 7f2df4ff9640 1 -- 172.21.15.106:0/1717067666 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f2df80af580 con 0x7f2df8159ce0 2023-11-24T08:08:54.398 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:54.398 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2dd806ae40 msgr2=0x7f2dd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2dd806ae40 0x7f2dd806d300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2de8006fd0 tx=0x7f2de8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 msgr2=0x7f2df809c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 0x7f2df809c250 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2df000c780 tx=0x7f2df000cc40 comp rx=0 tx=0).stop 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 shutdown_connections 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2df8159ce0 0x7f2df809c250 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2dd806ae40 0x7f2dd806d300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2df80cbf80 0x7f2df809a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 --2- 172.21.15.106:0/1717067666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df80b21d0 0x7f2df8152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 >> 172.21.15.106:0/1717067666 conn(0x7f2df806a920 msgr2=0x7f2df8071b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 shutdown_connections 2023-11-24T08:08:54.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.398+0000 7f2dfd9a9640 1 -- 172.21.15.106:0/1717067666 wait complete. 2023-11-24T08:08:54.417 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-24T08:08:54.418 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-24T08:08:54.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-24T08:08:54.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-24T08:08:54.420 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-24T08:08:54.420 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-24T08:08:54.420 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-24T08:08:54.421 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-24T08:08:54.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.711+0000 7faddaa1a640 1 Processor -- start 2023-11-24T08:08:54.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.712+0000 7faddaa1a640 1 -- start start 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.713+0000 7faddaa1a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd415d490 0x7fadd40cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.714+0000 7faddaa1a640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fadd40b63d0 0x7fadd40d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.714+0000 7faddaa1a640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40cea30 con 0x7fadd40ce270 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.714+0000 7faddaa1a640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd40d0450 con 0x7fadd40b63d0 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.714+0000 7faddaa1a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40d0bf0 con 0x7fadd415d490 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.715+0000 7fadd9217640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fadd40b63d0 0x7fadd40d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.715+0000 7fadd9217640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fadd40b63d0 0x7fadd40d3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39462/0 (socket says 172.21.15.106:39462) 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.715+0000 7fadd9217640 1 -- 172.21.15.106:0/2260982106 learned_addr learned my addr 172.21.15.106:0/2260982106 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.715+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd415d490 0x7fadd40cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.716+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2933605255 0 0) 0x7fadd40cea30 con 0x7fadd40ce270 2023-11-24T08:08:54.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.716+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadc0003520 con 0x7fadd40ce270 2023-11-24T08:08:54.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 500286512 0 0) 0x7fadc0003520 con 0x7fadd40ce270 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd415d490 msgr2=0x7fadd40cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd415d490 0x7fadd40cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 >> v2:172.21.15.156:3300/0 conn(0x7fadd40b63d0 msgr2=0x7fadd40d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 --2- 172.21.15.106:0/2260982106 >> v2:172.21.15.156:3300/0 conn(0x7fadd40b63d0 0x7fadd40d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40d2260 con 0x7fadd40ce270 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fadd0001670 con 0x7fadd40ce270 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd9217640 1 --2- 172.21.15.106:0/2260982106 >> v2:172.21.15.156:3300/0 conn(0x7fadd40b63d0 0x7fadd40d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 >> v1:172.21.15.156:6789/0 conn(0x7fadd40ce270 legacy=0x7fadd4160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadc0003970 0x7fadc0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0004360 0x7fadc0004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadc0004f60 0x7fadc000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40d0450 con 0x7fadc0004360 2023-11-24T08:08:54.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40d0bf0 con 0x7fadc0004f60 2023-11-24T08:08:54.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.717+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc000bef0 con 0x7fadc0003970 2023-11-24T08:08:54.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.718+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadc0003970 0x7fadc0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.718+0000 7fadda219640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadc0004f60 0x7fadc000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7f5d130d1640 1 Processor -- start 2023-11-24T08:08:54.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7f5d130d1640 1 -- start start 2023-11-24T08:08:54.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7fadd9a18640 1 -- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadc0004f60 msgr2=0x7fadc000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadc0004f60 0x7fadc000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7fadd9a18640 1 -- 172.21.15.106:0/2260982106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0004360 msgr2=0x7fadc0004840 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T08:08:54.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0004360 0x7fadc0004840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.719+0000 7fadd9a18640 1 -- 172.21.15.106:0/2260982106 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40d2260 con 0x7fadc0003970 2023-11-24T08:08:54.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.721+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadc0003970 0x7fadc0003d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fadd40d3200 tx=0x7fadc00057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7179d98829e40d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.722+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc402c5a0 con 0x7fadc0003970 2023-11-24T08:08:54.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.722+0000 7f5d130d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d040e32a0 0x7f5d04052c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.722+0000 7f5d130d1640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5d04053160 0x7f5d0404f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7f5d130d1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d040e3680 con 0x7f5d0404fd80 2023-11-24T08:08:54.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7f5d130d1640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d04050230 con 0x7f5d04053160 2023-11-24T08:08:54.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7f5d130d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d04053670 con 0x7f5d040e32a0 2023-11-24T08:08:54.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7f5d128d0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5d0404fd80 0x7f5d04005e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60944/0 (socket says 172.21.15.106:60944) 2023-11-24T08:08:54.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.722+0000 7fadd8a16640 1 -- 172.21.15.106:0/2260982106 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc402c700 con 0x7fadc0003970 2023-11-24T08:08:54.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7faddaa1a640 1 -- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadc0003970 msgr2=0x7fadc0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7f5d128d0640 1 -- 172.21.15.106:0/1001376657 learned_addr learned my addr 172.21.15.106:0/1001376657 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:54.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7faddaa1a640 1 --2- 172.21.15.106:0/2260982106 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadc0003970 0x7fadc0003d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fadd40d3200 tx=0x7fadc00057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.723+0000 7fadda219640 1 -- 172.21.15.106:0/2260982106 reap_dead start 2023-11-24T08:08:54.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7fef3647c640 1 Processor -- start 2023-11-24T08:08:54.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7f5d118ce640 1 --2- 172.21.15.106:0/1001376657 >> v2:172.21.15.156:3300/0 conn(0x7f5d04053160 0x7f5d0404f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7f5d120cf640 1 --2- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d040e32a0 0x7f5d04052c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7faddaa1a640 1 -- 172.21.15.106:0/2260982106 shutdown_connections 2023-11-24T08:08:54.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7f5d110cd640 1 -- 172.21.15.106:0/1001376657 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2041707381 0 0) 0x7f5d040e3680 con 0x7f5d0404fd80 2023-11-24T08:08:54.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7f5d110cd640 1 -- 172.21.15.106:0/1001376657 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf8003520 con 0x7f5d0404fd80 2023-11-24T08:08:54.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7faddaa1a640 1 -- 172.21.15.106:0/2260982106 >> 172.21.15.106:0/2260982106 conn(0x7fadd4061340 msgr2=0x7fadd4061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7f5d118ce640 1 -- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d040e32a0 msgr2=0x7f5d04052c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7f5d118ce640 1 --2- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d040e32a0 0x7f5d04052c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7f5d118ce640 1 -- 172.21.15.106:0/1001376657 >> v1:172.21.15.106:6789/0 conn(0x7f5d0404fd80 legacy=0x7f5d04005e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7f5d118ce640 1 -- 172.21.15.106:0/1001376657 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04053950 con 0x7f5d04053160 2023-11-24T08:08:54.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7faddaa1a640 1 -- 172.21.15.106:0/2260982106 shutdown_connections 2023-11-24T08:08:54.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7faddaa1a640 1 -- 172.21.15.106:0/2260982106 wait complete. 2023-11-24T08:08:54.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.725+0000 7f0035200640 1 Processor -- start 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.724+0000 7fef3647c640 1 -- start start 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7fef3647c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef3015d720 0x7fef300d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7fef3647c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef300b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7fef3647c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef3015db00 con 0x7fef300d3870 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7fef3647c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef300d3d20 con 0x7fef3015d720 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7fef3647c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef300d0ce0 con 0x7fef3006ca90 2023-11-24T08:08:54.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7f5d118ce640 1 --2- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d04053160 0x7f5d0404f840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0097a0 tx=0x7f5cfc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49b9dfd0ed43f7a3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.726+0000 7f0035200640 1 -- start start 2023-11-24T08:08:54.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7f5d110cd640 1 -- 172.21.15.106:0/1001376657 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc030610 con 0x7f5d04053160 2023-11-24T08:08:54.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7f5d110cd640 1 -- 172.21.15.106:0/1001376657 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cfc030770 con 0x7f5d04053160 2023-11-24T08:08:54.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7f5d110cd640 1 -- 172.21.15.106:0/1001376657 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cfc030610 con 0x7f5d04053160 2023-11-24T08:08:54.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7fef34a19640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef300b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7fef34a19640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef300b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48196/0 (socket says 172.21.15.106:48196) 2023-11-24T08:08:54.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7fef34a19640 1 -- 172.21.15.106:0/1657102991 learned_addr learned my addr 172.21.15.106:0/1657102991 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7faddaa1a640 1 Processor -- start 2023-11-24T08:08:54.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7fef2ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef3015d720 0x7fef300d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7faddaa1a640 1 -- start start 2023-11-24T08:08:54.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7fef34a19640 1 -- 172.21.15.106:0/1657102991 >> v2:172.21.15.106:3300/0 conn(0x7fef3015d720 msgr2=0x7fef300d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7fef34a19640 1 --2- 172.21.15.106:0/1657102991 >> v2:172.21.15.106:3300/0 conn(0x7fef3015d720 0x7fef300d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7fef34a19640 1 -- 172.21.15.106:0/1657102991 >> v1:172.21.15.156:6789/0 conn(0x7fef300d3870 legacy=0x7fef300b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.727+0000 7fef2f7fe640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fef300d3870 0x7fef300b0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59788/0 (socket says 172.21.15.106:59788) 2023-11-24T08:08:54.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f5d130d1640 1 -- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d04053160 msgr2=0x7f5d0404f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f5d130d1640 1 --2- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d04053160 0x7f5d0404f840 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc0097a0 tx=0x7f5cfc02b960 comp rx=0 tx=0).stop 2023-11-24T08:08:54.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f5d130d1640 1 -- 172.21.15.106:0/1001376657 shutdown_connections 2023-11-24T08:08:54.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f5d130d1640 1 --2- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d04053160 0x7f5d0404f840 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f5d130d1640 1 --2- 172.21.15.106:0/1001376657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d040e32a0 0x7f5d04052c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f5d130d1640 1 -- 172.21.15.106:0/1001376657 >> 172.21.15.106:0/1001376657 conn(0x7f5d04009f80 msgr2=0x7f5d0400a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7fef34a19640 1 -- 172.21.15.106:0/1657102991 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef3006d400 con 0x7fef3006ca90 2023-11-24T08:08:54.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f0035200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00280eb020 0x7f00280552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f0035200640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0028006160 0x7f00280065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f0035200640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0028057df0 con 0x7f00280eb400 2023-11-24T08:08:54.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f0035200640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0028057960 con 0x7f0028006160 2023-11-24T08:08:54.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f0035200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00280387a0 con 0x7f00280eb020 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.728+0000 7f002ed9d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00280eb020 0x7f00280552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f002ed9d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00280eb020 0x7f00280552c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48202/0 (socket says 172.21.15.106:48202) 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f002ed9d640 1 -- 172.21.15.106:0/359838511 learned_addr learned my addr 172.21.15.106:0/359838511 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.729+0000 7f002f59e640 1 --2- 172.21.15.106:0/359838511 >> v2:172.21.15.106:3300/0 conn(0x7f0028006160 0x7f00280065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7f002f59e640 1 -- 172.21.15.106:0/359838511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00280eb020 msgr2=0x7f00280552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7f002f59e640 1 --2- 172.21.15.106:0/359838511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00280eb020 0x7f00280552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7f002f59e640 1 -- 172.21.15.106:0/359838511 >> v1:172.21.15.106:6789/0 conn(0x7f00280eb400 legacy=0x7f00280eb880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7f002f59e640 1 -- 172.21.15.106:0/359838511 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00280ebf90 con 0x7f0028006160 2023-11-24T08:08:54.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7faddaa1a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0003970 0x7fadd4157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7f3fca0ea640 1 Processor -- start 2023-11-24T08:08:54.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7faddaa1a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd40cc320 0x7fadd40c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7faddaa1a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd41459d0 0x7fadd4143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7faddaa1a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4157b20 con 0x7fadc0003970 2023-11-24T08:08:54.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7f5d130d1640 1 -- 172.21.15.106:0/1001376657 shutdown_connections 2023-11-24T08:08:54.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7f5d130d1640 1 -- 172.21.15.106:0/1001376657 wait complete. 2023-11-24T08:08:54.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7faddaa1a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4145540 con 0x7fadd41459d0 2023-11-24T08:08:54.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.730+0000 7faddaa1a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4145240 con 0x7fadd40cc320 2023-11-24T08:08:54.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7f3fca0ea640 1 -- start start 2023-11-24T08:08:54.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fadda219640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd41459d0 0x7fadd4143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fd1e383d640 1 Processor -- start 2023-11-24T08:08:54.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fadda219640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd41459d0 0x7fadd4143110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39494/0 (socket says 172.21.15.106:39494) 2023-11-24T08:08:54.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fadda219640 1 -- 172.21.15.106:0/2464673086 learned_addr learned my addr 172.21.15.106:0/2464673086 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0003970 0x7fadd4157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fadd9217640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd40cc320 0x7fadd40c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7fef34a19640 1 --2- 172.21.15.106:0/1657102991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef300b15d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef28009e50 tx=0x7fef2802be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c1742d825ee8af6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fadd9a18640 1 -- 172.21.15.106:0/2464673086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd40cc320 msgr2=0x7fadd40c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fadd40cc320 0x7fadd40c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fadd9a18640 1 -- 172.21.15.106:0/2464673086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd41459d0 msgr2=0x7fadd4143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fadd41459d0 0x7fadd4143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fadd9a18640 1 -- 172.21.15.106:0/2464673086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd4143ac0 con 0x7fadc0003970 2023-11-24T08:08:54.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fef2effd640 1 -- 172.21.15.106:0/1657102991 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2802c9e0 con 0x7fef3006ca90 2023-11-24T08:08:54.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fef2effd640 1 -- 172.21.15.106:0/1657102991 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2802cb40 con 0x7fef3006ca90 2023-11-24T08:08:54.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fef2effd640 1 -- 172.21.15.106:0/1657102991 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef280343f0 con 0x7fef3006ca90 2023-11-24T08:08:54.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fadd9a18640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0003970 0x7fadd4157220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c080 tx=0x7fadc4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.732+0000 7fd1e383d640 1 -- start start 2023-11-24T08:08:54.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4003a20 con 0x7fadc0003970 2023-11-24T08:08:54.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc4003b80 con 0x7fadc0003970 2023-11-24T08:08:54.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc4034440 con 0x7fadc0003970 2023-11-24T08:08:54.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7f5d130d1640 1 Processor -- start 2023-11-24T08:08:54.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.731+0000 7f002f59e640 1 --2- 172.21.15.106:0/359838511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0028006160 0x7f00280065f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750ccaed618bf176 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7fef3647c640 1 -- 172.21.15.106:0/1657102991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 msgr2=0x7fef300b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7fef3647c640 1 --2- 172.21.15.106:0/1657102991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef300b15d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fef28009e50 tx=0x7fef2802be40 comp rx=0 tx=0).stop 2023-11-24T08:08:54.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7f5d130d1640 1 -- start start 2023-11-24T08:08:54.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.733+0000 7fd1e383d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd1dc15d720 0x7fd1dc0d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7f002dd9b640 1 -- 172.21.15.106:0/359838511 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024031780 con 0x7f0028006160 2023-11-24T08:08:54.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e383d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0d3870 0x7fd1dc0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd40ccb00 con 0x7fadc0003970 2023-11-24T08:08:54.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fef3647c640 1 -- 172.21.15.106:0/1657102991 shutdown_connections 2023-11-24T08:08:54.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fef3647c640 1 --2- 172.21.15.106:0/1657102991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef300b15d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fef3647c640 1 --2- 172.21.15.106:0/1657102991 >> v2:172.21.15.106:3300/0 conn(0x7fef3015d720 0x7fef300d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fef3647c640 1 -- 172.21.15.106:0/1657102991 >> 172.21.15.106:0/1657102991 conn(0x7fef3008af60 msgr2=0x7fef3009c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7f002dd9b640 1 -- 172.21.15.106:0/359838511 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00240318e0 con 0x7f0028006160 2023-11-24T08:08:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7f002dd9b640 1 -- 172.21.15.106:0/359838511 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024031c70 con 0x7f0028006160 2023-11-24T08:08:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e383d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1dc15db00 con 0x7fd1dc0b1400 2023-11-24T08:08:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e383d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1dc0d3d20 con 0x7fd1dc15d720 2023-11-24T08:08:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e383d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0d0ce0 con 0x7fd1dc0d3870 2023-11-24T08:08:54.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd40cc7a0 con 0x7fadc0003970 2023-11-24T08:08:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7f3fca0ea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40d6d10 0x7f3fc40bb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e15d9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd1dc15d720 0x7fd1dc0d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e15d9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd1dc15d720 0x7fd1dc0d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39498/0 (socket says 172.21.15.106:39498) 2023-11-24T08:08:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e15d9640 1 -- 172.21.15.106:0/2995580762 learned_addr learned my addr 172.21.15.106:0/2995580762 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fef3647c640 1 -- 172.21.15.106:0/1657102991 shutdown_connections 2023-11-24T08:08:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fef3647c640 1 -- 172.21.15.106:0/1657102991 wait complete. 2023-11-24T08:08:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.734+0000 7fd1e0dd8640 1 --2- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0d3870 0x7fd1dc0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f5d130d1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d0404fd80 0x7f5d040e6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f5d130d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d04053160 0x7f5d040e46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f5d130d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d040e32a0 0x7f5d04038b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f5d130d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040e5d30 con 0x7f5d040e32a0 2023-11-24T08:08:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f5d130d1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040e6690 con 0x7f5d0404fd80 2023-11-24T08:08:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f5d130d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d040e50b0 con 0x7f5d04053160 2023-11-24T08:08:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f0035200640 1 -- 172.21.15.106:0/359838511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0028006160 msgr2=0x7f00280065f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f0035200640 1 --2- 172.21.15.106:0/359838511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0028006160 0x7f00280065f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f00240097a0 tx=0x7f002402b820 comp rx=0 tx=0).stop 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f3fca0ea640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc4070d60 con 0x7f3fc406d4c0 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f3fca0ea640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fc41660f0 con 0x7f3fc40ba6d0 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f3fca0ea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc40715f0 con 0x7f3fc40d6d10 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f3fc88e7640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f3fc40ba6d0 0x7f3fc41652d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59806/0 (socket says 172.21.15.106:59806) 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f3fc88e7640 1 -- 172.21.15.106:0/2733632715 learned_addr learned my addr 172.21.15.106:0/2733632715 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7f3fc90e8640 1 --2- 172.21.15.106:0/2733632715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40d6d10 0x7f3fc40bb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f3fc3fff640 1 -- 172.21.15.106:0/2733632715 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816171327 0 0) 0x7f3fc41660f0 con 0x7f3fc40ba6d0 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f3fc3fff640 1 -- 172.21.15.106:0/2733632715 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fac003520 con 0x7f3fc40ba6d0 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2733632715 >> v1:172.21.15.156:6789/0 conn(0x7f3fc40ba6d0 legacy=0x7f3fc41652d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2733632715 >> v1:172.21.15.106:6789/0 conn(0x7f3fc406d4c0 legacy=0x7f3fc41659e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.736+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2733632715 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc40718d0 con 0x7f3fc40d6d10 2023-11-24T08:08:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7fd1e15d9640 1 -- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0d3870 msgr2=0x7fd1dc0b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7fd1e15d9640 1 --2- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0d3870 0x7fd1dc0b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7fd1e15d9640 1 -- 172.21.15.106:0/2995580762 >> v1:172.21.15.156:6789/0 conn(0x7fd1dc0b1400 legacy=0x7fd1dc0b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.735+0000 7fd1e15d9640 1 -- 172.21.15.106:0/2995580762 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1dc06d400 con 0x7fd1dc15d720 2023-11-24T08:08:54.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7f5d120cf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d0404fd80 0x7f5d040e6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7f5d118ce640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d04053160 0x7f5d040e46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7fd1e15d9640 1 --2- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0d3330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc08b890 tx=0x7fd1d802b880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30dceea02df98fab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7f5d128d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d040e32a0 0x7f5d04038b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadd40d0910 con 0x7fadc0003970 2023-11-24T08:08:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7fef3647c640 1 Processor -- start 2023-11-24T08:08:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7f5d120cf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d0404fd80 0x7f5d040e6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39500/0 (socket says 172.21.15.106:39500) 2023-11-24T08:08:54.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7f5d120cf640 1 -- 172.21.15.106:0/2066566300 learned_addr learned my addr 172.21.15.106:0/2066566300 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.737+0000 7fef3647c640 1 -- start start 2023-11-24T08:08:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f0035200640 1 -- 172.21.15.106:0/359838511 shutdown_connections 2023-11-24T08:08:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7fd1d3fff640 1 -- 172.21.15.106:0/2995580762 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d80317e0 con 0x7fd1dc15d720 2023-11-24T08:08:54.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f0035200640 1 --2- 172.21.15.106:0/359838511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0028006160 0x7f00280065f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f0035200640 1 --2- 172.21.15.106:0/359838511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00280eb020 0x7f00280552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f0035200640 1 -- 172.21.15.106:0/359838511 >> 172.21.15.106:0/359838511 conn(0x7f00280559a0 msgr2=0x7f0028055db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7fd1d3fff640 1 -- 172.21.15.106:0/2995580762 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d8031940 con 0x7fd1dc15d720 2023-11-24T08:08:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7fd1d3fff640 1 -- 172.21.15.106:0/2995580762 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d8031d10 con 0x7fd1dc15d720 2023-11-24T08:08:54.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f5d120cf640 1 -- 172.21.15.106:0/2066566300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d04053160 msgr2=0x7f5d040e46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f5d120cf640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5d04053160 0x7f5d040e46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f5d120cf640 1 -- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d040e32a0 msgr2=0x7f5d04038b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f5d120cf640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5d040e32a0 0x7f5d04038b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f0035200640 1 -- 172.21.15.106:0/359838511 shutdown_connections 2023-11-24T08:08:54.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f5d120cf640 1 -- 172.21.15.106:0/2066566300 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc009400 con 0x7f5d0404fd80 2023-11-24T08:08:54.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 -- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc15d720 msgr2=0x7fd1dc0d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 --2- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0d3330 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd1dc08b890 tx=0x7fd1d802b880 comp rx=0 tx=0).stop 2023-11-24T08:08:54.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef3647c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef30159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef3647c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef300d3870 0x7fef300cc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef3647c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef3015d720 0x7fef301f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 -- 172.21.15.106:0/2995580762 shutdown_connections 2023-11-24T08:08:54.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 --2- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0d3870 0x7fd1dc0b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 --2- 172.21.15.106:0/2995580762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0d3330 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 -- 172.21.15.106:0/2995580762 >> 172.21.15.106:0/2995580762 conn(0x7fd1dc08af60 msgr2=0x7fd1dc09c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fd1e383d640 1 -- 172.21.15.106:0/2995580762 shutdown_connections 2023-11-24T08:08:54.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef3647c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef301583b0 con 0x7fef300d3870 2023-11-24T08:08:54.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef3647c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef30158650 con 0x7fef3015d720 2023-11-24T08:08:54.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef3647c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef30158930 con 0x7fef3006ca90 2023-11-24T08:08:54.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7f3fc90e8640 1 --2- 172.21.15.106:0/2733632715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40d6d10 0x7f3fc40bb4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40097d0 tx=0x7f3fb402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f6e66321c356dd39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef2ffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef30159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef2f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef300d3870 0x7fef300cc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef2ffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef30159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48244/0 (socket says 172.21.15.106:48244) 2023-11-24T08:08:54.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef2f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef300d3870 0x7fef300cc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34200/0 (socket says 172.21.15.106:34200) 2023-11-24T08:08:54.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fadc403b070 con 0x7fadc0003970 2023-11-24T08:08:54.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.739+0000 7fef2f7fe640 1 -- 172.21.15.106:0/2342536181 learned_addr learned my addr 172.21.15.106:0/2342536181 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7f3fc3fff640 1 -- 172.21.15.106:0/2733632715 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb402d040 con 0x7f3fc40d6d10 2023-11-24T08:08:54.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7f3fc3fff640 1 -- 172.21.15.106:0/2733632715 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb4031810 con 0x7f3fc40d6d10 2023-11-24T08:08:54.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7f3fc3fff640 1 -- 172.21.15.106:0/2733632715 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4031ba0 con 0x7f3fc40d6d10 2023-11-24T08:08:54.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fd1e383d640 1 -- 172.21.15.106:0/2995580762 wait complete. 2023-11-24T08:08:54.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fef34a19640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef3015d720 0x7fef301f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fadca7fc640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fadb806ae40 0x7fadb806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.738+0000 7f0035200640 1 -- 172.21.15.106:0/359838511 wait complete. 2023-11-24T08:08:54.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7f0035200640 1 Processor -- start 2023-11-24T08:08:54.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.741+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadc40ab290 con 0x7fadc0003970 2023-11-24T08:08:54.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.741+0000 7fadca7fc640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fadb8072fe0 0x7fadb8075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.741+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 --> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadb8075ab0 con 0x7fadb8072fe0 2023-11-24T08:08:54.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fef2ffff640 1 -- 172.21.15.106:0/2342536181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef3015d720 msgr2=0x7fef301f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fef2ffff640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef3015d720 0x7fef301f2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fef2ffff640 1 -- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef300d3870 msgr2=0x7fef300cc1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fef2ffff640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef300d3870 0x7fef300cc1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.740+0000 7fef2ffff640 1 -- 172.21.15.106:0/2342536181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef28009b30 con 0x7fef3006ca90 2023-11-24T08:08:54.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.741+0000 7fef2ffff640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef30159410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef2400e850 tx=0x7fef2400ed10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.741+0000 7fadda219640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fadb8072fe0 0x7fadb8075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadc40763f0 con 0x7fadc0003970 2023-11-24T08:08:54.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2733632715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40d6d10 msgr2=0x7f3fc40bb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 --2- 172.21.15.106:0/2733632715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40d6d10 0x7f3fc40bb4d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40097d0 tx=0x7f3fb402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f5d120cf640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d0404fd80 0x7f5d040e6150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9b0 tx=0x7f5d0800ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2733632715 shutdown_connections 2023-11-24T08:08:54.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 --2- 172.21.15.106:0/2733632715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40d6d10 0x7f3fc40bb4d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2733632715 >> 172.21.15.106:0/2733632715 conn(0x7f3fc40b93d0 msgr2=0x7f3fc40b97b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400c940 con 0x7fef3006ca90 2023-11-24T08:08:54.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2733632715 shutdown_connections 2023-11-24T08:08:54.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2400caa0 con 0x7fef3006ca90 2023-11-24T08:08:54.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2400ce10 con 0x7fef3006ca90 2023-11-24T08:08:54.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef30158c10 con 0x7fef3006ca90 2023-11-24T08:08:54.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2733632715 wait complete. 2023-11-24T08:08:54.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d08009670 con 0x7f5d0404fd80 2023-11-24T08:08:54.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0800c9a0 con 0x7f5d0404fd80 2023-11-24T08:08:54.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0800cd50 con 0x7f5d0404fd80 2023-11-24T08:08:54.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.743+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d040e5390 con 0x7f5d0404fd80 2023-11-24T08:08:54.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.743+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d040e5950 con 0x7f5d0404fd80 2023-11-24T08:08:54.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.741+0000 7f0035200640 1 -- start start 2023-11-24T08:08:54.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.743+0000 7ffaa3d05640 1 Processor -- start 2023-11-24T08:08:54.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fadda219640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fadb8072fe0 0x7fadb8075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.743+0000 7fadd9217640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fadb806ae40 0x7fadb806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.744+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5d040541f0 con 0x7f5d0404fd80 2023-11-24T08:08:54.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.742+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef300cc720 con 0x7fef3006ca90 2023-11-24T08:08:54.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.744+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== osd.0 v2:172.21.15.106:6808/3403388309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fadb8075ab0 con 0x7fadb8072fe0 2023-11-24T08:08:54.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.744+0000 7f0035200640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0028006160 0x7f002800d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.744+0000 7f0035200640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00280eb020 0x7f002804f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.744+0000 7f0035200640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00280eb400 0x7f002800b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.744+0000 7ffaa3d05640 1 -- start start 2023-11-24T08:08:54.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.745+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fef300cef70 con 0x7fef3006ca90 2023-11-24T08:08:54.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.745+0000 7f3fca0ea640 1 Processor -- start 2023-11-24T08:08:54.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.745+0000 7f002e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00280eb020 0x7f002804f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.745+0000 7f002e59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00280eb020 0x7f002804f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39508/0 (socket says 172.21.15.106:39508) 2023-11-24T08:08:54.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.745+0000 7f3fca0ea640 1 -- start start 2023-11-24T08:08:54.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.745+0000 7fadd9217640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fadb806ae40 0x7fadb806d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadd4145170 tx=0x7fadcc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f002ed9d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0028006160 0x7f002800d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f002ed9d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0028006160 0x7f002800d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48260/0 (socket says 172.21.15.106:48260) 2023-11-24T08:08:54.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f002f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00280eb400 0x7f002800b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f002f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00280eb400 0x7f002800b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34210/0 (socket says 172.21.15.106:34210) 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f0035200640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002804e900 con 0x7f00280eb400 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f0035200640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002804dbb0 con 0x7f00280eb020 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7f0035200640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002804eb40 con 0x7f0028006160 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa3d05640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffa9c161260 0x7ffa9c1616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa3d05640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c0ceb10 0x7ffa9c0cefa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa3d05640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa9c161ce0 con 0x7ffa9c15d270 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa3d05640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa9c0cf4e0 con 0x7ffa9c161260 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa3d05640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c161fc0 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa1aa1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ffa9c15d270 0x7ffa9c15d650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:60966/0 (socket says 172.21.15.106:60966) 2023-11-24T08:08:54.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.746+0000 7ffaa1aa1640 1 -- 172.21.15.106:0/4005406646 learned_addr learned my addr 172.21.15.106:0/4005406646 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:54.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7ffaa12a0640 1 --2- 172.21.15.106:0/4005406646 >> v2:172.21.15.156:3300/0 conn(0x7ffa9c161260 0x7ffa9c1616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f002e59c640 1 -- 172.21.15.106:0/1092433688 learned_addr learned my addr 172.21.15.106:0/1092433688 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7ffaa22a2640 1 --2- 172.21.15.106:0/4005406646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c0ceb10 0x7ffa9c0cefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f3fca0ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc406d4c0 0x7f3fc4153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f3fca0ea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40ba6d0 0x7f3fc41563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f3fca0ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40d6d10 0x7f3fc415ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f3fca0ea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc4157ac0 con 0x7f3fc406d4c0 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f3fca0ea640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc4146a60 con 0x7f3fc40d6d10 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7f3fca0ea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc4156e40 con 0x7f3fc40ba6d0 2023-11-24T08:08:54.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7fd1e383d640 1 Processor -- start 2023-11-24T08:08:54.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f3fc90e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc406d4c0 0x7f3fc4153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f3fc90e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc406d4c0 0x7f3fc4153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34214/0 (socket says 172.21.15.106:34214) 2023-11-24T08:08:54.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2391529012 learned_addr learned my addr 172.21.15.106:0/2391529012 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7ffaa22a2640 1 -- 172.21.15.106:0/4005406646 >> v2:172.21.15.156:3300/0 conn(0x7ffa9c161260 msgr2=0x7ffa9c1616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7ffaa22a2640 1 --2- 172.21.15.106:0/4005406646 >> v2:172.21.15.156:3300/0 conn(0x7ffa9c161260 0x7ffa9c1616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.747+0000 7ffaa22a2640 1 -- 172.21.15.106:0/4005406646 >> v1:172.21.15.106:6789/0 conn(0x7ffa9c15d270 legacy=0x7ffa9c15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7ffaa22a2640 1 -- 172.21.15.106:0/4005406646 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c0ce310 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7fcabb59e640 1 Processor -- start 2023-11-24T08:08:54.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f002e59c640 1 -- 172.21.15.106:0/1092433688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0028006160 msgr2=0x7f002800d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f002e59c640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0028006160 0x7f002800d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f002e59c640 1 -- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00280eb400 msgr2=0x7f002800b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5d08013470 con 0x7f5d0404fd80 2023-11-24T08:08:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7fcabb59e640 1 -- start start 2023-11-24T08:08:54.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7f3fc98e9640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40d6d10 0x7f3fc415ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40ba6d0 msgr2=0x7f3fc41563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7f3fc90e8640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fc40ba6d0 0x7f3fc41563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40d6d10 msgr2=0x7f3fc415ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7f3fc90e8640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fc40d6d10 0x7f3fc415ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7f3fc90e8640 1 -- 172.21.15.106:0/2391529012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb4009400 con 0x7f3fc406d4c0 2023-11-24T08:08:54.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef24013470 con 0x7fef3006ca90 2023-11-24T08:08:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.748+0000 7fd1e383d640 1 -- start start 2023-11-24T08:08:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7fd1e383d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0b1400 0x7fd1dc0b8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7fd1e383d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc0d3870 0x7fd1dc0cea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7fd1e383d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0cef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7fd1e383d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b92e0 con 0x7fd1dc15d720 2023-11-24T08:08:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7fd1e383d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b7250 con 0x7fd1dc0d3870 2023-11-24T08:08:54.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.749+0000 7fd1e383d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1dc0b7530 con 0x7fd1dc0b1400 2023-11-24T08:08:54.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f3fc90e8640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc406d4c0 0x7f3fc4153bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4000c00 tx=0x7f3fb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f5d02ffd640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cf006af10 0x7f5cf006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f002e59c640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00280eb400 0x7f002800b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f002e59c640 1 -- 172.21.15.106:0/1092433688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0024009400 con 0x7f00280eb020 2023-11-24T08:08:54.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fd1e0dd8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc0d3870 0x7fd1dc0cea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f002f59e640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00280eb400 0x7f002800b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fef2cff9640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef1c06ae40 0x7fef1c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f002e59c640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00280eb020 0x7f002804f620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f002000c400 tx=0x7f002000c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb403ac60 con 0x7f3fc406d4c0 2023-11-24T08:08:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb403adc0 con 0x7f3fc406d4c0 2023-11-24T08:08:54.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb4044630 con 0x7f3fc406d4c0 2023-11-24T08:08:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fd1e1dda640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0cef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fd1e1dda640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0cef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34230/0 (socket says 172.21.15.106:34230) 2023-11-24T08:08:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fd1e1dda640 1 -- 172.21.15.106:0/3361750761 learned_addr learned my addr 172.21.15.106:0/3361750761 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fcabb59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcabc0b63d0 0x7fcabc0d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fcabb59e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc0cea30 con 0x7fcabc15d490 2023-11-24T08:08:54.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fcabb59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcabc0ce750 con 0x7fcabc0ce270 2023-11-24T08:08:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fcabb59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcabc0d0450 con 0x7fcabc0b63d0 2023-11-24T08:08:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7ffaa22a2640 1 --2- 172.21.15.106:0/4005406646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c0ceb10 0x7ffa9c0cefa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa940097a0 tx=0x7ffa9402b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=533119722c380ce2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef24089b80 con 0x7fef3006ca90 2023-11-24T08:08:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d08089b70 con 0x7f5d0404fd80 2023-11-24T08:08:54.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7f5d118ce640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cf006af10 0x7f5cf006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7fcab9d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcabc0b63d0 0x7fcabc0d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7fcab9d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcabc0b63d0 0x7fcabc0d3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34234/0 (socket says 172.21.15.106:34234) 2023-11-24T08:08:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7fcab9d9b640 1 -- 172.21.15.106:0/2237994710 learned_addr learned my addr 172.21.15.106:0/2237994710 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200052d0 con 0x7f00280eb020 2023-11-24T08:08:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020005430 con 0x7f00280eb020 2023-11-24T08:08:54.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0020003830 con 0x7f00280eb020 2023-11-24T08:08:54.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7fef2f7fe640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef1c06ae40 0x7fef1c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7ffaa0a9f640 1 -- 172.21.15.106:0/4005406646 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9402e070 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7ffaa0a9f640 1 -- 172.21.15.106:0/4005406646 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa940315a0 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7ffaa0a9f640 1 -- 172.21.15.106:0/4005406646 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa94031930 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7fef2cff9640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7fef1c073060 0x7fef1c075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fd1e1dda640 1 -- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0b1400 msgr2=0x7fd1dc0b8da0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T08:08:54.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fd1e1dda640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1dc0b1400 0x7fd1dc0b8da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fd1e1dda640 1 -- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc0d3870 msgr2=0x7fd1dc0cea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fcab9d9b640 1 -- 172.21.15.106:0/2237994710 >> v1:172.21.15.156:6789/0 conn(0x7fcabc0ce270 legacy=0x7fcabc0d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:54.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7f5d02ffd640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7f5cf0073130 0x7f5cf0075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 --> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5cf0075c00 con 0x7f5cf0073130 2023-11-24T08:08:54.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002804ee20 con 0x7f00280eb020 2023-11-24T08:08:54.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028006c00 con 0x7f00280eb020 2023-11-24T08:08:54.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fd1e1dda640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc0d3870 0x7fd1dc0cea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fd1e1dda640 1 -- 172.21.15.106:0/3361750761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1d80094d0 con 0x7fd1dc15d720 2023-11-24T08:08:54.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 --> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fef1c075b30 con 0x7fef1c073060 2023-11-24T08:08:54.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.750+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc4157120 con 0x7f3fc406d4c0 2023-11-24T08:08:54.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.751+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc41576e0 con 0x7f3fc406d4c0 2023-11-24T08:08:54.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5d08089f20 con 0x7f5d0404fd80 2023-11-24T08:08:54.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7fd1e0dd8640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1dc0d3870 0x7fd1dc0cea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:54.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef24089f30 con 0x7fef3006ca90 2023-11-24T08:08:54.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 -- 172.21.15.106:0/4005406646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c0ceb10 msgr2=0x7ffa9c0cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 --2- 172.21.15.106:0/4005406646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c0ceb10 0x7ffa9c0cefa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa940097a0 tx=0x7ffa9402b840 comp rx=0 tx=0).stop 2023-11-24T08:08:54.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7fef34a19640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7fef1c073060 0x7fef1c075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7fd1e1dda640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0cef50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c780 tx=0x7fd1d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 -- 172.21.15.106:0/4005406646 shutdown_connections 2023-11-24T08:08:54.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 --2- 172.21.15.106:0/4005406646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c0ceb10 0x7ffa9c0cefa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 --2- 172.21.15.106:0/4005406646 >> v2:172.21.15.156:3300/0 conn(0x7ffa9c161260 0x7ffa9c1616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 -- 172.21.15.106:0/4005406646 >> 172.21.15.106:0/4005406646 conn(0x7ffa9c08af60 msgr2=0x7ffa9c09c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7f5d118ce640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cf006af10 0x7f5cf006d3d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036000 tx=0x7f5cfc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7f5d128d0640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7f5cf0073130 0x7f5cf0075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d4004b60 con 0x7fd1dc15d720 2023-11-24T08:08:54.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 -- 172.21.15.106:0/4005406646 shutdown_connections 2023-11-24T08:08:54.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7ffaa3d05640 1 -- 172.21.15.106:0/4005406646 wait complete. 2023-11-24T08:08:54.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.753+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1dc0b7810 con 0x7fd1dc15d720 2023-11-24T08:08:54.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.754+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1dc0b7dd0 con 0x7fd1dc15d720 2023-11-24T08:08:54.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fcab9d9b640 1 -- 172.21.15.106:0/2237994710 >> v1:172.21.15.106:6789/0 conn(0x7fcabc15d490 legacy=0x7fcabc0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.752+0000 7fcab9d9b640 1 -- 172.21.15.106:0/2237994710 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcabc0d0630 con 0x7fcabc0b63d0 2023-11-24T08:08:54.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.754+0000 7fef34a19640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7fef1c073060 0x7fef1c075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.754+0000 7fef2f7fe640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef1c06ae40 0x7fef1c06d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef20006fd0 tx=0x7fef20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.754+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1d400ce90 con 0x7fd1dc15d720 2023-11-24T08:08:54.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.754+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1d400a850 con 0x7fd1dc15d720 2023-11-24T08:08:54.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.755+0000 7f5d128d0640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7f5cf0073130 0x7f5cf0075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.755+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0028057310 con 0x7f00280eb020 2023-11-24T08:08:54.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.754+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3fa4000fc0 con 0x7f3fc406d4c0 2023-11-24T08:08:54.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.756+0000 7ffaa3d05640 1 Processor -- start 2023-11-24T08:08:54.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.756+0000 7ffaa3d05640 1 -- start start 2023-11-24T08:08:54.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.756+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== osd.2 v2:172.21.15.106:6810/1671152390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fef1c075b30 con 0x7fef1c073060 2023-11-24T08:08:54.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.755+0000 7fcab9d9b640 1 --2- 172.21.15.106:0/2237994710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc0b63d0 0x7fcabc0d3230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009870 tx=0x7fcab002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3770f45853b0a740 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.756+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== osd.1 v2:172.21.15.106:6800/1347023088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f5cf0075c00 con 0x7f5cf0073130 2023-11-24T08:08:54.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.756+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd1bc000fc0 con 0x7fd1dc15d720 2023-11-24T08:08:54.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.756+0000 7fcab959a640 1 -- 172.21.15.106:0/2237994710 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0030860 con 0x7fcabc0b63d0 2023-11-24T08:08:54.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.757+0000 7fcab959a640 1 -- 172.21.15.106:0/2237994710 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab00309c0 con 0x7fcabc0b63d0 2023-11-24T08:08:54.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.757+0000 7fcab959a640 1 -- 172.21.15.106:0/2237994710 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0030d50 con 0x7fcabc0b63d0 2023-11-24T08:08:54.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.757+0000 7ffaa3d05640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa9c0ceb10 0x7ffa9c0b6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.757+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0020003990 con 0x7f00280eb020 2023-11-24T08:08:54.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa3d05640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c15d270 0x7ffa9c0c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa3d05640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa9c161260 0x7ffa9c0b7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7fcabb59e640 1 -- 172.21.15.106:0/2237994710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc0b63d0 msgr2=0x7fcabc0d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7fcabb59e640 1 --2- 172.21.15.106:0/2237994710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc0b63d0 0x7fcabc0d3230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009870 tx=0x7fcab002b9a0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.757+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fb404d450 con 0x7f3fc406d4c0 2023-11-24T08:08:54.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa3d05640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0c7dd0 con 0x7ffa9c161260 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa3d05640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0c6f20 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa1aa1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa9c0ceb10 0x7ffa9c0b6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa3d05640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa9c0c71c0 con 0x7ffa9c15d270 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa1aa1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa9c0ceb10 0x7ffa9c0b6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39546/0 (socket says 172.21.15.106:39546) 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.758+0000 7ffaa1aa1640 1 -- 172.21.15.106:0/13188633 learned_addr learned my addr 172.21.15.106:0/13188633 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7ffaa1aa1640 1 -- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c15d270 msgr2=0x7ffa9c0c8260 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7ffaa1aa1640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffa9c15d270 0x7ffa9c0c8260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7ffaa1aa1640 1 -- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa9c161260 msgr2=0x7ffa9c0b7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7ffaa1aa1640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffa9c161260 0x7ffa9c0b7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7ffaa1aa1640 1 -- 172.21.15.106:0/13188633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa94009400 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7fcabb59e640 1 -- 172.21.15.106:0/2237994710 shutdown_connections 2023-11-24T08:08:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7fcabb59e640 1 --2- 172.21.15.106:0/2237994710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc0b63d0 0x7fcabc0d3230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7fcabb59e640 1 -- 172.21.15.106:0/2237994710 >> 172.21.15.106:0/2237994710 conn(0x7fcabc061340 msgr2=0x7fcabc061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7f3fc1ffb640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fb006af10 0x7f3fb006d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7f3fc88e7640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fb006af10 0x7f3fb006d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.759+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3fb40ab720 con 0x7f3fc406d4c0 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f3fc88e7640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fb006af10 0x7f3fb006d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc000f60 tx=0x7f3fbc0078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f3fc1ffb640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7f3fb0073130 0x7f3fb0075550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 --> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3fb0075c00 con 0x7f3fb0073130 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3fb40add40 con 0x7f3fc406d4c0 2023-11-24T08:08:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1d400a9b0 con 0x7fd1dc15d720 2023-11-24T08:08:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7fcabb59e640 1 -- 172.21.15.106:0/2237994710 shutdown_connections 2023-11-24T08:08:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f00177fe640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f001c06af40 0x7f001c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0020088b30 con 0x7f00280eb020 2023-11-24T08:08:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f00177fe640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7f001c073160 0x7f001c075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 --> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f001c075c30 con 0x7f001c073160 2023-11-24T08:08:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f002008d2a0 con 0x7f00280eb020 2023-11-24T08:08:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7fcabb59e640 1 -- 172.21.15.106:0/2237994710 wait complete. 2023-11-24T08:08:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.760+0000 7ffaa1aa1640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa9c0ceb10 0x7ffa9c0b6a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa98007fd0 tx=0x7ffa9800c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7f002f59e640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7f001c073160 0x7f001c075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98005ce0 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7f3fc98e9640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7f3fb0073130 0x7f3fb0075550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98005e60 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98007310 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7f002ed9d640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f001c06af40 0x7f001c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.762+0000 7f002f59e640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7f001c073160 0x7f001c075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa9c0c74a0 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.761+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa9c0c7a60 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.762+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ffa84000fc0 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.763+0000 7fd1d1ffb640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1c006ae40 0x7fd1c006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.763+0000 7fcabb59e640 1 Processor -- start 2023-11-24T08:08:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.763+0000 7fcabb59e640 1 -- start start 2023-11-24T08:08:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.763+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== osd.3 v2:172.21.15.106:6817/906310218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f001c075c30 con 0x7f001c073160 2023-11-24T08:08:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7f3fc98e9640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7f3fb0073130 0x7f3fb0075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7f002ed9d640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f001c06af40 0x7f001c06d400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0030068cb0 tx=0x7f003008c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.763+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1d40899d0 con 0x7fd1dc15d720 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7fd1d1ffb640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7fd1c0073060 0x7fd1c0075480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 --> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd1c0075b30 con 0x7fd1c0073060 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd1d408e280 con 0x7fd1dc15d720 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7fd1e0dd8640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7fd1c0073060 0x7fd1c0075480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.765+0000 7fd1e15d9640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1c006ae40 0x7fd1c006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.764+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ffa98007470 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.765+0000 7fcabb59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcabc0ce270 0x7fcabc0cc100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7fcabb59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc15d490 0x7fcabc157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.765+0000 7fd1e0dd8640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7fd1c0073060 0x7fd1c0075480 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7ffa8e7fc640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa8006ae40 0x7ffa8006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa98088d00 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7fcabb59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcabc0c89a0 0x7fcabc15a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7fd1e15d9640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1c006ae40 0x7fd1c006d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8036000 tx=0x7fd1d8008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7ffaa12a0640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa8006ae40 0x7ffa8006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7ffa8e7fc640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ffa80072fe0 0x7ffa80075400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 --> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffa80075ab0 con 0x7ffa80072fe0 2023-11-24T08:08:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.766+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcabc0ce270 0x7fcabc0cc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa98053d10 con 0x7ffa9c0ceb10 2023-11-24T08:08:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcaba59c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcabc0ce270 0x7fcabc0cc100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39550/0 (socket says 172.21.15.106:39550) 2023-11-24T08:08:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== osd.4 v2:172.21.15.156:6800/2602399217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3fb0075c00 con 0x7f3fb0073130 2023-11-24T08:08:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcabb59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0d0bf0 con 0x7fcabc15d490 2023-11-24T08:08:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcabb59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0cb060 con 0x7fcabc0ce270 2023-11-24T08:08:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcaba59c640 1 -- 172.21.15.106:0/1307111895 learned_addr learned my addr 172.21.15.106:0/1307111895 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcabc0cb340 con 0x7fcabc0c89a0 2023-11-24T08:08:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7ffaa22a2640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ffa80072fe0 0x7ffa80075400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7ffaa12a0640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa8006ae40 0x7ffa8006d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0b7850 tx=0x7ffa9000b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcabad9d640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcabc0c89a0 0x7fcabc15a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.767+0000 7fcab9d9b640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc15d490 0x7fcabc157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.768+0000 7fcaba59c640 1 -- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcabc0c89a0 msgr2=0x7fcabc15a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.768+0000 7fcaba59c640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcabc0c89a0 0x7fcabc15a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.768+0000 7fcaba59c640 1 -- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc15d490 msgr2=0x7fcabc157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.768+0000 7fcaba59c640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc15d490 0x7fcabc157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.769+0000 7fcaba59c640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab00094d0 con 0x7fcabc0ce270 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.769+0000 7fcab9d9b640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcabc15d490 0x7fcabc157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.769+0000 7fcaba59c640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcabc0ce270 0x7fcabc0cc100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c760 tx=0x7fcab400cc20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.768+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== osd.5 v2:172.21.15.156:6816/3158094073 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fd1c0075b30 con 0x7fd1c0073060 2023-11-24T08:08:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.770+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab4003b70 con 0x7fcabc0ce270 2023-11-24T08:08:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.770+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcabc0cb620 con 0x7fcabc0ce270 2023-11-24T08:08:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.770+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcabc0cbbe0 con 0x7fcabc0ce270 2023-11-24T08:08:54.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.771+0000 7ffaa22a2640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ffa80072fe0 0x7ffa80075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.771+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab4003cf0 con 0x7fcabc0ce270 2023-11-24T08:08:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.771+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab400aa90 con 0x7fcabc0ce270 2023-11-24T08:08:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.771+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcabc04e840 con 0x7fcabc0ce270 2023-11-24T08:08:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.771+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 --> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fadd40d0630 con 0x7fadb8072fe0 2023-11-24T08:08:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.775+0000 7fadca7fc640 1 -- 172.21.15.106:0/2464673086 <== osd.0 v2:172.21.15.106:6808/3403388309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fadb8075ab0 con 0x7fadb8072fe0 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.774+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== osd.6 v2:172.21.15.156:6808/2357036477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ffa80075ab0 con 0x7ffa80072fe0 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.773+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcab400abf0 con 0x7fcabc0ce270 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.775+0000 7fcaaaffd640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fca9c06af40 0x7fca9c06d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.775+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcab4089d70 con 0x7fcabc0ce270 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.775+0000 7fcaaaffd640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fca9c073160 0x7fca9c075580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.775+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca9c075c30 con 0x7fca9c073160 2023-11-24T08:08:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.775+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcab408a120 con 0x7fcabc0ce270 2023-11-24T08:08:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.777+0000 7fcab9d9b640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fca9c06af40 0x7fca9c06d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.777+0000 7fcabad9d640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fca9c073160 0x7fca9c075580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.777+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fadb8072fe0 msgr2=0x7fadb8075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.777+0000 7faddaa1a640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fadb8072fe0 0x7fadb8075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fadb806ae40 msgr2=0x7fadb806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fadb806ae40 0x7fadb806d300 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fadd4145170 tx=0x7fadcc009040 comp rx=0 tx=0).stop 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0003970 msgr2=0x7fadd4157220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 --2- 172.21.15.106:0/2464673086 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fadc0003970 0x7fadd4157220 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fadc000c080 tx=0x7fadc4004730 comp rx=0 tx=0).stop 2023-11-24T08:08:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7fcabad9d640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fca9c073160 0x7fca9c075580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7fcab9d9b640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fca9c06af40 0x7fca9c06d400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcab0036000 tx=0x7fcab0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7fadda219640 1 -- 172.21.15.106:0/2464673086 reap_dead start 2023-11-24T08:08:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 shutdown_connections 2023-11-24T08:08:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 >> 172.21.15.106:0/2464673086 conn(0x7fadd4061340 msgr2=0x7fadd415c880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 shutdown_connections 2023-11-24T08:08:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.778+0000 7faddaa1a640 1 -- 172.21.15.106:0/2464673086 wait complete. 2023-11-24T08:08:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.781+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== osd.7 v2:172.21.15.156:6824/1912715038 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fca9c075c30 con 0x7fca9c073160 2023-11-24T08:08:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.787+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 --> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5d04053f10 con 0x7f5cf0073130 2023-11-24T08:08:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.792+0000 7f5d02ffd640 1 -- 172.21.15.106:0/2066566300 <== osd.1 v2:172.21.15.106:6800/1347023088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5cf0075c00 con 0x7f5cf0073130 2023-11-24T08:08:54.831 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:54.831 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T08:08:54.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7f5cf0073130 msgr2=0x7f5cf0075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7f5cf0073130 0x7f5cf0075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cf006af10 msgr2=0x7f5cf006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cf006af10 0x7f5cf006d3d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc036000 tx=0x7f5cfc008030 comp rx=0 tx=0).stop 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d0404fd80 msgr2=0x7f5d040e6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 --2- 172.21.15.106:0/2066566300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5d0404fd80 0x7f5d040e6150 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800e9b0 tx=0x7f5d0800ee70 comp rx=0 tx=0).stop 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d128d0640 1 -- 172.21.15.106:0/2066566300 reap_dead start 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 shutdown_connections 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 >> 172.21.15.106:0/2066566300 conn(0x7f5d04009f80 msgr2=0x7f5d04005780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 shutdown_connections 2023-11-24T08:08:54.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.795+0000 7f5d130d1640 1 -- 172.21.15.106:0/2066566300 wait complete. 2023-11-24T08:08:54.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.796+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 --> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0028057030 con 0x7f001c073160 2023-11-24T08:08:54.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.797+0000 7f00177fe640 1 -- 172.21.15.106:0/1092433688 <== osd.3 v2:172.21.15.106:6817/906310218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0024031af0 con 0x7f001c073160 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.797+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7f001c073160 msgr2=0x7f001c075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.797+0000 7f0035200640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7f001c073160 0x7f001c075580 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f001c06af40 msgr2=0x7f001c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f0035200640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f001c06af40 0x7f001c06d400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0030068cb0 tx=0x7f003008c040 comp rx=0 tx=0).stop 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00280eb020 msgr2=0x7f002804f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f0035200640 1 --2- 172.21.15.106:0/1092433688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00280eb020 0x7f002804f620 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f002000c400 tx=0x7f002000c8c0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f002f59e640 1 -- 172.21.15.106:0/1092433688 reap_dead start 2023-11-24T08:08:54.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 shutdown_connections 2023-11-24T08:08:54.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.798+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 >> 172.21.15.106:0/1092433688 conn(0x7f00280559a0 msgr2=0x7f00280388e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.801+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 --> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3fa4002cb0 con 0x7f3fb0073130 2023-11-24T08:08:54.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.801+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 shutdown_connections 2023-11-24T08:08:54.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.801+0000 7f0035200640 1 -- 172.21.15.106:0/1092433688 wait complete. 2023-11-24T08:08:54.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fc1ffb640 1 -- 172.21.15.106:0/2391529012 <== osd.4 v2:172.21.15.156:6800/2602399217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3fa4002cb0 con 0x7f3fb0073130 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7f3fb0073130 msgr2=0x7f3fb0075550 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fca0ea640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7f3fb0073130 0x7f3fb0075550 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fb006af10 msgr2=0x7f3fb006d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fca0ea640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fb006af10 0x7f3fb006d3d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc000f60 tx=0x7f3fbc0078f0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc406d4c0 msgr2=0x7f3fc4153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fca0ea640 1 --2- 172.21.15.106:0/2391529012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fc406d4c0 0x7f3fc4153bf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3fb4000c00 tx=0x7f3fb40046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.802+0000 7f3fc98e9640 1 -- 172.21.15.106:0/2391529012 reap_dead start 2023-11-24T08:08:54.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.803+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 shutdown_connections 2023-11-24T08:08:54.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.803+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 >> 172.21.15.106:0/2391529012 conn(0x7f3fc40b93d0 msgr2=0x7f3fc40bb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.803+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 shutdown_connections 2023-11-24T08:08:54.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.803+0000 7f3fca0ea640 1 -- 172.21.15.106:0/2391529012 wait complete. 2023-11-24T08:08:54.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.805+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 --> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd1bc002cb0 con 0x7fd1c0073060 2023-11-24T08:08:54.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.806+0000 7fd1d1ffb640 1 -- 172.21.15.106:0/3361750761 <== osd.5 v2:172.21.15.156:6816/3158094073 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd1bc002cb0 con 0x7fd1c0073060 2023-11-24T08:08:54.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.806+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 --> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fef300b06c0 con 0x7fef1c073060 2023-11-24T08:08:54.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.807+0000 7fef2cff9640 1 -- 172.21.15.106:0/2342536181 <== osd.2 v2:172.21.15.106:6810/1671152390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fef2802c7e0 con 0x7fef1c073060 2023-11-24T08:08:54.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.807+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 --> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ffa84002cb0 con 0x7ffa80072fe0 2023-11-24T08:08:54.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7fd1c0073060 msgr2=0x7fd1c0075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7fd1c0073060 0x7fd1c0075480 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7fef1c073060 msgr2=0x7fef1c075480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fef3647c640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7fef1c073060 0x7fef1c075480 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1c006ae40 msgr2=0x7fd1c006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1c006ae40 0x7fd1c006d300 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd1d8036000 tx=0x7fd1d8008030 comp rx=0 tx=0).stop 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc15d720 msgr2=0x7fd1dc0cef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 --2- 172.21.15.106:0/3361750761 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1dc15d720 0x7fd1dc0cef50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd1d400c780 tx=0x7fd1d400cc40 comp rx=0 tx=0).stop 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e1dda640 1 -- 172.21.15.106:0/3361750761 reap_dead start 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 shutdown_connections 2023-11-24T08:08:54.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 >> 172.21.15.106:0/3361750761 conn(0x7fd1dc08af60 msgr2=0x7fd1dc06ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7ffa8e7fc640 1 -- 172.21.15.106:0/13188633 <== osd.6 v2:172.21.15.156:6808/2357036477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7ffa84002cb0 con 0x7ffa80072fe0 2023-11-24T08:08:54.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef1c06ae40 msgr2=0x7fef1c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fef3647c640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef1c06ae40 0x7fef1c06d300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fef20006fd0 tx=0x7fef20008040 comp rx=0 tx=0).stop 2023-11-24T08:08:54.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 msgr2=0x7fef30159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.808+0000 7fef3647c640 1 --2- 172.21.15.106:0/2342536181 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef3006ca90 0x7fef30159410 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fef2400e850 tx=0x7fef2400ed10 comp rx=0 tx=0).stop 2023-11-24T08:08:54.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7fef34a19640 1 -- 172.21.15.106:0/2342536181 reap_dead start 2023-11-24T08:08:54.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 shutdown_connections 2023-11-24T08:08:54.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7fd1e383d640 1 -- 172.21.15.106:0/3361750761 wait complete. 2023-11-24T08:08:54.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 shutdown_connections 2023-11-24T08:08:54.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 >> 172.21.15.106:0/2342536181 conn(0x7fef3008af60 msgr2=0x7fef300cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ffa80072fe0 msgr2=0x7ffa80075400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7ffaa3d05640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ffa80072fe0 0x7ffa80075400 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa8006ae40 msgr2=0x7ffa8006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7ffaa3d05640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa8006ae40 0x7ffa8006d300 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0b7850 tx=0x7ffa9000b040 comp rx=0 tx=0).stop 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa9c0ceb10 msgr2=0x7ffa9c0b6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.809+0000 7ffaa3d05640 1 --2- 172.21.15.106:0/13188633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa9c0ceb10 0x7ffa9c0b6a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa98007fd0 tx=0x7ffa9800c8e0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.810+0000 7ffaa22a2640 1 -- 172.21.15.106:0/13188633 reap_dead start 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.810+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 shutdown_connections 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.810+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 >> 172.21.15.106:0/13188633 conn(0x7ffa9c08af60 msgr2=0x7ffa9c0d20a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.811+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 shutdown_connections 2023-11-24T08:08:54.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.811+0000 7ffaa3d05640 1 -- 172.21.15.106:0/13188633 wait complete. 2023-11-24T08:08:54.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.811+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 shutdown_connections 2023-11-24T08:08:54.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.811+0000 7fef3647c640 1 -- 172.21.15.106:0/2342536181 wait complete. 2023-11-24T08:08:54.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.812+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 --> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcabc158d40 con 0x7fca9c073160 2023-11-24T08:08:54.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.813+0000 7fcaaaffd640 1 -- 172.21.15.106:0/1307111895 <== osd.7 v2:172.21.15.156:6824/1912715038 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcabc158d40 con 0x7fca9c073160 2023-11-24T08:08:54.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.814+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fca9c073160 msgr2=0x7fca9c075580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.814+0000 7fcabb59e640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fca9c073160 0x7fca9c075580 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fca9c06af40 msgr2=0x7fca9c06d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fca9c06af40 0x7fca9c06d400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fcab0036000 tx=0x7fcab0008030 comp rx=0 tx=0).stop 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcabc0ce270 msgr2=0x7fcabc0cc100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 --2- 172.21.15.106:0/1307111895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcabc0ce270 0x7fcabc0cc100 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcab400c760 tx=0x7fcab400cc20 comp rx=0 tx=0).stop 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabad9d640 1 -- 172.21.15.106:0/1307111895 reap_dead start 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 shutdown_connections 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 >> 172.21.15.106:0/1307111895 conn(0x7fcabc061340 msgr2=0x7fcabc0b36e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 shutdown_connections 2023-11-24T08:08:54.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:54.815+0000 7fcabb59e640 1 -- 172.21.15.106:0/1307111895 wait complete. 2023-11-24T08:08:54.850 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:54.850 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T08:08:54.851 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:54.851 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T08:08:54.852 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:54.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T08:08:54.853 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:54.854 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T08:08:54.854 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:54.855 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T08:08:54.855 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:54.856 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T08:08:54.856 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:54.857 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T08:08:55.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.190+0000 7fb36ab53640 1 Processor -- start 2023-11-24T08:08:55.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.190+0000 7fb36ab53640 1 -- start start 2023-11-24T08:08:55.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.192+0000 7fb36ab53640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c04c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.192+0000 7fb36ab53640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb35c058980 0x7fb35c0534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.192+0000 7fb36ab53640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb35c057d50 con 0x7fb35c056a70 2023-11-24T08:08:55.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.192+0000 7fb36ab53640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35c006220 con 0x7fb35c058980 2023-11-24T08:08:55.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.192+0000 7fb36ab53640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c006500 con 0x7fb35c0e76d0 2023-11-24T08:08:55.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.193+0000 7fb3688ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c04c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.194+0000 7fb3688ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c04c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48310/0 (socket says 172.21.15.106:48310) 2023-11-24T08:08:55.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.194+0000 7fb3688ef640 1 -- 172.21.15.106:0/1722152951 learned_addr learned my addr 172.21.15.106:0/1722152951 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.194+0000 7fb363fff640 1 --2- 172.21.15.106:0/1722152951 >> v2:172.21.15.156:3300/0 conn(0x7fb35c058980 0x7fb35c0534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7fb3688ef640 1 -- 172.21.15.106:0/1722152951 >> v2:172.21.15.156:3300/0 conn(0x7fb35c058980 msgr2=0x7fb35c0534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7fb3688ef640 1 --2- 172.21.15.106:0/1722152951 >> v2:172.21.15.156:3300/0 conn(0x7fb35c058980 0x7fb35c0534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7fb3688ef640 1 -- 172.21.15.106:0/1722152951 >> v1:172.21.15.106:6789/0 conn(0x7fb35c056a70 legacy=0x7fb35c006fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7feb29097640 1 Processor -- start 2023-11-24T08:08:55.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7fb3637fe640 1 -- 172.21.15.106:0/1722152951 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819855931 0 0) 0x7fb35c057d50 con 0x7fb35c056a70 2023-11-24T08:08:55.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7fb3688ef640 1 -- 172.21.15.106:0/1722152951 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35c0067e0 con 0x7fb35c0e76d0 2023-11-24T08:08:55.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.195+0000 7feb29097640 1 -- start start 2023-11-24T08:08:55.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.197+0000 7fb3688ef640 1 --2- 172.21.15.106:0/1722152951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c04c9f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb350009870 tx=0x7fb35002b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22749b4ba12ef89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.197+0000 7fb3637fe640 1 -- 172.21.15.106:0/1722152951 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350031870 con 0x7fb35c0e76d0 2023-11-24T08:08:55.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.198+0000 7feb29097640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feb241609e0 0x7feb2406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.198+0000 7feb29097640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb240ceb70 con 0x7feb240d31a0 2023-11-24T08:08:55.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.198+0000 7feb29097640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb2406c1f0 con 0x7feb240ce010 2023-11-24T08:08:55.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.198+0000 7feb29097640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb240d1070 con 0x7feb241609e0 2023-11-24T08:08:55.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.199+0000 7fb3637fe640 1 -- 172.21.15.106:0/1722152951 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3500319d0 con 0x7fb35c0e76d0 2023-11-24T08:08:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.199+0000 7fb3637fe640 1 -- 172.21.15.106:0/1722152951 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb350031d60 con 0x7fb35c0e76d0 2023-11-24T08:08:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.199+0000 7feb2359e640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7feb240ce010 0x7feb240ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59814/0 (socket says 172.21.15.106:59814) 2023-11-24T08:08:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.199+0000 7feb2359e640 1 -- 172.21.15.106:0/43252438 learned_addr learned my addr 172.21.15.106:0/43252438 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:55.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7fb36ab53640 1 -- 172.21.15.106:0/1722152951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 msgr2=0x7fb35c04c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1722152951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c04c9f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb350009870 tx=0x7fb35002b910 comp rx=0 tx=0).stop 2023-11-24T08:08:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7f04ce0df640 1 Processor -- start 2023-11-24T08:08:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265291386 0 0) 0x7feb2406c1f0 con 0x7feb240ce010 2023-11-24T08:08:55.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb0c003520 con 0x7feb240ce010 2023-11-24T08:08:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7fb36ab53640 1 -- 172.21.15.106:0/1722152951 shutdown_connections 2023-11-24T08:08:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1722152951 >> v2:172.21.15.156:3300/0 conn(0x7fb35c058980 0x7fb35c0534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1722152951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c04c9f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.200+0000 7fb36ab53640 1 -- 172.21.15.106:0/1722152951 >> 172.21.15.106:0/1722152951 conn(0x7fb35c058ea0 msgr2=0x7fb35c059280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7fb36ab53640 1 -- 172.21.15.106:0/1722152951 shutdown_connections 2023-11-24T08:08:55.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb22d9d640 1 --2- 172.21.15.106:0/43252438 >> v2:172.21.15.106:3300/0 conn(0x7feb241609e0 0x7feb2406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2878716528 0 0) 0x7feb0c003520 con 0x7feb240ce010 2023-11-24T08:08:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 >> v2:172.21.15.106:3300/0 conn(0x7feb241609e0 msgr2=0x7feb2406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 --2- 172.21.15.106:0/43252438 >> v2:172.21.15.106:3300/0 conn(0x7feb241609e0 0x7feb2406bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 >> v1:172.21.15.106:6789/0 conn(0x7feb240d31a0 legacy=0x7feb240d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240cedc0 con 0x7feb240ce010 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7feb1c001670 con 0x7feb240ce010 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 >> v1:172.21.15.156:6789/0 conn(0x7feb240ce010 legacy=0x7feb240ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c003970 0x7feb0c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0047c0 0x7feb0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb240d1070 con 0x7feb0c005180 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c00c0d0 con 0x7feb0c0047c0 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c00c290 con 0x7feb0c003970 2023-11-24T08:08:55.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.202+0000 7feb2359e640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.202+0000 7feb2259c640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0047c0 0x7feb0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.202+0000 7feb22d9d640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c003970 0x7feb0c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.202+0000 7f04ce0df640 1 -- start start 2023-11-24T08:08:55.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7feb2359e640 1 -- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c003970 msgr2=0x7feb0c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7feb2359e640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c003970 0x7feb0c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7feb2359e640 1 -- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0047c0 msgr2=0x7feb0c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7feb2359e640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0047c0 0x7feb0c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7feb2359e640 1 -- 172.21.15.106:0/43252438 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb240cedc0 con 0x7feb0c005180 2023-11-24T08:08:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7feb2259c640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0047c0 0x7feb0c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:55.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.201+0000 7fb36ab53640 1 -- 172.21.15.106:0/1722152951 wait complete. 2023-11-24T08:08:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.203+0000 7fb36ab53640 1 Processor -- start 2023-11-24T08:08:55.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.204+0000 7fb36ab53640 1 -- start start 2023-11-24T08:08:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.204+0000 7feb2359e640 1 -- 172.21.15.106:0/43252438 reap_dead start 2023-11-24T08:08:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.205+0000 7feb2359e640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb0c00b980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c1b0 tx=0x7feb0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a165757292703d57 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.205+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c005ad0 con 0x7feb0c005180 2023-11-24T08:08:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7f04ce0df640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f04c003c070 0x7f04c00e7750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7f04ce0df640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c0007b40 con 0x7f04c0053c80 2023-11-24T08:08:55.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7f04ce0df640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c00587e0 con 0x7f04c00e3130 2023-11-24T08:08:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7f04ce0df640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c0058ac0 con 0x7f04c003c070 2023-11-24T08:08:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.205+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c005c50 con 0x7feb0c005180 2023-11-24T08:08:55.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7f04cd0dd640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f04c00e3130 0x7f04c0058020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59822/0 (socket says 172.21.15.106:59822) 2023-11-24T08:08:55.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7f04cd0dd640 1 -- 172.21.15.106:0/3622787793 learned_addr learned my addr 172.21.15.106:0/3622787793 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:55.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.205+0000 7feb21d9b640 1 -- 172.21.15.106:0/43252438 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c02c810 con 0x7feb0c005180 2023-11-24T08:08:55.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7feb29097640 1 -- 172.21.15.106:0/43252438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 msgr2=0x7feb0c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7feb29097640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb0c00b980 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c1b0 tx=0x7feb0c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.206+0000 7feb29097640 1 -- 172.21.15.106:0/43252438 shutdown_connections 2023-11-24T08:08:55.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7feb29097640 1 --2- 172.21.15.106:0/43252438 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb0c00b980 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7f04cc8dc640 1 --2- 172.21.15.106:0/3622787793 >> v2:172.21.15.106:3300/0 conn(0x7f04c003c070 0x7f04c00e7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7feb29097640 1 -- 172.21.15.106:0/43252438 >> 172.21.15.106:0/43252438 conn(0x7feb2408af60 msgr2=0x7feb2409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7feb29097640 1 -- 172.21.15.106:0/43252438 shutdown_connections 2023-11-24T08:08:55.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7feb29097640 1 -- 172.21.15.106:0/43252438 wait complete. 2023-11-24T08:08:55.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7f04cc8dc640 1 -- 172.21.15.106:0/3622787793 >> v1:172.21.15.156:6789/0 conn(0x7f04c00e3130 legacy=0x7f04c0058020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.207+0000 7f04cc8dc640 1 -- 172.21.15.106:0/3622787793 >> v1:172.21.15.106:6789/0 conn(0x7f04c0053c80 legacy=0x7f04c00540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.208+0000 7f04cc8dc640 1 -- 172.21.15.106:0/3622787793 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c0006330 con 0x7f04c003c070 2023-11-24T08:08:55.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7fb36ab53640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c056a70 0x7fb35c04ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7fb36ab53640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35c058980 0x7fb35c050560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7fb36ab53640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c0d6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7fef873d5640 1 Processor -- start 2023-11-24T08:08:55.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7fef873d5640 1 -- start start 2023-11-24T08:08:55.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fb36ab53640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c0e9150 con 0x7fb35c058980 2023-11-24T08:08:55.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fb36ab53640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c050fb0 con 0x7fb35c056a70 2023-11-24T08:08:55.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7f718e6bb640 1 Processor -- start 2023-11-24T08:08:55.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7f718e6bb640 1 -- start start 2023-11-24T08:08:55.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fb36ab53640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35c051290 con 0x7fb35c0e76d0 2023-11-24T08:08:55.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.209+0000 7fb3688ef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c056a70 0x7fb35c04ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fb3688ef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c056a70 0x7fb35c04ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39584/0 (socket says 172.21.15.106:39584) 2023-11-24T08:08:55.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7f04cc8dc640 1 --2- 172.21.15.106:0/3622787793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c003c070 0x7f04c00e7750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04b80097a0 tx=0x7f04b802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59e2a48ef5e5b96 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fb3688ef640 1 -- 172.21.15.106:0/1784575580 learned_addr learned my addr 172.21.15.106:0/1784575580 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fc9339b0640 1 Processor -- start 2023-11-24T08:08:55.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7f04bffff640 1 -- 172.21.15.106:0/3622787793 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8030840 con 0x7f04c003c070 2023-11-24T08:08:55.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7f04bffff640 1 -- 172.21.15.106:0/3622787793 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b80309a0 con 0x7f04c003c070 2023-11-24T08:08:55.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7f04bffff640 1 -- 172.21.15.106:0/3622787793 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b8030d70 con 0x7f04c003c070 2023-11-24T08:08:55.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fb3690f0640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c0d6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7feb29097640 1 Processor -- start 2023-11-24T08:08:55.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7feb29097640 1 -- start start 2023-11-24T08:08:55.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f718e6bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f718e6bb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f71880cd1d0 0x7f71880cd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7fb363fff640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35c058980 0x7fb35c050560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7fb3690f0640 1 -- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c056a70 msgr2=0x7fb35c04ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7fb3690f0640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c056a70 0x7fb35c04ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7fb3690f0640 1 -- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35c058980 msgr2=0x7fb35c050560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f718e6bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718816ec30 con 0x7f71880cea40 2023-11-24T08:08:55.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f718e6bb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718808a210 con 0x7f71880cd1d0 2023-11-24T08:08:55.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f718e6bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880c97b0 con 0x7f718816e7d0 2023-11-24T08:08:55.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f04ce0df640 1 -- 172.21.15.106:0/3622787793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c003c070 msgr2=0x7f04c00e7750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.211+0000 7f04ce0df640 1 --2- 172.21.15.106:0/3622787793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c003c070 0x7f04c00e7750 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f04b80097a0 tx=0x7f04b802b980 comp rx=0 tx=0).stop 2023-11-24T08:08:55.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fb3690f0640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35c058980 0x7fb35c050560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f04ce0df640 1 -- 172.21.15.106:0/3622787793 shutdown_connections 2023-11-24T08:08:55.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f04ce0df640 1 --2- 172.21.15.106:0/3622787793 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c003c070 0x7f04c00e7750 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f04ce0df640 1 -- 172.21.15.106:0/3622787793 >> 172.21.15.106:0/3622787793 conn(0x7f04c0058eb0 msgr2=0x7f04c0059290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fb3690f0640 1 -- 172.21.15.106:0/1784575580 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3500094d0 con 0x7fb35c0e76d0 2023-11-24T08:08:55.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fef873d5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fef800d31a0 0x7fef800d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fef873d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef800ce010 0x7fef800ce430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fef873d5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef800d1070 con 0x7fef801609e0 2023-11-24T08:08:55.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fef873d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800d3b50 con 0x7fef800ce010 2023-11-24T08:08:55.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fef873d5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef800cedc0 con 0x7fef800d31a0 2023-11-24T08:08:55.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.210+0000 7fc9339b0640 1 -- start start 2023-11-24T08:08:55.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f04ce0df640 1 -- 172.21.15.106:0/3622787793 shutdown_connections 2023-11-24T08:08:55.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fb363fff640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35c058980 0x7fb35c050560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:55.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f04ce0df640 1 -- 172.21.15.106:0/3622787793 wait complete. 2023-11-24T08:08:55.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f718d6b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f71880cea40 0x7f71880cb310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32798/0 (socket says 172.21.15.106:32798) 2023-11-24T08:08:55.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f718d6b9640 1 -- 172.21.15.106:0/4156772374 learned_addr learned my addr 172.21.15.106:0/4156772374 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:55.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7f718ceb8640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fb3690f0640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c0d6a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb364064320 tx=0x7fb364068010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.212+0000 7fc9339b0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc92c0ce640 0x7fc92c0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fc9339b0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c0ce270 con 0x7fc92c0cfb60 2023-11-24T08:08:55.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fc9339b0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c06d0a0 con 0x7fc92c0cc470 2023-11-24T08:08:55.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fc9339b0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c06c560 con 0x7fc92c0ce640 2023-11-24T08:08:55.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7feb29097640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb24157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7feb29097640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 0x7feb24159cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7f718deba640 1 --2- 172.21.15.106:0/4156772374 >> v2:172.21.15.156:3300/0 conn(0x7f71880cd1d0 0x7f71880cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fef85171640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fef801609e0 0x7fef8006bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59828/0 (socket says 172.21.15.106:59828) 2023-11-24T08:08:55.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fef85171640 1 -- 172.21.15.106:0/1453023377 learned_addr learned my addr 172.21.15.106:0/1453023377 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:55.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fef85972640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fef800ce010 0x7fef800ce430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fc93174c640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc92c0cc470 0x7fc92c0cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59844/0 (socket says 172.21.15.106:59844) 2023-11-24T08:08:55.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fc93174c640 1 -- 172.21.15.106:0/3477881294 learned_addr learned my addr 172.21.15.106:0/3477881294 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:55.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7feb29097640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb2415a200 0x7feb24158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7f7187fff640 1 -- 172.21.15.106:0/4156772374 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198995058 0 0) 0x7f718816ec30 con 0x7f71880cea40 2023-11-24T08:08:55.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7f7187fff640 1 -- 172.21.15.106:0/4156772374 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7178003520 con 0x7f71880cea40 2023-11-24T08:08:55.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.213+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3640723c0 con 0x7fb35c0e76d0 2023-11-24T08:08:55.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fef84970640 1 --2- 172.21.15.106:0/1453023377 >> v2:172.21.15.156:3300/0 conn(0x7fef800d31a0 0x7fef800d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb35c051570 con 0x7fb35c0e76d0 2023-11-24T08:08:55.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7feb29097640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2415a680 con 0x7feb0c005180 2023-11-24T08:08:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb364072520 con 0x7fb35c0e76d0 2023-11-24T08:08:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fef85972640 1 -- 172.21.15.106:0/1453023377 >> v2:172.21.15.156:3300/0 conn(0x7fef800d31a0 msgr2=0x7fef800d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fef85972640 1 --2- 172.21.15.106:0/1453023377 >> v2:172.21.15.156:3300/0 conn(0x7fef800d31a0 0x7fef800d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fef85972640 1 -- 172.21.15.106:0/1453023377 >> v1:172.21.15.156:6789/0 conn(0x7fef801609e0 legacy=0x7fef8006bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fef85972640 1 -- 172.21.15.106:0/1453023377 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800cf0a0 con 0x7fef800ce010 2023-11-24T08:08:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb364072930 con 0x7fb35c0e76d0 2023-11-24T08:08:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7feb2259c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 0x7feb24159cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7feb2259c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 0x7feb24159cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48336/0 (socket says 172.21.15.106:48336) 2023-11-24T08:08:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7feb2259c640 1 -- 172.21.15.106:0/3853997594 learned_addr learned my addr 172.21.15.106:0/3853997594 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7feb22d9d640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb24157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fc931f4d640 1 --2- 172.21.15.106:0/3477881294 >> v2:172.21.15.156:3300/0 conn(0x7fc92c0ce640 0x7fc92c0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35c04bf70 con 0x7fb35c0e76d0 2023-11-24T08:08:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.214+0000 7feb2359e640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb2415a200 0x7feb24158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb24158040 con 0x7feb2415a200 2023-11-24T08:08:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7f718ceb8640 1 -- 172.21.15.106:0/4156772374 >> v2:172.21.15.156:3300/0 conn(0x7f71880cd1d0 msgr2=0x7f71880cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7f718ceb8640 1 --2- 172.21.15.106:0/4156772374 >> v2:172.21.15.156:3300/0 conn(0x7f71880cd1d0 0x7f71880cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7f718ceb8640 1 -- 172.21.15.106:0/4156772374 >> v1:172.21.15.106:6789/0 conn(0x7f71880cea40 legacy=0x7f71880cb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7f718ceb8640 1 -- 172.21.15.106:0/4156772374 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71880c9a90 con 0x7f718816e7d0 2023-11-24T08:08:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7f04ce0df640 1 Processor -- start 2023-11-24T08:08:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.216+0000 7f04ce0df640 1 -- start start 2023-11-24T08:08:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7fc923fff640 1 -- 172.21.15.106:0/3477881294 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92172967 0 0) 0x7fc92c06d0a0 con 0x7fc92c0cc470 2023-11-24T08:08:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7fc923fff640 1 -- 172.21.15.106:0/3477881294 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc918003520 con 0x7fc92c0cc470 2023-11-24T08:08:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7fc931f4d640 1 -- 172.21.15.106:0/3477881294 >> v1:172.21.15.156:6789/0 conn(0x7fc92c0cc470 legacy=0x7fc92c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7fc931f4d640 1 -- 172.21.15.106:0/3477881294 >> v1:172.21.15.106:6789/0 conn(0x7fc92c0cfb60 legacy=0x7fc92c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7fa660cbb640 1 Processor -- start 2023-11-24T08:08:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7fa660cbb640 1 -- start start 2023-11-24T08:08:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.216+0000 7fef85972640 1 --2- 172.21.15.106:0/1453023377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800ce010 0x7fef800ce430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef800d43c0 tx=0x7fef7802bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4783098c39c4b66b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.216+0000 7f718ceb8640 1 --2- 172.21.15.106:0/4156772374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880ccc90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f717400ba40 tx=0x7f717402de70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a97d891e3d1520 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7fef77fff640 1 -- 172.21.15.106:0/1453023377 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7802ca30 con 0x7fef800ce010 2023-11-24T08:08:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7fef77fff640 1 -- 172.21.15.106:0/1453023377 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef7802cb90 con 0x7fef800ce010 2023-11-24T08:08:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7fef77fff640 1 -- 172.21.15.106:0/1453023377 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef7802ca30 con 0x7fef800ce010 2023-11-24T08:08:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7f7187fff640 1 -- 172.21.15.106:0/4156772374 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0019f0 con 0x7f718816e7d0 2023-11-24T08:08:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7f7187fff640 1 -- 172.21.15.106:0/4156772374 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717402e9a0 con 0x7f718816e7d0 2023-11-24T08:08:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7f7187fff640 1 -- 172.21.15.106:0/4156772374 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717402ed30 con 0x7f718816e7d0 2023-11-24T08:08:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.216+0000 7fc931f4d640 1 -- 172.21.15.106:0/3477881294 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c06c770 con 0x7fc92c0ce640 2023-11-24T08:08:55.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7f04ce0df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c003c070 0x7f04c0052830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.215+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb240b2e40 con 0x7feb240b6510 2023-11-24T08:08:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7feb2359e640 1 -- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 msgr2=0x7feb24159cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7feb2359e640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 0x7feb24159cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7feb2359e640 1 -- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 msgr2=0x7feb24157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7feb2359e640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb24157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7feb2259c640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 0x7feb24159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fc931f4d640 1 --2- 172.21.15.106:0/3477881294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0ce640 0x7fc92c0abfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc924009d90 tx=0x7fc92402bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2de7d8d6ecd62d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 -- 172.21.15.106:0/1453023377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800ce010 msgr2=0x7fef800ce430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 --2- 172.21.15.106:0/1453023377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800ce010 0x7fef800ce430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fef800d43c0 tx=0x7fef7802bd00 comp rx=0 tx=0).stop 2023-11-24T08:08:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7f04ce0df640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 0x7f04c00d1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7fa660cbb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa65c06e6b0 0x7fa65c0ae480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.217+0000 7fa660cbb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa65c06da40 0x7fa65c06de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa660cbb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa65c06e3b0 con 0x7fa65c0ce450 2023-11-24T08:08:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa660cbb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c06cc70 con 0x7fa65c06da40 2023-11-24T08:08:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa660cbb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa65c0cff70 con 0x7fa65c06e6b0 2023-11-24T08:08:55.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa65ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa65c06da40 0x7fa65c06de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35c0e9290 con 0x7fb35c0e76d0 2023-11-24T08:08:55.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7feb2359e640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24158ea0 con 0x7feb2415a200 2023-11-24T08:08:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7f04ce0df640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c00e3130 0x7f04c0051880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 -- 172.21.15.106:0/1453023377 shutdown_connections 2023-11-24T08:08:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 --2- 172.21.15.106:0/1453023377 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800ce010 0x7fef800ce430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 --2- 172.21.15.106:0/1453023377 >> v2:172.21.15.156:3300/0 conn(0x7fef800d31a0 0x7fef800d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 -- 172.21.15.106:0/1453023377 >> 172.21.15.106:0/1453023377 conn(0x7fef8008af60 msgr2=0x7fef8009c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fef873d5640 1 -- 172.21.15.106:0/1453023377 shutdown_connections 2023-11-24T08:08:55.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa65ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa65c06da40 0x7fa65c06de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34292/0 (socket says 172.21.15.106:34292) 2023-11-24T08:08:55.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa65ad9d640 1 -- 172.21.15.106:0/2265694541 learned_addr learned my addr 172.21.15.106:0/2265694541 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.218+0000 7fa65a59c640 1 --2- 172.21.15.106:0/2265694541 >> v2:172.21.15.156:3300/0 conn(0x7fa65c06e6b0 0x7fa65c0ae480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04ce0df640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00df080 con 0x7f04c00e3130 2023-11-24T08:08:55.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04ce0df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00df260 con 0x7f04c003c070 2023-11-24T08:08:55.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04ce0df640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c00df540 con 0x7f04c0053c80 2023-11-24T08:08:55.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 -- 172.21.15.106:0/4156772374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 msgr2=0x7f71880ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4156772374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880ccc90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f717400ba40 tx=0x7f717402de70 comp rx=0 tx=0).stop 2023-11-24T08:08:55.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fa65ad9d640 1 -- 172.21.15.106:0/2265694541 >> v2:172.21.15.156:3300/0 conn(0x7fa65c06e6b0 msgr2=0x7fa65c0ae480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fa65ad9d640 1 --2- 172.21.15.106:0/2265694541 >> v2:172.21.15.156:3300/0 conn(0x7fa65c06e6b0 0x7fa65c0ae480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fa65ad9d640 1 -- 172.21.15.106:0/2265694541 >> v1:172.21.15.156:6789/0 conn(0x7fa65c0ce450 legacy=0x7fa65c06c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb364074940 con 0x7fb35c0e76d0 2023-11-24T08:08:55.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 -- 172.21.15.106:0/4156772374 shutdown_connections 2023-11-24T08:08:55.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4156772374 >> v2:172.21.15.156:3300/0 conn(0x7f71880cd1d0 0x7f71880cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fc923fff640 1 -- 172.21.15.106:0/3477881294 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92402ccf0 con 0x7fc92c0ce640 2023-11-24T08:08:55.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4156772374 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880ccc90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04cc8dc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 0x7f04c00d1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fc923fff640 1 -- 172.21.15.106:0/3477881294 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92402ce50 con 0x7fc92c0ce640 2023-11-24T08:08:55.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fc923fff640 1 -- 172.21.15.106:0/3477881294 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924034660 con 0x7fc92c0ce640 2023-11-24T08:08:55.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fa65ad9d640 1 -- 172.21.15.106:0/2265694541 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa65c0b3970 con 0x7fa65c06da40 2023-11-24T08:08:55.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 -- 172.21.15.106:0/4156772374 >> 172.21.15.106:0/4156772374 conn(0x7f71880cbb40 msgr2=0x7f71880cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04cd8de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c00e3130 0x7f04c0051880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04cc8dc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 0x7f04c00d1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48344/0 (socket says 172.21.15.106:48344) 2023-11-24T08:08:55.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04cc8dc640 1 -- 172.21.15.106:0/2759003436 learned_addr learned my addr 172.21.15.106:0/2759003436 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f04cd0dd640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c003c070 0x7f04c0052830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7feb2359e640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb2415a200 0x7feb24158200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb2408bc10 tx=0x7feb1c000fe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7f718e6bb640 1 -- 172.21.15.106:0/4156772374 shutdown_connections 2023-11-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7f04cc8dc640 1 -- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c003c070 msgr2=0x7f04c0052830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7f04cc8dc640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c003c070 0x7f04c0052830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7f04cc8dc640 1 -- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c00e3130 msgr2=0x7f04c0051880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7f04cc8dc640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c00e3130 0x7f04c0051880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7fc9339b0640 1 -- 172.21.15.106:0/3477881294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0ce640 msgr2=0x7fc92c0abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7fc9339b0640 1 --2- 172.21.15.106:0/3477881294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0ce640 0x7fc92c0abfc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc924009d90 tx=0x7fc92402bf40 comp rx=0 tx=0).stop 2023-11-24T08:08:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7f04cc8dc640 1 -- 172.21.15.106:0/2759003436 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b8009400 con 0x7f04c0053c80 2023-11-24T08:08:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7fc9339b0640 1 -- 172.21.15.106:0/3477881294 shutdown_connections 2023-11-24T08:08:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7fc9339b0640 1 --2- 172.21.15.106:0/3477881294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0ce640 0x7fc92c0abfc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7fc9339b0640 1 -- 172.21.15.106:0/3477881294 >> 172.21.15.106:0/3477881294 conn(0x7fc92c06a920 msgr2=0x7fc92c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fb3617fa640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb34806ae40 0x7fb34806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c039050 con 0x7feb2415a200 2023-11-24T08:08:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c03a040 con 0x7feb2415a200 2023-11-24T08:08:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c0072b0 con 0x7feb2415a200 2023-11-24T08:08:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb24159180 con 0x7feb2415a200 2023-11-24T08:08:55.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.220+0000 7f718e6bb640 1 -- 172.21.15.106:0/4156772374 wait complete. 2023-11-24T08:08:55.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7f04cc8dc640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 0x7f04c00d1d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098c0 tx=0x7f04b8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fc9339b0640 1 -- 172.21.15.106:0/3477881294 shutdown_connections 2023-11-24T08:08:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.219+0000 7fef873d5640 1 -- 172.21.15.106:0/1453023377 wait complete. 2023-11-24T08:08:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb240b3370 con 0x7feb2415a200 2023-11-24T08:08:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fef873d5640 1 Processor -- start 2023-11-24T08:08:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fef873d5640 1 -- start start 2023-11-24T08:08:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fc9339b0640 1 -- 172.21.15.106:0/3477881294 wait complete. 2023-11-24T08:08:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fa65ad9d640 1 --2- 172.21.15.106:0/2265694541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06da40 0x7fa65c06de70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0ce3d0 tx=0x7fa64c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d3690edfdf16711 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.221+0000 7fb3688ef640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb34806ae40 0x7fb34806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.222+0000 7f718e6bb640 1 Processor -- start 2023-11-24T08:08:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.222+0000 7fa65959a640 1 -- 172.21.15.106:0/2265694541 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c031640 con 0x7fa65c06da40 2023-11-24T08:08:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.222+0000 7fa65959a640 1 -- 172.21.15.106:0/2265694541 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa64c0317a0 con 0x7fa65c06da40 2023-11-24T08:08:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.222+0000 7fa65959a640 1 -- 172.21.15.106:0/2265694541 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa64c031b30 con 0x7fa65c06da40 2023-11-24T08:08:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.223+0000 7f718e6bb640 1 -- start start 2023-11-24T08:08:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.222+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3641036f0 con 0x7fb35c0e76d0 2023-11-24T08:08:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.223+0000 7fef873d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef800ce010 0x7fef80170c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.223+0000 7fa660cbb640 1 -- 172.21.15.106:0/2265694541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06da40 msgr2=0x7fa65c06de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.223+0000 7fa660cbb640 1 --2- 172.21.15.106:0/2265694541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06da40 0x7fa65c06de70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa65c0ce3d0 tx=0x7fa64c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fa660cbb640 1 -- 172.21.15.106:0/2265694541 shutdown_connections 2023-11-24T08:08:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fa660cbb640 1 --2- 172.21.15.106:0/2265694541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06da40 0x7fa65c06de70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fa660cbb640 1 --2- 172.21.15.106:0/2265694541 >> v2:172.21.15.156:3300/0 conn(0x7fa65c06e6b0 0x7fa65c0ae480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fa660cbb640 1 -- 172.21.15.106:0/2265694541 >> 172.21.15.106:0/2265694541 conn(0x7fa65c06a920 msgr2=0x7fa65c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fef873d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800d31a0 0x7fef800b69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fa660cbb640 1 -- 172.21.15.106:0/2265694541 shutdown_connections 2023-11-24T08:08:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fef873d5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 0x7fef8006e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb240b3560 con 0x7feb2415a200 2023-11-24T08:08:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fef873d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800b7f60 con 0x7fef800d31a0 2023-11-24T08:08:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fb3688ef640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb34806ae40 0x7fb34806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3500090b0 tx=0x7fb35003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.224+0000 7fa660cbb640 1 -- 172.21.15.106:0/2265694541 wait complete. 2023-11-24T08:08:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fef85972640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 0x7fef8006e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fef85972640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 0x7fef8006e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48354/0 (socket says 172.21.15.106:48354) 2023-11-24T08:08:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fef85972640 1 -- 172.21.15.106:0/813782519 learned_addr learned my addr 172.21.15.106:0/813782519 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7f718e6bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 0x7f718816d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef800b7350 con 0x7fef800ce010 2023-11-24T08:08:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef800b7630 con 0x7fef801609e0 2023-11-24T08:08:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7f718e6bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71880cea40 0x7f71880c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fc9339b0640 1 Processor -- start 2023-11-24T08:08:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fef85171640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef800ce010 0x7fef80170c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7fc9339b0640 1 -- start start 2023-11-24T08:08:55.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7f718d6b9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 0x7f718816d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.225+0000 7f718d6b9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 0x7f718816d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39648/0 (socket says 172.21.15.106:39648) 2023-11-24T08:08:55.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef84970640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800d31a0 0x7fef800b69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7f718d6b9640 1 -- 172.21.15.106:0/4086319045 learned_addr learned my addr 172.21.15.106:0/4086319045 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7f98eb2d1640 1 Processor -- start 2023-11-24T08:08:55.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7f718ceb8640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71880cea40 0x7f71880c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7f98eb2d1640 1 -- start start 2023-11-24T08:08:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef85972640 1 -- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef800ce010 msgr2=0x7fef80170c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef85972640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef800ce010 0x7fef80170c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fa660cbb640 1 Processor -- start 2023-11-24T08:08:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef85972640 1 -- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800d31a0 msgr2=0x7fef800b69e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef85972640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800d31a0 0x7fef800b69e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fa660cbb640 1 -- start start 2023-11-24T08:08:55.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef85972640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef78009a70 con 0x7fef801609e0 2023-11-24T08:08:55.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef85171640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef800ce010 0x7fef80170c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:55.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7fef84970640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800d31a0 0x7fef800b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:55.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.226+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880c7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7feb1c032070 con 0x7feb2415a200 2023-11-24T08:08:55.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc9339b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0cc470 0x7fc92c15e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880c5310 con 0x7f71880cea40 2023-11-24T08:08:55.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc9339b0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 0x7fc92c162ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71880b2c10 con 0x7f71880cd1d0 2023-11-24T08:08:55.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc9339b0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c0cfb60 0x7fc92c09ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc9339b0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c163c90 con 0x7fc92c0ce640 2023-11-24T08:08:55.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc9339b0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1636d0 con 0x7fc92c0cc470 2023-11-24T08:08:55.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc9339b0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c1639b0 con 0x7fc92c0cfb60 2023-11-24T08:08:55.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71880b28e0 con 0x7f718816e7d0 2023-11-24T08:08:55.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc930f4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 0x7fc92c162ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc930f4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 0x7fc92c162ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34334/0 (socket says 172.21.15.106:34334) 2023-11-24T08:08:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fef85972640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 0x7fef8006e6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fef78009e80 tx=0x7fef78004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc930f4b640 1 -- 172.21.15.106:0/805025582 learned_addr learned my addr 172.21.15.106:0/805025582 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7fc93174c640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0cc470 0x7fc92c15e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fc931f4d640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c0cfb60 0x7fc92c09ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.227+0000 7f98eb2d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98e40b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f98eb2d1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40d1d80 con 0x7f98e40d1610 2023-11-24T08:08:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f98eb2d1640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98e40d2060 con 0x7f98e415d5c0 2023-11-24T08:08:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f98eb2d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40d2340 con 0x7f98e40d08b0 2023-11-24T08:08:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f718deba640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880c7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa660cbb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa65c06da40 0x7fa65c146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa660cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 0x7fa65c166a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa660cbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa65c0ce450 0x7fa65c1499c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa660cbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c146da0 con 0x7fa65c06e6b0 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa660cbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c071460 con 0x7fa65c0ce450 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa660cbb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa65c071740 con 0x7fa65c06da40 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fc930f4b640 1 -- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c0cfb60 msgr2=0x7fc92c09ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fc930f4b640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c0cfb60 0x7fc92c09ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fc930f4b640 1 -- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0cc470 msgr2=0x7fc92c15e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fc930f4b640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0cc470 0x7fc92c15e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fc930f4b640 1 -- 172.21.15.106:0/805025582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc924009a70 con 0x7fc92c0ce640 2023-11-24T08:08:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7feb0b7fe640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb1006af40 0x7feb1006d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb1c0aaf50 con 0x7feb2415a200 2023-11-24T08:08:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f718d6b9640 1 -- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 msgr2=0x7f71880c7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f718d6b9640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880c7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7f98ea2cf640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f98e415d5c0 0x7f98e40d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59862/0 (socket says 172.21.15.106:59862) 2023-11-24T08:08:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7f98ea2cf640 1 -- 172.21.15.106:0/2715678987 learned_addr learned my addr 172.21.15.106:0/2715678987 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7feb22d9d640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb1006af40 0x7feb1006d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa659d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 0x7fa65c166a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa659d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 0x7fa65c166a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34344/0 (socket says 172.21.15.106:34344) 2023-11-24T08:08:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7fa659d9b640 1 -- 172.21.15.106:0/721928989 learned_addr learned my addr 172.21.15.106:0/721928989 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f718d6b9640 1 -- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71880cea40 msgr2=0x7f71880c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.228+0000 7f718d6b9640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71880cea40 0x7f71880c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7f98e9ace640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98e40b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7f718d6b9640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717400b6a0 con 0x7f71880cd1d0 2023-11-24T08:08:55.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa65a59c640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa65c06da40 0x7fa65c146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa65ad9d640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa65c0ce450 0x7fa65c1499c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa659d9b640 1 -- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa65c06da40 msgr2=0x7fa65c146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa659d9b640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa65c06da40 0x7fa65c146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa659d9b640 1 -- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa65c0ce450 msgr2=0x7fa65c1499c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa659d9b640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa65c0ce450 0x7fa65c1499c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fa659d9b640 1 -- 172.21.15.106:0/721928989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa64c0094d0 con 0x7fa65c06e6b0 2023-11-24T08:08:55.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7f718d6b9640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 0x7f718816d380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f717c00cbc0 tx=0x7f717c00a430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fc930f4b640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 0x7fc92c162ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c400 tx=0x7fc92800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280052d0 con 0x7fc92c0ce640 2023-11-24T08:08:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928005430 con 0x7fc92c0ce640 2023-11-24T08:08:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928003830 con 0x7fc92c0ce640 2023-11-24T08:08:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.229+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3975615639 0 0) 0x7f98e40d1d80 con 0x7f98e40d1610 2023-11-24T08:08:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98d0003520 con 0x7f98e40d1610 2023-11-24T08:08:55.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c099300 con 0x7fc92c0ce640 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fa659d9b640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 0x7fa65c166a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b3f0 tx=0x7fa65000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c15eeb0 con 0x7fc92c0ce640 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3255277840 0 0) 0x7f98d0003520 con 0x7f98e40d1610 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 msgr2=0x7f98e40b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7f98e92cd640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98e40b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 >> v1:172.21.15.156:6789/0 conn(0x7f98e415d5c0 legacy=0x7f98e40d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40d2620 con 0x7f98e40d1610 2023-11-24T08:08:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7feb22d9d640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb1006af40 0x7feb1006d400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb14003bd0 tx=0x7feb14009760 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650011830 con 0x7fa65c06e6b0 2023-11-24T08:08:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa65c071a20 con 0x7fa65c06e6b0 2023-11-24T08:08:55.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.230+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa65c166f90 con 0x7fa65c06e6b0 2023-11-24T08:08:55.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f98dc0019f0 con 0x7f98e40d1610 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 >> v1:172.21.15.106:6789/0 conn(0x7f98e40d1610 legacy=0x7f98e40b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d00039a0 0x7f98d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98d0004d00 0x7f98d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40d2340 con 0x7f98d0004d00 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d000bae0 con 0x7f98d00039a0 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d000bca0 con 0x7f98e40d08b0 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98ea2cf640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d00039a0 0x7f98d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.231+0000 7f98e9ace640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa650009220 con 0x7fa65c06e6b0 2023-11-24T08:08:55.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa650011b90 con 0x7fa65c06e6b0 2023-11-24T08:08:55.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98eaad0640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98d0004d00 0x7f98d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98ea2cf640 1 -- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 msgr2=0x7f98d00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98ea2cf640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98d00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98ea2cf640 1 -- 172.21.15.106:0/2715678987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98d0004d00 msgr2=0x7f98d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98ea2cf640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98d0004d00 0x7f98d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98ea2cf640 1 -- 172.21.15.106:0/2715678987 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d0003520 con 0x7f98d00039a0 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98e9ace640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98e40d08b0 0x7f98d00047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.232+0000 7f98eaad0640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98d0004d00 0x7f98d0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:55.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.233+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa640005250 con 0x7fa65c06e6b0 2023-11-24T08:08:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.233+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c061a50 con 0x7fc92c0ce640 2023-11-24T08:08:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.233+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc928003990 con 0x7fc92c0ce640 2023-11-24T08:08:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.234+0000 7fc921ffb640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc91006ae90 0x7fc91006d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.234+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc928088c30 con 0x7fc92c0ce640 2023-11-24T08:08:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.234+0000 7f98ea2cf640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d00039a0 0x7f98d0003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98e0001810 tx=0x7f98d00057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3358daf54fbc687 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.234+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa650011cf0 con 0x7fa65c06e6b0 2023-11-24T08:08:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.234+0000 7fa64affd640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa63c06ae40 0x7fa63c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.235+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa650089790 con 0x7fa65c06e6b0 2023-11-24T08:08:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.235+0000 7fc93174c640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc91006ae90 0x7fc91006d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.235+0000 7fa65a59c640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa63c06ae40 0x7fa63c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.235+0000 7fc93174c640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc91006ae90 0x7fc91006d350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0011e0 tx=0x7fc91c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.236+0000 7fa65a59c640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa63c06ae40 0x7fa63c06d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa654009a60 tx=0x7fa654003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.239+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc928053cc0 con 0x7fc92c0ce640 2023-11-24T08:08:55.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.239+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b803e040 con 0x7f04c0053c80 2023-11-24T08:08:55.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c00df820 con 0x7f04c0053c80 2023-11-24T08:08:55.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b80443f0 con 0x7f04c0053c80 2023-11-24T08:08:55.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80447a0 con 0x7f04c0053c80 2023-11-24T08:08:55.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c00dfde0 con 0x7f04c0053c80 2023-11-24T08:08:55.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef78003e10 con 0x7fef801609e0 2023-11-24T08:08:55.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef78004250 con 0x7fef801609e0 2023-11-24T08:08:55.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.241+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb35c0e9290 con 0x7fb35c0e76d0 2023-11-24T08:08:55.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.240+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef800b7910 con 0x7fef801609e0 2023-11-24T08:08:55.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.241+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef80171170 con 0x7fef801609e0 2023-11-24T08:08:55.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.241+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c0005360 con 0x7f04c0053c80 2023-11-24T08:08:55.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.241+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef780045c0 con 0x7fef801609e0 2023-11-24T08:08:55.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.242+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef60005250 con 0x7fef801609e0 2023-11-24T08:08:55.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.244+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f04b804d3b0 con 0x7f04c0053c80 2023-11-24T08:08:55.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.244+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fef7803b070 con 0x7fef801609e0 2023-11-24T08:08:55.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.245+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa6500547a0 con 0x7fa65c06e6b0 2023-11-24T08:08:55.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f04bdffb640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f04ac06af10 0x7f04ac06d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004aa0 con 0x7f98d00039a0 2023-11-24T08:08:55.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c004d60 con 0x7f71880cd1d0 2023-11-24T08:08:55.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71880c8440 con 0x7f71880cd1d0 2023-11-24T08:08:55.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7fef75ffb640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef6406af10 0x7fef6406d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb1c0761b0 con 0x7feb2415a200 2023-11-24T08:08:55.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71880c8a00 con 0x7f71880cd1d0 2023-11-24T08:08:55.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef780abba0 con 0x7fef801609e0 2023-11-24T08:08:55.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e002e070 con 0x7f98d00039a0 2023-11-24T08:08:55.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.246+0000 7f98e92cd640 1 -- 172.21.15.106:0/2715678987 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e0004e00 con 0x7f98d00039a0 2023-11-24T08:08:55.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.247+0000 7fef85171640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef6406af10 0x7fef6406d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.247+0000 7fef85171640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef6406af10 0x7fef6406d3d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef70000f60 tx=0x7fef70007e00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.247+0000 7f98eb2d1640 1 -- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d00039a0 msgr2=0x7f98d0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.247+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717c00ab40 con 0x7f71880cd1d0 2023-11-24T08:08:55.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.247+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717c0070d0 con 0x7f71880cd1d0 2023-11-24T08:08:55.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.248+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f04b80ac250 con 0x7f04c0053c80 2023-11-24T08:08:55.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.247+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/2715678987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98d00039a0 0x7f98d0003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f98e0001810 tx=0x7f98d00057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.248+0000 7f98eaad0640 1 -- 172.21.15.106:0/2715678987 reap_dead start 2023-11-24T08:08:55.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.248+0000 7f98eb2d1640 1 -- 172.21.15.106:0/2715678987 shutdown_connections 2023-11-24T08:08:55.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.248+0000 7f98eb2d1640 1 -- 172.21.15.106:0/2715678987 >> 172.21.15.106:0/2715678987 conn(0x7f98e4061310 msgr2=0x7f98e4061720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.248+0000 7f04cd0dd640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f04ac06af10 0x7f04ac06d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.248+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7168005250 con 0x7f71880cd1d0 2023-11-24T08:08:55.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.249+0000 7f98eb2d1640 1 -- 172.21.15.106:0/2715678987 shutdown_connections 2023-11-24T08:08:55.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.249+0000 7f98eb2d1640 1 -- 172.21.15.106:0/2715678987 wait complete. 2023-11-24T08:08:55.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.249+0000 7f04cd0dd640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f04ac06af10 0x7f04ac06d3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04c4005880 tx=0x7f04c4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.252+0000 7f98eb2d1640 1 Processor -- start 2023-11-24T08:08:55.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.252+0000 7f98eb2d1640 1 -- start start 2023-11-24T08:08:55.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.251+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f717c007230 con 0x7f71880cd1d0 2023-11-24T08:08:55.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.252+0000 7f7185ffb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f716c06ae40 0x7f716c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.252+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f717c089180 con 0x7f71880cd1d0 2023-11-24T08:08:55.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.252+0000 7f718ceb8640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f716c06ae40 0x7f716c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.253+0000 7f98eb2d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d00039a0 0x7f98e40c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.253+0000 7f98eb2d1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 0x7f98e40cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.253+0000 7f98eb2d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98e40cc820 0x7f98e4159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.253+0000 7f98eb2d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40ccca0 con 0x7f98e40cc820 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.253+0000 7f98eb2d1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40cc0e0 con 0x7f98e4157a20 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.253+0000 7f98eb2d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98e40c8510 con 0x7f98d00039a0 2023-11-24T08:08:55.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.254+0000 7f718ceb8640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f716c06ae40 0x7f716c06d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7174038000 tx=0x7f7174032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.254+0000 7f98ea2cf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d00039a0 0x7f98e40c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.254+0000 7f98e9ace640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 0x7f98e40cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.254+0000 7f98e9ace640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 0x7f98e40cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39690/0 (socket says 172.21.15.106:39690) 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.254+0000 7f98e9ace640 1 -- 172.21.15.106:0/1311596373 learned_addr learned my addr 172.21.15.106:0/1311596373 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.254+0000 7f98eaad0640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98e40cc820 0x7f98e4159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.255+0000 7f98e9ace640 1 -- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d00039a0 msgr2=0x7f98e40c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.255+0000 7f98e9ace640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d00039a0 0x7f98e40c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.255+0000 7f98e9ace640 1 -- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98e40cc820 msgr2=0x7f98e4159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.255+0000 7f98e9ace640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98e40cc820 0x7f98e4159070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.255+0000 7f98e9ace640 1 -- 172.21.15.106:0/1311596373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98e40c7750 con 0x7f98e4157a20 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.255+0000 7f98eaad0640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98e40cc820 0x7f98e4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:55.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.256+0000 7f98e9ace640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 0x7f98e40cc2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011ca0 tx=0x7f98d400f610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.258+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f04b8077300 con 0x7f04c0053c80 2023-11-24T08:08:55.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.267+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4011cd0 con 0x7f98e4157a20 2023-11-24T08:08:55.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.267+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e40c7a30 con 0x7f98e4157a20 2023-11-24T08:08:55.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.267+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d4011e30 con 0x7f98e4157a20 2023-11-24T08:08:55.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.267+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d4014680 con 0x7f98e4157a20 2023-11-24T08:08:55.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.268+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e40c7ff0 con 0x7f98e4157a20 2023-11-24T08:08:55.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.268+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f717c054370 con 0x7f71880cd1d0 2023-11-24T08:08:55.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.270+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98e40c8210 con 0x7f98e4157a20 2023-11-24T08:08:55.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.272+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f98d403dd50 con 0x7f98e4157a20 2023-11-24T08:08:55.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.272+0000 7f98daffd640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f98c806ae70 0x7f98c806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:55.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.272+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98d4089fc0 con 0x7f98e4157a20 2023-11-24T08:08:55.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.273+0000 7f98ea2cf640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f98c806ae70 0x7f98c806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:55.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.273+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fef78076e90 con 0x7fef801609e0 2023-11-24T08:08:55.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.274+0000 7f98ea2cf640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f98c806ae70 0x7f98c806d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98e0033000 tx=0x7f98e0009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:55.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.286+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98d4055050 con 0x7f98e4157a20 2023-11-24T08:08:55.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.640+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc92c0ac050 con 0x7fc92c0ce640 2023-11-24T08:08:55.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.641+0000 7fc921ffb640 1 -- 172.21.15.106:0/805025582 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc928017020 con 0x7fc92c0ce640 2023-11-24T08:08:55.643 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:55.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.644+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc91006ae90 msgr2=0x7fc91006d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.644+0000 7fc9339b0640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc91006ae90 0x7fc91006d350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0011e0 tx=0x7fc91c009040 comp rx=0 tx=0).stop 2023-11-24T08:08:55.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.644+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 msgr2=0x7fc92c162ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.644+0000 7fc9339b0640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 0x7fc92c162ce0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc92800c400 tx=0x7fc92800c8c0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.645+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 shutdown_connections 2023-11-24T08:08:55.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.645+0000 7fc9339b0640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc91006ae90 0x7fc91006d350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.645+0000 7fc9339b0640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc92c0cfb60 0x7fc92c09ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.645+0000 7fc9339b0640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc92c0ce640 0x7fc92c162ce0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.645+0000 7fc9339b0640 1 --2- 172.21.15.106:0/805025582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc92c0cc470 0x7fc92c15e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.645+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 >> 172.21.15.106:0/805025582 conn(0x7fc92c06a920 msgr2=0x7fc92c0734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.646+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 shutdown_connections 2023-11-24T08:08:55.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.646+0000 7fc9339b0640 1 -- 172.21.15.106:0/805025582 wait complete. 2023-11-24T08:08:55.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.654+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7feb24159340 con 0x7feb2415a200 2023-11-24T08:08:55.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.654+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa640005500 con 0x7fa65c06e6b0 2023-11-24T08:08:55.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.654+0000 7fa64affd640 1 -- 172.21.15.106:0/721928989 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa650057dc0 con 0x7fa65c06e6b0 2023-11-24T08:08:55.656 INFO:teuthology.orchestra.run.smithi106.stdout:60129542147 2023-11-24T08:08:55.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.655+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3853997594 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7feb1c046090 con 0x7feb2415a200 2023-11-24T08:08:55.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.655+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb35c0e9290 con 0x7fb35c0e76d0 2023-11-24T08:08:55.657 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:55.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.656+0000 7fb3617fa640 1 -- 172.21.15.106:0/1784575580 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb35c0e9290 con 0x7fb35c0e76d0 2023-11-24T08:08:55.661 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:55.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb1006af40 msgr2=0x7feb1006d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7feb29097640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb1006af40 0x7feb1006d400 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb14003bd0 tx=0x7feb14009760 comp rx=0 tx=0).stop 2023-11-24T08:08:55.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb2415a200 msgr2=0x7feb24158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7feb29097640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb2415a200 0x7feb24158200 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feb2408bc10 tx=0x7feb1c000fe0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 shutdown_connections 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb2415a200 0x7feb24158200 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb1006af40 0x7feb1006d400 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb240b6510 0x7feb24159cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 --2- 172.21.15.106:0/3853997594 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c005180 0x7feb24157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 >> 172.21.15.106:0/3853997594 conn(0x7feb2408af60 msgr2=0x7feb240b2050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 shutdown_connections 2023-11-24T08:08:55.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa63c06ae40 msgr2=0x7fa63c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7fa660cbb640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa63c06ae40 0x7fa63c06d300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa654009a60 tx=0x7fa654003990 comp rx=0 tx=0).stop 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 msgr2=0x7fa65c166a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.658+0000 7fa660cbb640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 0x7fa65c166a50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b3f0 tx=0x7fa65000b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 shutdown_connections 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7fa660cbb640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa65c0ce450 0x7fa65c1499c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7fa660cbb640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa65c06e6b0 0x7fa65c166a50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7fa660cbb640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa63c06ae40 0x7fa63c06d300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7fa660cbb640 1 --2- 172.21.15.106:0/721928989 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa65c06da40 0x7fa65c146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 >> 172.21.15.106:0/721928989 conn(0x7fa65c06a920 msgr2=0x7fa65c0ae120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.659+0000 7feb29097640 1 -- 172.21.15.106:0/3853997594 wait complete. 2023-11-24T08:08:55.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.660+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 shutdown_connections 2023-11-24T08:08:55.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.660+0000 7fa660cbb640 1 -- 172.21.15.106:0/721928989 wait complete. 2023-11-24T08:08:55.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.660+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb34806ae40 msgr2=0x7fb34806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb34806ae40 0x7fb34806d300 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3500090b0 tx=0x7fb35003c040 comp rx=0 tx=0).stop 2023-11-24T08:08:55.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 msgr2=0x7fb35c0d6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c0d6a80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb364064320 tx=0x7fb364068010 comp rx=0 tx=0).stop 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 shutdown_connections 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb34806ae40 0x7fb34806d300 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb3500090b0 tx=0x7fb35003c040 comp rx=0 tx=0).stop 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb35c0e76d0 0x7fb35c0d6a80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35c058980 0x7fb35c050560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 --2- 172.21.15.106:0/1784575580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb35c056a70 0x7fb35c04ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.661+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 >> 172.21.15.106:0/1784575580 conn(0x7fb35c058ea0 msgr2=0x7fb35c04d780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.663+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 shutdown_connections 2023-11-24T08:08:55.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.663+0000 7fb36ab53640 1 -- 172.21.15.106:0/1784575580 wait complete. 2023-11-24T08:08:55.673 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.7 2023-11-24T08:08:55.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.679+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f04c00df680 con 0x7f04c0053c80 2023-11-24T08:08:55.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.679+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f71680057d0 con 0x7f71880cd1d0 2023-11-24T08:08:55.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.680+0000 7f04bdffb640 1 -- 172.21.15.106:0/2759003436 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f04c00df680 con 0x7f04c0053c80 2023-11-24T08:08:55.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.680+0000 7f7185ffb640 1 -- 172.21.15.106:0/4086319045 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f717c018020 con 0x7f71880cd1d0 2023-11-24T08:08:55.682 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:55.683 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:55.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.1 2023-11-24T08:08:55.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.683+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f716c06ae40 msgr2=0x7f716c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.683+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f716c06ae40 0x7f716c06d300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7174038000 tx=0x7f7174032000 comp rx=0 tx=0).stop 2023-11-24T08:08:55.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.683+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fef600057d0 con 0x7fef801609e0 2023-11-24T08:08:55.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.683+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 msgr2=0x7f718816d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.683+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 0x7f718816d380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f717c00cbc0 tx=0x7f717c00a430 comp rx=0 tx=0).stop 2023-11-24T08:08:55.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 shutdown_connections 2023-11-24T08:08:55.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f716c06ae40 0x7f716c06d300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718816e7d0 0x7f71880c7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f71880cea40 0x7f71880c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 --2- 172.21.15.106:0/4086319045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71880cd1d0 0x7f718816d380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 >> 172.21.15.106:0/4086319045 conn(0x7f71880cbb40 msgr2=0x7f71880ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 shutdown_connections 2023-11-24T08:08:55.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f718e6bb640 1 -- 172.21.15.106:0/4086319045 wait complete. 2023-11-24T08:08:55.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f04ac06af10 msgr2=0x7f04ac06d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f04ac06af10 0x7f04ac06d3d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f04c4005880 tx=0x7f04c4009040 comp rx=0 tx=0).stop 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 msgr2=0x7f04c00d1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 0x7f04c00d1d60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f04b80098c0 tx=0x7f04b8004730 comp rx=0 tx=0).stop 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 shutdown_connections 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f04c00e3130 0x7f04c0051880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f04ac06af10 0x7f04ac06d3d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f04c0053c80 0x7f04c00d1d60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 --2- 172.21.15.106:0/2759003436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f04c003c070 0x7f04c0052830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.684+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 >> 172.21.15.106:0/2759003436 conn(0x7f04c0058eb0 msgr2=0x7f04c0039590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.685+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 shutdown_connections 2023-11-24T08:08:55.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.685+0000 7f04ce0df640 1 -- 172.21.15.106:0/2759003436 wait complete. 2023-11-24T08:08:55.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.685+0000 7fef75ffb640 1 -- 172.21.15.106:0/813782519 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef78042090 con 0x7fef801609e0 2023-11-24T08:08:55.689 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:55.690 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542147 for osd.5 2023-11-24T08:08:55.691 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.0 2023-11-24T08:08:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef6406af10 msgr2=0x7fef6406d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef6406af10 0x7fef6406d3d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fef70000f60 tx=0x7fef70007e00 comp rx=0 tx=0).stop 2023-11-24T08:08:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 msgr2=0x7fef8006e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 0x7fef8006e6c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fef78009e80 tx=0x7fef78004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 shutdown_connections 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fef6406af10 0x7fef6406d3d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fef801609e0 0x7fef8006e6c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fef800d31a0 0x7fef800b69e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 --2- 172.21.15.106:0/813782519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef800ce010 0x7fef80170c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.688+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 >> 172.21.15.106:0/813782519 conn(0x7fef8008af60 msgr2=0x7fef800ce5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.689+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 shutdown_connections 2023-11-24T08:08:55.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.689+0000 7fef873d5640 1 -- 172.21.15.106:0/813782519 wait complete. 2023-11-24T08:08:55.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.696+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f98e404e840 con 0x7f98e4157a20 2023-11-24T08:08:55.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.697+0000 7f98daffd640 1 -- 172.21.15.106:0/1311596373 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98d4058670 con 0x7f98e4157a20 2023-11-24T08:08:55.698 INFO:teuthology.orchestra.run.smithi106.stdout:60129542148 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f98c806ae70 msgr2=0x7f98c806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f98c806ae70 0x7f98c806d330 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f98e0033000 tx=0x7f98e0009030 comp rx=0 tx=0).stop 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 msgr2=0x7f98e40cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 0x7f98e40cc2e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98d4011ca0 tx=0x7f98d400f610 comp rx=0 tx=0).stop 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 shutdown_connections 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f98e40cc820 0x7f98e4159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e4157a20 0x7f98e40cc2e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f98c806ae70 0x7f98c806d330 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 --2- 172.21.15.106:0/1311596373 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f98d00039a0 0x7f98e40c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:55.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 >> 172.21.15.106:0/1311596373 conn(0x7f98e4061310 msgr2=0x7f98e415f510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:55.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 shutdown_connections 2023-11-24T08:08:55.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:55.700+0000 7f98eb2d1640 1 -- 172.21.15.106:0/1311596373 wait complete. 2023-11-24T08:08:55.706 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.4 2023-11-24T08:08:55.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2023-11-24T08:08:55.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.6 2023-11-24T08:08:55.717 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.2 2023-11-24T08:08:56.674 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-24T08:08:56.684 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-24T08:08:56.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-24T08:08:56.691 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-24T08:08:56.707 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-24T08:08:56.712 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-24T08:08:56.716 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-24T08:08:56.717 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-24T08:08:56.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.968+0000 7ff8ee3e2640 1 Processor -- start 2023-11-24T08:08:56.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.968+0000 7ff8ee3e2640 1 -- start start 2023-11-24T08:08:56.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.970+0000 7ff8ee3e2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff8e00dcf40 0x7ff8e00df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.970+0000 7ff8ee3e2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff8e00531e0 0x7ff8e004fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.971+0000 7ff8ee3e2640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e0005df0 con 0x7ff8e004ffc0 2023-11-24T08:08:56.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.971+0000 7ff8edbe1640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7ff8e004ffc0 0x7ff8e00e2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59872/0 (socket says 172.21.15.106:59872) 2023-11-24T08:08:56.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.971+0000 7ff8edbe1640 1 -- 172.21.15.106:0/3236587959 learned_addr learned my addr 172.21.15.106:0/3236587959 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:56.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.971+0000 7ff8ee3e2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e0053670 con 0x7ff8e00531e0 2023-11-24T08:08:56.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.972+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/3236587959 --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e004f790 con 0x7ff8e00dcf40 2023-11-24T08:08:56.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.972+0000 7ff8ed3e0640 1 --2- 172.21.15.106:0/3236587959 >> v2:172.21.15.156:3300/0 conn(0x7ff8e00dcf40 0x7ff8e00df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.973+0000 7ff8ecbdf640 1 --2- 172.21.15.106:0/3236587959 >> v2:172.21.15.106:3300/0 conn(0x7ff8e00531e0 0x7ff8e004fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.974+0000 7ff8ecbdf640 1 -- 172.21.15.106:0/3236587959 >> v2:172.21.15.156:3300/0 conn(0x7ff8e00dcf40 msgr2=0x7ff8e00df770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:56.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.974+0000 7ff8ecbdf640 1 --2- 172.21.15.106:0/3236587959 >> v2:172.21.15.156:3300/0 conn(0x7ff8e00dcf40 0x7ff8e00df770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.974+0000 7ff8ecbdf640 1 -- 172.21.15.106:0/3236587959 >> v1:172.21.15.156:6789/0 conn(0x7ff8e004ffc0 legacy=0x7ff8e00e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.974+0000 7ff8ecbdf640 1 -- 172.21.15.106:0/3236587959 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00503d0 con 0x7ff8e00531e0 2023-11-24T08:08:56.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.975+0000 7ff8ecbdf640 1 --2- 172.21.15.106:0/3236587959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00531e0 0x7ff8e004fa80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8050140 tx=0x7ff8e80a6030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e7622012196667 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:56.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.976+0000 7ff8dffff640 1 -- 172.21.15.106:0/3236587959 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80a6e20 con 0x7ff8e00531e0 2023-11-24T08:08:56.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.976+0000 7ff8dffff640 1 -- 172.21.15.106:0/3236587959 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e80ae440 con 0x7ff8e00531e0 2023-11-24T08:08:56.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.976+0000 7ff8dffff640 1 -- 172.21.15.106:0/3236587959 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e80ae830 con 0x7ff8e00531e0 2023-11-24T08:08:56.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.977+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/3236587959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00531e0 msgr2=0x7ff8e004fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.977+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/3236587959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00531e0 0x7ff8e004fa80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8050140 tx=0x7ff8e80a6030 comp rx=0 tx=0).stop 2023-11-24T08:08:56.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.978+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/3236587959 shutdown_connections 2023-11-24T08:08:56.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.978+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/3236587959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00531e0 0x7ff8e004fa80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.978+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/3236587959 >> v2:172.21.15.156:3300/0 conn(0x7ff8e00dcf40 0x7ff8e00df770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.978+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/3236587959 >> 172.21.15.106:0/3236587959 conn(0x7ff8e0054a20 msgr2=0x7ff8e0054e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:56.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.979+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/3236587959 shutdown_connections 2023-11-24T08:08:56.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.979+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/3236587959 wait complete. 2023-11-24T08:08:56.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.982+0000 7ff8ee3e2640 1 Processor -- start 2023-11-24T08:08:56.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.982+0000 7ff8ee3e2640 1 -- start start 2023-11-24T08:08:56.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.985+0000 7ff8ee3e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 0x7ff8e004b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.985+0000 7ff8ee3e2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 0x7ff8e00d4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.986+0000 7ff8ee3e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 0x7ff8e004e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.986+0000 7ff8ed3e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 0x7ff8e004b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.986+0000 7ff8ed3e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 0x7ff8e004b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39704/0 (socket says 172.21.15.106:39704) 2023-11-24T08:08:56.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.986+0000 7ff8ecbdf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 0x7ff8e00d4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.986+0000 7ff8ecbdf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 0x7ff8e00d4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48424/0 (socket says 172.21.15.106:48424) 2023-11-24T08:08:56.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.987+0000 7ff8edbe1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 0x7ff8e004e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:56.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.987+0000 7ff8edbe1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 0x7ff8e004e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34376/0 (socket says 172.21.15.106:34376) 2023-11-24T08:08:56.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.987+0000 7ff8ed3e0640 1 -- 172.21.15.106:0/4282695468 learned_addr learned my addr 172.21.15.106:0/4282695468 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:56.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.988+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e004ce80 con 0x7ff8e00dcf40 2023-11-24T08:08:56.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.988+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e004d0d0 con 0x7ff8e004ffc0 2023-11-24T08:08:56.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.988+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e004d3b0 con 0x7ff8e00531e0 2023-11-24T08:08:56.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.989+0000 7ff8edbe1640 1 -- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 msgr2=0x7ff8e00d4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.989+0000 7ff8edbe1640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 0x7ff8e00d4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.989+0000 7ff8edbe1640 1 -- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 msgr2=0x7ff8e004b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:56.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.989+0000 7ff8edbe1640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 0x7ff8e004b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:56.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.990+0000 7ff8edbe1640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e804f730 con 0x7ff8e00dcf40 2023-11-24T08:08:56.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.990+0000 7ff8ecbdf640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 0x7ff8e00d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:56.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.991+0000 7ff8edbe1640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 0x7ff8e004e170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c780 tx=0x7ff8e400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:56.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.991+0000 7ff8ed3e0640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 0x7ff8e004b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:56.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.992+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e40072d0 con 0x7ff8e00dcf40 2023-11-24T08:08:56.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.992+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e400ce90 con 0x7ff8e00dcf40 2023-11-24T08:08:56.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.992+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e4007670 con 0x7ff8e00dcf40 2023-11-24T08:08:56.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.993+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e004d690 con 0x7ff8e00dcf40 2023-11-24T08:08:56.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.997+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e004dc50 con 0x7ff8e00dcf40 2023-11-24T08:08:57.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.997+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff8e40047e0 con 0x7ff8e00dcf40 2023-11-24T08:08:57.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:56.999+0000 7ff8ddffb640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8cc06ae90 0x7ff8cc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.000+0000 7ff8ed3e0640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8cc06ae90 0x7ff8cc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.001+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8e4057dd0 con 0x7ff8e00dcf40 2023-11-24T08:08:57.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.002+0000 7ff8ed3e0640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8cc06ae90 0x7ff8cc06d350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8006fd0 tx=0x7ff8d8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.003+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8e0008930 con 0x7ff8e00dcf40 2023-11-24T08:08:57.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.013+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff8e4018070 con 0x7ff8e00dcf40 2023-11-24T08:08:57.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.015+0000 7f2b37fff640 1 Processor -- start 2023-11-24T08:08:57.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.016+0000 7f2b37fff640 1 -- start start 2023-11-24T08:08:57.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.017+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b300e3230 0x7f2b3004f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.017+0000 7f2b37fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2b3004fd80 0x7f2b30052c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.017+0000 7f2b37fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b300501e0 con 0x7f2b30053180 2023-11-24T08:08:57.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.017+0000 7f2b37fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b300e3610 con 0x7f2b3004fd80 2023-11-24T08:08:57.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.017+0000 7f2b37fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b30053900 con 0x7f2b300e3230 2023-11-24T08:08:57.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.018+0000 7f2b367fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2b3004fd80 0x7f2b30052c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.019+0000 7f2b377fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2b30053180 0x7f2b30005e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32830/0 (socket says 172.21.15.106:32830) 2023-11-24T08:08:57.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.019+0000 7f2b377fe640 1 -- 172.21.15.106:0/138160247 learned_addr learned my addr 172.21.15.106:0/138160247 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:57.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.019+0000 7f2b36ffd640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b300e3230 0x7f2b3004f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.021+0000 7f2b367fc640 1 -- 172.21.15.106:0/138160247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b300e3230 msgr2=0x7f2b3004f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.021+0000 7f2b35ffb640 1 -- 172.21.15.106:0/138160247 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 937563577 0 0) 0x7f2b300501e0 con 0x7f2b30053180 2023-11-24T08:08:57.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.021+0000 7f2b367fc640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b300e3230 0x7f2b3004f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.021+0000 7f2b367fc640 1 -- 172.21.15.106:0/138160247 >> v1:172.21.15.106:6789/0 conn(0x7f2b30053180 legacy=0x7f2b30005e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.021+0000 7f2b367fc640 1 -- 172.21.15.106:0/138160247 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30053be0 con 0x7f2b3004fd80 2023-11-24T08:08:57.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.022+0000 7f2b36ffd640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b300e3230 0x7f2b3004f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:57.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.023+0000 7f2b367fc640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b3004fd80 0x7f2b30052c40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0097a0 tx=0x7f2b2c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24af812c69fe082 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.023+0000 7f2b35ffb640 1 -- 172.21.15.106:0/138160247 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2c030610 con 0x7f2b3004fd80 2023-11-24T08:08:57.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.024+0000 7f2b35ffb640 1 -- 172.21.15.106:0/138160247 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2c030770 con 0x7f2b3004fd80 2023-11-24T08:08:57.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.024+0000 7f2b37fff640 1 -- 172.21.15.106:0/138160247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b3004fd80 msgr2=0x7f2b30052c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.024+0000 7f2b37fff640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b3004fd80 0x7f2b30052c40 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0097a0 tx=0x7f2b2c02b960 comp rx=0 tx=0).stop 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.025+0000 7f2b37fff640 1 -- 172.21.15.106:0/138160247 shutdown_connections 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.025+0000 7f2b37fff640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b3004fd80 0x7f2b30052c40 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.025+0000 7f2b37fff640 1 --2- 172.21.15.106:0/138160247 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b300e3230 0x7f2b3004f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.025+0000 7f2b37fff640 1 -- 172.21.15.106:0/138160247 >> 172.21.15.106:0/138160247 conn(0x7f2b30009f80 msgr2=0x7f2b3000a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.025+0000 7f2b37fff640 1 -- 172.21.15.106:0/138160247 shutdown_connections 2023-11-24T08:08:57.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.025+0000 7f2b37fff640 1 -- 172.21.15.106:0/138160247 wait complete. 2023-11-24T08:08:57.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.026+0000 7f79a9272640 1 Processor -- start 2023-11-24T08:08:57.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.027+0000 7f79a9272640 1 -- start start 2023-11-24T08:08:57.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.029+0000 7f2b37fff640 1 Processor -- start 2023-11-24T08:08:57.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.029+0000 7f2b37fff640 1 -- start start 2023-11-24T08:08:57.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.029+0000 7f79a9272640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f79a40b1ec0 0x7f79a40b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.030+0000 7f79a9272640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a40cbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.030+0000 7f79a9272640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a40ccd90 con 0x7f79a406d810 2023-11-24T08:08:57.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.030+0000 7f79a9272640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a41403d0 con 0x7f79a40b1ec0 2023-11-24T08:08:57.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.030+0000 7f79a9272640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a40b27e0 con 0x7f79a40b3f30 2023-11-24T08:08:57.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a40cbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f79a37fe640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a40cbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48446/0 (socket says 172.21.15.106:48446) 2023-11-24T08:08:57.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3004fd80 0x7f2b3004c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b30053180 0x7f2b300e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f2b37fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 0x7f2b300e5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f2b37fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b300d0e40 con 0x7f2b300e3230 2023-11-24T08:08:57.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f79a37fe640 1 -- 172.21.15.106:0/246662870 learned_addr learned my addr 172.21.15.106:0/246662870 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f2b37fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3004a400 con 0x7f2b3004fd80 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.031+0000 7f2b37fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3004a6e0 con 0x7f2b30053180 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 0x7f2b300e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 0x7f2b300e5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34418/0 (socket says 172.21.15.106:34418) 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 -- 172.21.15.106:0/270821719 learned_addr learned my addr 172.21.15.106:0/270821719 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 -- 172.21.15.106:0/270821719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b30053180 msgr2=0x7f2b300e4ff0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b30053180 0x7f2b300e4ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 -- 172.21.15.106:0/270821719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3004fd80 msgr2=0x7f2b3004c3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3004fd80 0x7f2b3004c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f2b377fe640 1 -- 172.21.15.106:0/270821719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c009400 con 0x7f2b300e3230 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.033+0000 7f2b377fe640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 0x7f2b300e5530 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b38079c50 tx=0x7f2b38067f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.033+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380721c0 con 0x7f2b300e3230 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f79a37fe640 1 -- 172.21.15.106:0/246662870 >> v2:172.21.15.106:3300/0 conn(0x7f79a40b1ec0 msgr2=0x7f79a40b22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f79a37fe640 1 --2- 172.21.15.106:0/246662870 >> v2:172.21.15.106:3300/0 conn(0x7f79a40b1ec0 0x7f79a40b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f79a37fe640 1 -- 172.21.15.106:0/246662870 >> v1:172.21.15.106:6789/0 conn(0x7f79a406d810 legacy=0x7f79a406dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f79a37fe640 1 -- 172.21.15.106:0/246662870 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a40cc500 con 0x7f79a40b3f30 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.033+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b38072380 con 0x7f2b300e3230 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.033+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b380727f0 con 0x7f2b300e3230 2023-11-24T08:08:57.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.033+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3004a9c0 con 0x7f2b300e3230 2023-11-24T08:08:57.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.033+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b30048620 con 0x7f2b300e3230 2023-11-24T08:08:57.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f28bf0e0640 1 Processor -- start 2023-11-24T08:08:57.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.032+0000 7f28bf0e0640 1 -- start start 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.034+0000 7f79a37fe640 1 --2- 172.21.15.106:0/246662870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a40cbc00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f799c009e50 tx=0x7f799c02bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=538724f34cfaa52a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.035+0000 7f79a2ffd640 1 -- 172.21.15.106:0/246662870 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c02ca70 con 0x7f79a40b3f30 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.035+0000 7f79a2ffd640 1 -- 172.21.15.106:0/246662870 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f799c02cbd0 con 0x7f79a40b3f30 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.035+0000 7f79a2ffd640 1 -- 172.21.15.106:0/246662870 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f799c0343f0 con 0x7f79a40b3f30 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 -- 172.21.15.106:0/246662870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 msgr2=0x7f79a40cbc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 --2- 172.21.15.106:0/246662870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a40cbc00 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f799c009e50 tx=0x7f799c02bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 -- 172.21.15.106:0/246662870 shutdown_connections 2023-11-24T08:08:57.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b30048840 con 0x7f2b300e3230 2023-11-24T08:08:57.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 --2- 172.21.15.106:0/246662870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a40cbc00 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 --2- 172.21.15.106:0/246662870 >> v2:172.21.15.106:3300/0 conn(0x7f79a40b1ec0 0x7f79a40b22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 -- 172.21.15.106:0/246662870 >> 172.21.15.106:0/246662870 conn(0x7f79a40b0bc0 msgr2=0x7f79a40b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 -- 172.21.15.106:0/246662870 shutdown_connections 2023-11-24T08:08:57.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7f79a9272640 1 -- 172.21.15.106:0/246662870 wait complete. 2023-11-24T08:08:57.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.036+0000 7fc1f64e0640 1 Processor -- start 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7f28bf0e0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7f28bf0e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b80d1ae0 con 0x7f28b80d3160 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7f28bf0e0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28b80d1dc0 con 0x7f28b80d0700 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7f28bf0e0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80d18f0 con 0x7f28b80d0320 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7f28bce7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.038+0000 7f28bce7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b806bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48466/0 (socket says 172.21.15.106:48466) 2023-11-24T08:08:57.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.038+0000 7f28bce7c640 1 -- 172.21.15.106:0/2270858273 learned_addr learned my addr 172.21.15.106:0/2270858273 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b38074940 con 0x7f2b300e3230 2023-11-24T08:08:57.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.037+0000 7fc1f64e0640 1 -- start start 2023-11-24T08:08:57.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.038+0000 7f2f6c0d3640 1 Processor -- start 2023-11-24T08:08:57.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7f79a9272640 1 Processor -- start 2023-11-24T08:08:57.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7fc1f64e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1f00b56f0 0x7fc1f00ce3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7fc1f64e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00b3160 con 0x7fc1f00cc640 2023-11-24T08:08:57.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7fc1f64e0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1f00cea00 con 0x7fc1f00cc260 2023-11-24T08:08:57.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7fc1f64e0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1f00cece0 con 0x7fc1f00b56f0 2023-11-24T08:08:57.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7f79a9272640 1 -- start start 2023-11-24T08:08:57.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7fc1effff640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7fc1f00cc260 0x7fc1f00b4fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:59886/0 (socket says 172.21.15.106:59886) 2023-11-24T08:08:57.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7fc1effff640 1 -- 172.21.15.106:0/2357671513 learned_addr learned my addr 172.21.15.106:0/2357671513 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:57.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7f2b1f7fe640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b2406aec0 0x7f2b2406d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.040+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3749970832 0 0) 0x7fc1f00cea00 con 0x7fc1f00cc260 2023-11-24T08:08:57.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.040+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b381031b0 con 0x7f2b300e3230 2023-11-24T08:08:57.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.040+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1d8003520 con 0x7fc1f00cc260 2023-11-24T08:08:57.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.040+0000 7f2b36ffd640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b2406aec0 0x7f2b2406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.040+0000 7fc1f4a7d640 1 --2- 172.21.15.106:0/2357671513 >> v2:172.21.15.106:3300/0 conn(0x7fc1f00b56f0 0x7fc1f00ce3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.039+0000 7f2f6c0d3640 1 -- start start 2023-11-24T08:08:57.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 <== mon.1 v1:172.21.15.156:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 249003458 0 0) 0x7fc1d8003520 con 0x7fc1f00cc260 2023-11-24T08:08:57.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 >> v2:172.21.15.106:3300/0 conn(0x7fc1f00b56f0 msgr2=0x7fc1f00ce3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7f79a9272640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a406d810 0x7f79a414c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.038+0000 7f28bce7c640 1 -- 172.21.15.106:0/2270858273 >> v1:172.21.15.156:6789/0 conn(0x7f28b80d0700 legacy=0x7f28b80d11e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-24T08:08:57.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.038+0000 7f28bce7c640 1 -- 172.21.15.106:0/2270858273 >> v1:172.21.15.106:6789/0 conn(0x7f28b80d3160 legacy=0x7f28b80d35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7f79a9272640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a40b1ec0 0x7f79a41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.038+0000 7f28bce7c640 1 -- 172.21.15.106:0/2270858273 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b80d20a0 con 0x7f28b80d0320 2023-11-24T08:08:57.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7f2b36ffd640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b2406aec0 0x7f2b2406d380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800b810 tx=0x7f2b28008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f79a9272640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a4143b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f28bce7c640 1 --2- 172.21.15.106:0/2270858273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b806bcb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009e80 tx=0x7f28b002beb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f86b8e4ed96a44 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7fc1eeffd640 1 --2- 172.21.15.106:0/2357671513 >> v2:172.21.15.106:3300/0 conn(0x7fc1f00b56f0 0x7fc1f00ce3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 >> v1:172.21.15.106:6789/0 conn(0x7fc1f00cc640 legacy=0x7fc1f00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 --> v1:172.21.15.156:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00b33d0 con 0x7fc1f00cc260 2023-11-24T08:08:57.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f2f6c0d3640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2f641609e0 0x7f2f6406bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f2f6c0d3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f640ce010 0x7f2f640ce4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f2f6c0d3640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6406c1f0 con 0x7f2f640d31a0 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.041+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 <== mon.1 v1:172.21.15.156:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc1e0001670 con 0x7fc1f00cc260 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 >> v1:172.21.15.156:6789/0 conn(0x7fc1f00cc260 legacy=0x7fc1f00b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1d8003970 0x7fc1d8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80047c0 0x7fc1d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8005180 0x7fc1d8014580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f00cece0 con 0x7fc1d8005180 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8009b80 con 0x7fc1d80047c0 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f2f6c0d3640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f640d1070 con 0x7f2f641609e0 2023-11-24T08:08:57.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f2f6c0d3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f640cedc0 con 0x7f2f640ce010 2023-11-24T08:08:57.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d8014ac0 con 0x7fc1d8003970 2023-11-24T08:08:57.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f9a1d2b4640 1 Processor -- start 2023-11-24T08:08:57.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7fc1f4a7d640 1 --2- 172.21.15.106:0/2357671513 >> v2:172.21.15.106:3300/0 conn(0x7fc1f00b56f0 0x7fc1f00ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:57.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f28b77fe640 1 -- 172.21.15.106:0/2270858273 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b002ca50 con 0x7f28b80d0320 2023-11-24T08:08:57.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f28b77fe640 1 -- 172.21.15.106:0/2270858273 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b002cbb0 con 0x7f28b80d0320 2023-11-24T08:08:57.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f9a1d2b4640 1 -- start start 2023-11-24T08:08:57.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.042+0000 7f2f69e6f640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2f641609e0 0x7f2f6406bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f28b77fe640 1 -- 172.21.15.106:0/2270858273 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b002ca50 con 0x7f28b80d0320 2023-11-24T08:08:57.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1f4a7d640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8005180 0x7fc1d8014580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f2f69e6f640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2f641609e0 0x7f2f6406bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39718/0 (socket says 172.21.15.106:39718) 2023-11-24T08:08:57.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f2f69e6f640 1 -- 172.21.15.106:0/1769808886 learned_addr learned my addr 172.21.15.106:0/1769808886 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f2f6a670640 1 --2- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f640ce010 0x7f2f640ce4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1ef7fe640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80047c0 0x7fc1d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1f4a7d640 1 -- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1d8003970 msgr2=0x7fc1d8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1f4a7d640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1d8003970 0x7fc1d8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1f4a7d640 1 -- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80047c0 msgr2=0x7fc1d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1f4a7d640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80047c0 0x7fc1d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1f4a7d640 1 -- 172.21.15.106:0/2357671513 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f00b33d0 con 0x7fc1d8005180 2023-11-24T08:08:57.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7fc1ef7fe640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80047c0 0x7fc1d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:57.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f28bf0e0640 1 -- 172.21.15.106:0/2270858273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 msgr2=0x7f28b806bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/2270858273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b806bcb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f28b0009e80 tx=0x7f28b002beb0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f2f69e6f640 1 -- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f640ce010 msgr2=0x7f2f640ce4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f2f69e6f640 1 --2- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f640ce010 0x7f2f640ce4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.043+0000 7f2f69e6f640 1 -- 172.21.15.106:0/1769808886 >> v1:172.21.15.156:6789/0 conn(0x7f2f640d31a0 legacy=0x7f2f640d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f2f69e6f640 1 -- 172.21.15.106:0/1769808886 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f640cf0a0 con 0x7f2f641609e0 2023-11-24T08:08:57.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a9272640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a4136270 con 0x7f79a406d810 2023-11-24T08:08:57.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a9272640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a413d970 con 0x7f79a40b1ec0 2023-11-24T08:08:57.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a9272640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a413dbc0 con 0x7f79a40b3f30 2023-11-24T08:08:57.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a8a71640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a4143b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a406d810 0x7f79a414c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a8a71640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a4143b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48492/0 (socket says 172.21.15.106:48492) 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f79a8a71640 1 -- 172.21.15.106:0/3490635053 learned_addr learned my addr 172.21.15.106:0/3490635053 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f9a1d2b4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a1815d740 0x7f9a180b1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f9a1d2b4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a180b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.044+0000 7f9a1d2b4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a180ae8d0 0x7f9a180aed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a1d2b4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180b1610 con 0x7f9a180ae8d0 2023-11-24T08:08:57.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a1d2b4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180acbd0 con 0x7f9a1815d740 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a1d2b4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180abfc0 con 0x7f9a180b0c70 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a16ffd640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a1815d740 0x7f9a180b1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a16ffd640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a1815d740 0x7f9a180b1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39736/0 (socket says 172.21.15.106:39736) 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a16ffd640 1 -- 172.21.15.106:0/476763109 learned_addr learned my addr 172.21.15.106:0/476763109 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f79a37fe640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a40b1ec0 0x7f79a41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f79a3fff640 1 -- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 msgr2=0x7f79a4143b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f79a3fff640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a4143b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f9a167fc640 1 --2- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a180b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f79a3fff640 1 -- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a40b1ec0 msgr2=0x7f79a41435c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f79a3fff640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a40b1ec0 0x7f79a41435c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.045+0000 7f79a3fff640 1 -- 172.21.15.106:0/3490635053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c009b30 con 0x7f79a406d810 2023-11-24T08:08:57.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f28bf0e0640 1 -- 172.21.15.106:0/2270858273 shutdown_connections 2023-11-24T08:08:57.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f2f69e6f640 1 --2- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f641609e0 0x7f2f6406bcb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009870 tx=0x7f2f6002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ca9f402a0ff2575 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/2270858273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b806bcb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f28bf0e0640 1 -- 172.21.15.106:0/2270858273 >> 172.21.15.106:0/2270858273 conn(0x7f28b808af60 msgr2=0x7f28b809c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f79a3fff640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a406d810 0x7f79a414c4e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f799800b3f0 tx=0x7f799800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a16ffd640 1 -- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 msgr2=0x7f9a180b10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a16ffd640 1 --2- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a180b10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a16ffd640 1 -- 172.21.15.106:0/476763109 >> v2:172.21.15.106:3300/0 conn(0x7f9a180ae8d0 msgr2=0x7f9a180aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a177fe640 1 --2- 172.21.15.106:0/476763109 >> v2:172.21.15.106:3300/0 conn(0x7f9a180ae8d0 0x7f9a180aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a16ffd640 1 --2- 172.21.15.106:0/476763109 >> v2:172.21.15.106:3300/0 conn(0x7f9a180ae8d0 0x7f9a180aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a16ffd640 1 -- 172.21.15.106:0/476763109 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180ac2a0 con 0x7f9a1815d740 2023-11-24T08:08:57.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7fc1f4a7d640 1 -- 172.21.15.106:0/2357671513 reap_dead start 2023-11-24T08:08:57.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f2f68e6d640 1 -- 172.21.15.106:0/1769808886 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60030840 con 0x7f2f641609e0 2023-11-24T08:08:57.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f2f68e6d640 1 -- 172.21.15.106:0/1769808886 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f600309a0 con 0x7f2f641609e0 2023-11-24T08:08:57.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f2f68e6d640 1 -- 172.21.15.106:0/1769808886 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f60030d30 con 0x7f2f641609e0 2023-11-24T08:08:57.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.046+0000 7f9a177fe640 1 --2- 172.21.15.106:0/476763109 >> v2:172.21.15.106:3300/0 conn(0x7f9a180ae8d0 0x7f9a180aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:57.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.047+0000 7f28bf0e0640 1 -- 172.21.15.106:0/2270858273 shutdown_connections 2023-11-24T08:08:57.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.047+0000 7fc1f4a7d640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8005180 0x7fc1d8014580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80073b0 tx=0x7fc1d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf57d32ea0abae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.047+0000 7f28bf0e0640 1 -- 172.21.15.106:0/2270858273 wait complete. 2023-11-24T08:08:57.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.047+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f641609e0 msgr2=0x7f2f6406bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.047+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f641609e0 0x7f2f6406bcb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2f60009870 tx=0x7f2f6002b980 comp rx=0 tx=0).stop 2023-11-24T08:08:57.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f9a16ffd640 1 --2- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a180b1fe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9a1806ff20 tx=0x7f9a1002bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca74ef805c77362 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/1769808886 shutdown_connections 2023-11-24T08:08:57.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f640ce010 0x7f2f640ce4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/1769808886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f641609e0 0x7f2f6406bcb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/1769808886 >> 172.21.15.106:0/1769808886 conn(0x7f2f6408af60 msgr2=0x7f2f6409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e802cd90 con 0x7fc1d8005180 2023-11-24T08:08:57.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e8034440 con 0x7fc1d8005180 2023-11-24T08:08:57.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7fc1eeffd640 1 -- 172.21.15.106:0/2357671513 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e80347f0 con 0x7fc1d8005180 2023-11-24T08:08:57.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/1769808886 shutdown_connections 2023-11-24T08:08:57.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f9a15ffb640 1 -- 172.21.15.106:0/476763109 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1002ca80 con 0x7f9a1815d740 2023-11-24T08:08:57.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f9a15ffb640 1 -- 172.21.15.106:0/476763109 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1002cbe0 con 0x7f9a1815d740 2023-11-24T08:08:57.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f9a15ffb640 1 -- 172.21.15.106:0/476763109 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100343f0 con 0x7f9a1815d740 2023-11-24T08:08:57.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f0938d83640 1 Processor -- start 2023-11-24T08:08:57.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998011830 con 0x7f79a406d810 2023-11-24T08:08:57.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.048+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7998011990 con 0x7f79a406d810 2023-11-24T08:08:57.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7998011d40 con 0x7f79a406d810 2023-11-24T08:08:57.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a414f890 con 0x7f79a406d810 2023-11-24T08:08:57.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a414fe50 con 0x7f79a406d810 2023-11-24T08:08:57.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f0938d83640 1 -- start start 2023-11-24T08:08:57.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f28bf0e0640 1 Processor -- start 2023-11-24T08:08:57.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 msgr2=0x7f9a180b1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f28bf0e0640 1 -- start start 2023-11-24T08:08:57.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.049+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a180b1fe0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9a1806ff20 tx=0x7f9a1002bee0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b380ce360 con 0x7f2b300e3230 2023-11-24T08:08:57.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7fc1f64e0640 1 -- 172.21.15.106:0/2357671513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8005180 msgr2=0x7fc1d8014580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8005180 0x7fc1d8014580 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc1e80073b0 tx=0x7fc1d80057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/476763109 shutdown_connections 2023-11-24T08:08:57.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/476763109 >> v2:172.21.15.106:3300/0 conn(0x7f9a180ae8d0 0x7f9a180aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a180b10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/476763109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a180b1fe0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/476763109 >> 172.21.15.106:0/476763109 conn(0x7f9a180b4500 msgr2=0x7f9a180b4910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/1769808886 wait complete. 2023-11-24T08:08:57.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/476763109 shutdown_connections 2023-11-24T08:08:57.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/476763109 wait complete. 2023-11-24T08:08:57.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a404e840 con 0x7f79a406d810 2023-11-24T08:08:57.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7fc1f64e0640 1 -- 172.21.15.106:0/2357671513 shutdown_connections 2023-11-24T08:08:57.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/2357671513 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1d8005180 0x7fc1d8014580 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.050+0000 7fc1f64e0640 1 -- 172.21.15.106:0/2357671513 >> 172.21.15.106:0/2357671513 conn(0x7fc1f0071d40 msgr2=0x7fc1f006a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7fc1f64e0640 1 -- 172.21.15.106:0/2357671513 shutdown_connections 2023-11-24T08:08:57.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7fc1f64e0640 1 -- 172.21.15.106:0/2357671513 wait complete. 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7f0938d83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 0x7f09340cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7f0938d83640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09340cf2f0 0x7f093406ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7f0938d83640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09340ce0a0 con 0x7f093415da20 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7f0938d83640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09340d0d70 con 0x7f09340cf2f0 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.051+0000 7f0938d83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340d0a90 con 0x7f09340d0480 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 0x7f09340cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 0x7f09340cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48504/0 (socket says 172.21.15.106:48504) 2023-11-24T08:08:57.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 -- 172.21.15.106:0/1884638636 learned_addr learned my addr 172.21.15.106:0/1884638636 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 -- 172.21.15.106:0/1884638636 >> v2:172.21.15.106:3300/0 conn(0x7f09340cf2f0 msgr2=0x7f093406ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 --2- 172.21.15.106:0/1884638636 >> v2:172.21.15.106:3300/0 conn(0x7f09340cf2f0 0x7f093406ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 -- 172.21.15.106:0/1884638636 >> v1:172.21.15.106:6789/0 conn(0x7f093415da20 legacy=0x7f09340d1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.053+0000 7f2f6c0d3640 1 Processor -- start 2023-11-24T08:08:57.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.053+0000 7f9a1d2b4640 1 Processor -- start 2023-11-24T08:08:57.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.052+0000 7f0931d9b640 1 -- 172.21.15.106:0/1884638636 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340ce2d0 con 0x7f09340d0480 2023-11-24T08:08:57.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.053+0000 7f9a1d2b4640 1 -- start start 2023-11-24T08:08:57.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.053+0000 7fc1f64e0640 1 Processor -- start 2023-11-24T08:08:57.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.053+0000 7f2f6c0d3640 1 -- start start 2023-11-24T08:08:57.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.054+0000 7fc1f64e0640 1 -- start start 2023-11-24T08:08:57.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.054+0000 7f0931d9b640 1 --2- 172.21.15.106:0/1884638636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 0x7f09340cedb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0928009870 tx=0x7f092802b9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd8f28fc2d5e302f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.054+0000 7f28bf0e0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b815fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f9a1d2b4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 0x7f9a18153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bf0e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28b80d0700 0x7f28b80ca4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bf0e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28b80d3160 0x7f28b80cbdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bce7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b815fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bf0e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b815ffa0 con 0x7f28b80d0700 2023-11-24T08:08:57.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bf0e0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80cb930 con 0x7f28b80d3160 2023-11-24T08:08:57.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bf0e0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28b80cad20 con 0x7f28b80d0320 2023-11-24T08:08:57.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bce7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b815fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48508/0 (socket says 172.21.15.106:48508) 2023-11-24T08:08:57.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f28bce7c640 1 -- 172.21.15.106:0/1637683394 learned_addr learned my addr 172.21.15.106:0/1637683394 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7f9a1d2b4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a181564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f2f6c0d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f640ce010 0x7f2f6414d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f093159a640 1 -- 172.21.15.106:0/1884638636 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928030860 con 0x7f09340d0480 2023-11-24T08:08:57.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7fc1f64e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d8005180 0x7fc1f014f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7fc1f64e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1f01576b0 0x7fc1f014fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7fc1f64e0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1f01511a0 0x7fc1f0151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7fc1f64e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006a5a0 con 0x7fc1f01576b0 2023-11-24T08:08:57.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7fc1f64e0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006a840 con 0x7fc1d8005180 2023-11-24T08:08:57.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.055+0000 7fc1f64e0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f006ab20 con 0x7fc1f01511a0 2023-11-24T08:08:57.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f093159a640 1 -- 172.21.15.106:0/1884638636 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09280309c0 con 0x7f09340d0480 2023-11-24T08:08:57.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f093159a640 1 -- 172.21.15.106:0/1884638636 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928030d90 con 0x7f09340d0480 2023-11-24T08:08:57.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f2f6c0d3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f640d31a0 0x7f2f640b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f28bd67d640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28b80d3160 0x7f28b80cbdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f28b7fff640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28b80d0700 0x7f28b80ca4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f2f6c0d3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 0x7f2f640b7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f2f6c0d3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6415c620 con 0x7f2f640ce010 2023-11-24T08:08:57.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f2f6c0d3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6414dcd0 con 0x7f2f640d31a0 2023-11-24T08:08:57.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f2f6c0d3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6415ac50 con 0x7f2f641609e0 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a1d2b4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a18144490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a1d2b4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1815f110 con 0x7f9a180ae8d0 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a1d2b4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a181518e0 con 0x7f9a1815d740 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a1d2b4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1815faa0 con 0x7f9a180b0c70 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a16ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 0x7f9a18153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a16ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 0x7f9a18153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34490/0 (socket says 172.21.15.106:34490) 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7f9a16ffd640 1 -- 172.21.15.106:0/2213191343 learned_addr learned my addr 172.21.15.106:0/2213191343 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a167fc640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a181564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f0938d83640 1 -- 172.21.15.106:0/1884638636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 msgr2=0x7f09340cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f0938d83640 1 --2- 172.21.15.106:0/1884638636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 0x7f09340cedb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0928009870 tx=0x7f092802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a177fe640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a18144490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7fc1effff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d8005180 0x7fc1f014f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7fc1f4a7d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1f01511a0 0x7fc1f0151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7fc1f4a7d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1f01511a0 0x7fc1f0151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48510/0 (socket says 172.21.15.106:48510) 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7fc1ef7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1f01576b0 0x7fc1f014fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.056+0000 7fc1f4a7d640 1 -- 172.21.15.106:0/1930067962 learned_addr learned my addr 172.21.15.106:0/1930067962 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a16ffd640 1 -- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 msgr2=0x7f9a181564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a16ffd640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a181564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a16ffd640 1 -- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 msgr2=0x7f9a18144490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7fc1ef7fe640 1 -- 172.21.15.106:0/1930067962 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1f01511a0 msgr2=0x7fc1f0151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7fc1ef7fe640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1f01511a0 0x7fc1f0151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7fc1ef7fe640 1 -- 172.21.15.106:0/1930067962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d8005180 msgr2=0x7fc1f014f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7fc1ef7fe640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d8005180 0x7fc1f014f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a16ffd640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a18144490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f9a16ffd640 1 -- 172.21.15.106:0/2213191343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a10009b30 con 0x7f9a180ae8d0 2023-11-24T08:08:57.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7fc1ef7fe640 1 -- 172.21.15.106:0/1930067962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f006ae00 con 0x7fc1f01576b0 2023-11-24T08:08:57.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f28bce7c640 1 -- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28b80d3160 msgr2=0x7f28b80cbdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f28bce7c640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28b80d3160 0x7f28b80cbdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f28bce7c640 1 -- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28b80d0700 msgr2=0x7f28b80ca4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f0938d83640 1 -- 172.21.15.106:0/1884638636 shutdown_connections 2023-11-24T08:08:57.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f0938d83640 1 --2- 172.21.15.106:0/1884638636 >> v2:172.21.15.106:3300/0 conn(0x7f09340cf2f0 0x7f093406ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f0938d83640 1 --2- 172.21.15.106:0/1884638636 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09340d0480 0x7f09340cedb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f0938d83640 1 -- 172.21.15.106:0/1884638636 >> 172.21.15.106:0/1884638636 conn(0x7f093409c7e0 msgr2=0x7f093408af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f9a16ffd640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 0x7f9a18153750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000f80 tx=0x7f9a10004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f28bce7c640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28b80d0700 0x7f28b80ca4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f28bce7c640 1 -- 172.21.15.106:0/1637683394 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b0009b30 con 0x7f28b80d0320 2023-11-24T08:08:57.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7fc1ef7fe640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1f01576b0 0x7fc1f014fbb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400d540 tx=0x7fc1e400da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f799801a400 con 0x7f79a406d810 2023-11-24T08:08:57.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f0938d83640 1 -- 172.21.15.106:0/1884638636 shutdown_connections 2023-11-24T08:08:57.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f0938d83640 1 -- 172.21.15.106:0/1884638636 wait complete. 2023-11-24T08:08:57.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f2f69e6f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f640ce010 0x7f2f6414d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.057+0000 7f2f6a670640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 0x7f2f640b7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f2f6a670640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 0x7f2f640b7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48534/0 (socket says 172.21.15.106:48534) 2023-11-24T08:08:57.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f2f6a670640 1 -- 172.21.15.106:0/468160650 learned_addr learned my addr 172.21.15.106:0/468160650 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f2f6966e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f640d31a0 0x7f2f640b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.058+0000 7f28b7fff640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28b80d0700 0x7f28b80ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:08:57.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f6a670640 1 -- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f640d31a0 msgr2=0x7f2f640b75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f6a670640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f640d31a0 0x7f2f640b75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f6a670640 1 -- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f640ce010 msgr2=0x7f2f6414d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f6a670640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f640ce010 0x7f2f6414d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f6a670640 1 -- 172.21.15.106:0/468160650 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f600094d0 con 0x7f2f641609e0 2023-11-24T08:08:57.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f69e6f640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f640ce010 0x7f2f6414d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:08:57.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f28bce7c640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b815fa60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000f80 tx=0x7f28b0004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f79a0ff9640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f799006aec0 0x7f799006d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7998089c10 con 0x7f79a406d810 2023-11-24T08:08:57.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f79a37fe640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f799006aec0 0x7f799006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.059+0000 7f2f6a670640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 0x7f2f640b7b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00efc0 tx=0x7f2f5c00c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b0003e80 con 0x7f28b80d0320 2023-11-24T08:08:57.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28b0009510 con 0x7f28b80d0320 2023-11-24T08:08:57.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28b00048a0 con 0x7f28b80d0320 2023-11-24T08:08:57.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c014480 con 0x7f2f641609e0 2023-11-24T08:08:57.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5c00f040 con 0x7f2f641609e0 2023-11-24T08:08:57.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c014820 con 0x7f2f641609e0 2023-11-24T08:08:57.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f640b4010 con 0x7f2f641609e0 2023-11-24T08:08:57.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.060+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f640b45d0 con 0x7f2f641609e0 2023-11-24T08:08:57.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.061+0000 7f79a37fe640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f799006aec0 0x7f799006d380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f799c035000 tx=0x7f799c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.061+0000 7f0938d83640 1 Processor -- start 2023-11-24T08:08:57.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.061+0000 7f0938d83640 1 -- start start 2023-11-24T08:08:57.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.061+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28b80d0d00 con 0x7f28b80d0320 2023-11-24T08:08:57.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.061+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28b80cb5c0 con 0x7f28b80d0320 2023-11-24T08:08:57.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.062+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f64067470 con 0x7f2f641609e0 2023-11-24T08:08:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.062+0000 7f0938d83640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09340d0480 0x7f09340b7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.062+0000 7f0938d83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f093415da20 0x7f093415c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.062+0000 7f0938d83640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 0x7f093415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0938d83640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cc060 con 0x7f093415bc10 2023-11-24T08:08:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0938d83640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340cc340 con 0x7f09340d0480 2023-11-24T08:08:57.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0938d83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093415fac0 con 0x7f093415da20 2023-11-24T08:08:57.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28b80cb120 con 0x7f28b80d0320 2023-11-24T08:08:57.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 0x7f093415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0931d9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f093415da20 0x7f093415c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 0x7f093415c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34526/0 (socket says 172.21.15.106:34526) 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 -- 172.21.15.106:0/3192772800 learned_addr learned my addr 172.21.15.106:0/3192772800 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 -- 172.21.15.106:0/3192772800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f093415da20 msgr2=0x7f093415c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f093415da20 0x7f093415c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 -- 172.21.15.106:0/3192772800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09340d0480 msgr2=0x7f09340b7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.063+0000 7f0932d9d640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09340d0480 0x7f09340b7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.064+0000 7f0932d9d640 1 -- 172.21.15.106:0/3192772800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09280094d0 con 0x7f093415bc10 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.064+0000 7f0932d9d640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 0x7f093415c5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0924009fd0 tx=0x7f092400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.064+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4005480 con 0x7fc1f01576b0 2023-11-24T08:08:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f5c014980 con 0x7f2f641609e0 2023-11-24T08:08:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.064+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100079a0 con 0x7f9a180ae8d0 2023-11-24T08:08:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a10007b00 con 0x7f9a180ae8d0 2023-11-24T08:08:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a1004b420 con 0x7f9a180ae8d0 2023-11-24T08:08:57.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a181449d0 con 0x7f9a180ae8d0 2023-11-24T08:08:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a1815f5d0 con 0x7f9a180ae8d0 2023-11-24T08:08:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924011830 con 0x7f093415bc10 2023-11-24T08:08:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.066+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924011990 con 0x7f093415bc10 2023-11-24T08:08:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.066+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924011d00 con 0x7f093415bc10 2023-11-24T08:08:57.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.064+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f006b0e0 con 0x7fc1f01576b0 2023-11-24T08:08:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f0151d40 con 0x7fc1f01576b0 2023-11-24T08:08:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1e4003610 con 0x7fc1f01576b0 2023-11-24T08:08:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.065+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1e4003980 con 0x7fc1f01576b0 2023-11-24T08:08:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.066+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7998054d30 con 0x7f79a406d810 2023-11-24T08:08:57.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.068+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc1e401c460 con 0x7fc1f01576b0 2023-11-24T08:08:57.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.067+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99fc005250 con 0x7f9a180ae8d0 2023-11-24T08:08:57.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.066+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f093415fda0 con 0x7f093415bc10 2023-11-24T08:08:57.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.067+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340b77b0 con 0x7f093415bc10 2023-11-24T08:08:57.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.068+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f00cd5a0 con 0x7fc1f01576b0 2023-11-24T08:08:57.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.069+0000 7f2f567fc640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f4806af10 0x7f2f4806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.066+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f28b003e450 con 0x7f28b80d0320 2023-11-24T08:08:57.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.070+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9a10049070 con 0x7f9a180ae8d0 2023-11-24T08:08:57.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.067+0000 7f28b57fa640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f289806ae90 0x7f289806d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.067+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28b00ac780 con 0x7f28b80d0320 2023-11-24T08:08:57.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.070+0000 7f28b7fff640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f289806ae90 0x7f289806d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.069+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f5c08a3a0 con 0x7f2f641609e0 2023-11-24T08:08:57.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.069+0000 7fc1ecff9640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc1dc06ae70 0x7fc1dc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.071+0000 7f9a0f7fe640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f99f806af10 0x7f99f806d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.071+0000 7f2f69e6f640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f4806af10 0x7f2f4806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.071+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a100b3f60 con 0x7f9a180ae8d0 2023-11-24T08:08:57.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.069+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc1e4089410 con 0x7fc1f01576b0 2023-11-24T08:08:57.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.071+0000 7f28b7fff640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f289806ae90 0x7f289806d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006fd0 tx=0x7f28a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.069+0000 7fc1effff640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc1dc06ae70 0x7fc1dc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.070+0000 7fc1effff640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc1dc06ae70 0x7fc1dc06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0001050 tx=0x7fc1e00019b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.072+0000 7f9a167fc640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f99f806af10 0x7f99f806d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.072+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09340b7a90 con 0x7f093415bc10 2023-11-24T08:08:57.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.072+0000 7f2f69e6f640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f4806af10 0x7f2f4806d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f60036000 tx=0x7f2f60008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.073+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f092401b460 con 0x7f093415bc10 2023-11-24T08:08:57.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.073+0000 7f9a167fc640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f99f806af10 0x7f99f806d3d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a1815ee80 tx=0x7f9a08009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.074+0000 7f0922ffd640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f091406ae90 0x7f091406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.075+0000 7f093259c640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f091406ae90 0x7f091406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.075+0000 7f093259c640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f091406ae90 0x7f091406d350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09340d1a50 tx=0x7f092c007d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.076+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a100e0e90 con 0x7f9a180ae8d0 2023-11-24T08:08:57.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.078+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28b00778a0 con 0x7f28b80d0320 2023-11-24T08:08:57.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.082+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09240588f0 con 0x7f093415bc10 2023-11-24T08:08:57.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.082+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc1e4054440 con 0x7fc1f01576b0 2023-11-24T08:08:57.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.091+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f5c0554e0 con 0x7f2f641609e0 2023-11-24T08:08:57.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.091+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0924089e60 con 0x7f093415bc10 2023-11-24T08:08:57.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.416+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7ff8e004de40 con 0x7ff8e00dcf40 2023-11-24T08:08:57.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.417+0000 7ff8ddffb640 1 -- 172.21.15.106:0/4282695468 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8e40574a0 con 0x7ff8e00dcf40 2023-11-24T08:08:57.420 INFO:teuthology.orchestra.run.smithi106.stdout:60129542151 2023-11-24T08:08:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.420+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8cc06ae90 msgr2=0x7ff8cc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.421+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8cc06ae90 0x7ff8cc06d350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff8d8006fd0 tx=0x7ff8d8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:57.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.421+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 msgr2=0x7ff8e004e170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.421+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 0x7ff8e004e170 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff8e400c780 tx=0x7ff8e400cb80 comp rx=0 tx=0).stop 2023-11-24T08:08:57.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.422+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 shutdown_connections 2023-11-24T08:08:57.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.422+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff8e00dcf40 0x7ff8e004e170 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.422+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8cc06ae90 0x7ff8cc06d350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.422+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff8e00531e0 0x7ff8e00d4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.422+0000 7ff8ee3e2640 1 --2- 172.21.15.106:0/4282695468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff8e004ffc0 0x7ff8e004b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.422+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 >> 172.21.15.106:0/4282695468 conn(0x7ff8e0054a20 msgr2=0x7ff8e00dc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.423+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 shutdown_connections 2023-11-24T08:08:57.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.423+0000 7ff8ee3e2640 1 -- 172.21.15.106:0/4282695468 wait complete. 2023-11-24T08:08:57.452 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.7 2023-11-24T08:08:57.453 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.462+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f79a4168030 con 0x7f79a406d810 2023-11-24T08:08:57.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.463+0000 7f79a0ff9640 1 -- 172.21.15.106:0/3490635053 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7998015090 con 0x7f79a406d810 2023-11-24T08:08:57.465 INFO:teuthology.orchestra.run.smithi106.stdout:60129542149 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f799006aec0 msgr2=0x7f799006d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f799006aec0 0x7f799006d380 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f799c035000 tx=0x7f799c030000 comp rx=0 tx=0).stop 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a406d810 msgr2=0x7f79a414c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a406d810 0x7f79a414c4e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f799800b3f0 tx=0x7f799800b8b0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f2b30005360 con 0x7f2b300e3230 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 shutdown_connections 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f799006aec0 0x7f799006d380 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f79a40b3f30 0x7f79a4143b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79a40b1ec0 0x7f79a41435c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 --2- 172.21.15.106:0/3490635053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f79a406d810 0x7f79a414c4e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.467+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 >> 172.21.15.106:0/3490635053 conn(0x7f79a40b0bc0 msgr2=0x7f79a40b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.468+0000 7f2b1f7fe640 1 -- 172.21.15.106:0/270821719 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b380d1980 con 0x7f2b300e3230 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.468+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 shutdown_connections 2023-11-24T08:08:57.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.468+0000 7f79a9272640 1 -- 172.21.15.106:0/3490635053 wait complete. 2023-11-24T08:08:57.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.469+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fc1f006b2d0 con 0x7fc1f01576b0 2023-11-24T08:08:57.473 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:57.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.470+0000 7fc1ecff9640 1 -- 172.21.15.106:0/1930067962 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc1e4057a60 con 0x7fc1f01576b0 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.471+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b2406aec0 msgr2=0x7f2b2406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.471+0000 7f2b37fff640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b2406aec0 0x7f2b2406d380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b2800b810 tx=0x7f2b28008040 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.471+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 msgr2=0x7f2b300e5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.471+0000 7f2b37fff640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 0x7f2b300e5530 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2b38079c50 tx=0x7f2b38067f50 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.472+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 shutdown_connections 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.472+0000 7f2b37fff640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b300e3230 0x7f2b300e5530 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.472+0000 7f2b37fff640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b2406aec0 0x7f2b2406d380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.472+0000 7f2b37fff640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b30053180 0x7f2b300e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.472+0000 7f2b37fff640 1 --2- 172.21.15.106:0/270821719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b3004fd80 0x7f2b3004c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.472+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 >> 172.21.15.106:0/270821719 conn(0x7f2b30009f80 msgr2=0x7f2b300d8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 shutdown_connections 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc1dc06ae70 msgr2=0x7fc1dc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc1dc06ae70 0x7fc1dc06d330 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc1e0001050 tx=0x7fc1e00019b0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1f01576b0 msgr2=0x7fc1f014fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1f01576b0 0x7fc1f014fbb0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400d540 tx=0x7fc1e400da00 comp rx=0 tx=0).stop 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 shutdown_connections 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc1dc06ae70 0x7fc1dc06d330 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc1f01511a0 0x7fc1f0151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1f01576b0 0x7fc1f014fbb0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc1e400d540 tx=0x7fc1e400da00 comp rx=0 tx=0).stop 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 --2- 172.21.15.106:0/1930067962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d8005180 0x7fc1f014f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 >> 172.21.15.106:0/1930067962 conn(0x7fc1f0071d40 msgr2=0x7fc1f00b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.473+0000 7f2b37fff640 1 -- 172.21.15.106:0/270821719 wait complete. 2023-11-24T08:08:57.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.474+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 shutdown_connections 2023-11-24T08:08:57.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.474+0000 7fc1f64e0640 1 -- 172.21.15.106:0/1930067962 wait complete. 2023-11-24T08:08:57.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.485+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f99fc0057d0 con 0x7f9a180ae8d0 2023-11-24T08:08:57.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.486+0000 7f9a0f7fe640 1 -- 172.21.15.106:0/2213191343 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a10050090 con 0x7f9a180ae8d0 2023-11-24T08:08:57.489 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:57.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.488+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f99f806af10 msgr2=0x7f99f806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.488+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f99f806af10 0x7f99f806d3d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9a1815ee80 tx=0x7f9a08009190 comp rx=0 tx=0).stop 2023-11-24T08:08:57.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.488+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 msgr2=0x7f9a18153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.488+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 0x7f9a18153750 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9a10000f80 tx=0x7f9a10004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:57.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 shutdown_connections 2023-11-24T08:08:57.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a1815d740 0x7f9a18144490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f99f806af10 0x7f99f806d3d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a180b0c70 0x7f9a181564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 --2- 172.21.15.106:0/2213191343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a180ae8d0 0x7f9a18153750 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 >> 172.21.15.106:0/2213191343 conn(0x7f9a180b4500 msgr2=0x7f9a180b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 shutdown_connections 2023-11-24T08:08:57.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.489+0000 7f9a1d2b4640 1 -- 172.21.15.106:0/2213191343 wait complete. 2023-11-24T08:08:57.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.492+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f28b80cb120 con 0x7f28b80d0320 2023-11-24T08:08:57.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.493+0000 7f28b57fa640 1 -- 172.21.15.106:0/1637683394 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f28b007aec0 con 0x7f28b80d0320 2023-11-24T08:08:57.495 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:57.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.496+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f0934067b80 con 0x7f093415bc10 2023-11-24T08:08:57.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542148 got 60129542149 for osd.5 2023-11-24T08:08:57.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.496+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f289806ae90 msgr2=0x7f289806d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.496+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f289806ae90 0x7f289806d350 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f28a8006fd0 tx=0x7f28a8008040 comp rx=0 tx=0).stop 2023-11-24T08:08:57.500 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f0922ffd640 1 -- 172.21.15.106:0/3192772800 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0924011e60 con 0x7f093415bc10 2023-11-24T08:08:57.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.496+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 msgr2=0x7f28b815fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.496+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b815fa60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f28b0000f80 tx=0x7f28b0004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:57.501 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:57.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 shutdown_connections 2023-11-24T08:08:57.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28b80d3160 0x7f28b80cbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28b80d0700 0x7f28b80ca4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f289806ae90 0x7f289806d350 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 --2- 172.21.15.106:0/1637683394 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28b80d0320 0x7f28b815fa60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 >> 172.21.15.106:0/1637683394 conn(0x7f28b808af60 msgr2=0x7f28b80ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 shutdown_connections 2023-11-24T08:08:57.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.497+0000 7f28bf0e0640 1 -- 172.21.15.106:0/1637683394 wait complete. 2023-11-24T08:08:57.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f091406ae90 msgr2=0x7f091406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.503 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.1 2023-11-24T08:08:57.503 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f091406ae90 0x7f091406d350 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09340d1a50 tx=0x7f092c007d50 comp rx=0 tx=0).stop 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 msgr2=0x7f093415c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 0x7f093415c5d0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0924009fd0 tx=0x7f092400b8e0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 shutdown_connections 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f093415bc10 0x7f093415c5d0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f091406ae90 0x7f091406d350 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f09340d1a50 tx=0x7f092c007d50 comp rx=0 tx=0).stop 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f093415da20 0x7f093415c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 --2- 172.21.15.106:0/3192772800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09340d0480 0x7f09340b7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 >> 172.21.15.106:0/3192772800 conn(0x7f093409c7e0 msgr2=0x7f09340b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.505 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.4 2023-11-24T08:08:57.505 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 shutdown_connections 2023-11-24T08:08:57.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.500+0000 7f0938d83640 1 -- 172.21.15.106:0/3192772800 wait complete. 2023-11-24T08:08:57.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.502+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f2f6406b750 con 0x7f2f641609e0 2023-11-24T08:08:57.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.503+0000 7f2f567fc640 1 -- 172.21.15.106:0/468160650 <== mon.2 v2:172.21.15.156:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2f5c058b00 con 0x7f2f641609e0 2023-11-24T08:08:57.506 INFO:teuthology.orchestra.run.smithi106.stdout:60129542150 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f4806af10 msgr2=0x7f2f4806d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f4806af10 0x7f2f4806d3d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f60036000 tx=0x7f2f60008030 comp rx=0 tx=0).stop 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 msgr2=0x7f2f640b7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 0x7f2f640b7b00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c00efc0 tx=0x7f2f5c00c490 comp rx=0 tx=0).stop 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 shutdown_connections 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f4806af10 0x7f2f4806d3d0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2f60036000 tx=0x7f2f60008030 comp rx=0 tx=0).stop 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f641609e0 0x7f2f640b7b00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f640d31a0 0x7f2f640b75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 --2- 172.21.15.106:0/468160650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f640ce010 0x7f2f6414d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 >> 172.21.15.106:0/468160650 conn(0x7f2f6408af60 msgr2=0x7f2f640ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 shutdown_connections 2023-11-24T08:08:57.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.507+0000 7f2f6c0d3640 1 -- 172.21.15.106:0/468160650 wait complete. 2023-11-24T08:08:57.511 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.3 2023-11-24T08:08:57.511 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.522 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.2 2023-11-24T08:08:57.522 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.524 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.0 2023-11-24T08:08:57.524 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.529 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.6 2023-11-24T08:08:57.530 DEBUG:teuthology.parallel:result is None 2023-11-24T08:08:57.530 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T08:08:57.530 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:57.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.780+0000 7f8608800640 1 Processor -- start 2023-11-24T08:08:57.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.780+0000 7f8608800640 1 -- start start 2023-11-24T08:08:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.781+0000 7f8608800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 0x7f860006ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8608800640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f860006bfd0 0x7f8600161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8608800640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86000cdf30 con 0x7f8600161860 2023-11-24T08:08:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8608800640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8600161080 con 0x7f860006bfd0 2023-11-24T08:08:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8608800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8600161d50 con 0x7f8600160ca0 2023-11-24T08:08:57.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8606d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8600161860 0x7f86000ce9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32856/0 (socket says 172.21.15.106:32856) 2023-11-24T08:08:57.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8606d9d640 1 -- 172.21.15.106:0/3247702608 learned_addr learned my addr 172.21.15.106:0/3247702608 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:57.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3247702608 >> v2:172.21.15.156:3300/0 conn(0x7f860006bfd0 0x7f8600161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f860659c640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 0x7f860006ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747336398 0 0) 0x7f86000cdf30 con 0x7f8600161860 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.782+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85ec003520 con 0x7f8600161860 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2280185148 0 0) 0x7f85ec003520 con 0x7f8600161860 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 msgr2=0x7f860006ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 0x7f860006ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 >> v2:172.21.15.156:3300/0 conn(0x7f860006bfd0 msgr2=0x7f8600161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 --2- 172.21.15.106:0/3247702608 >> v2:172.21.15.156:3300/0 conn(0x7f860006bfd0 0x7f8600161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8600162030 con 0x7f8600161860 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f85fc0019f0 con 0x7f8600161860 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 >> v1:172.21.15.106:6789/0 conn(0x7f8600161860 legacy=0x7f86000ce9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 0x7f85ec003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f85ec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85ec004d00 0x7f85ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8600161080 con 0x7f85ec003ef0 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8600161d50 con 0x7f85ec004d00 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85ec00b980 con 0x7f8600160ca0 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f85ec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f8605d9b640 1 -- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 msgr2=0x7f85ec003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8600160ca0 0x7f85ec003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f8605d9b640 1 -- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85ec004d00 msgr2=0x7f85ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f85ec004d00 0x7f85ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.783+0000 7f8605d9b640 1 -- 172.21.15.106:0/3247702608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ec003520 con 0x7f85ec003ef0 2023-11-24T08:08:57.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.784+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f85ec0047c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009a90 tx=0x7f85ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7834515704c0cbb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f802c6b0 con 0x7f85ec003ef0 2023-11-24T08:08:57.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f802c810 con 0x7f85ec003ef0 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f860559a640 1 -- 172.21.15.106:0/3247702608 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f802cba0 con 0x7f85ec003ef0 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8608800640 1 -- 172.21.15.106:0/3247702608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 msgr2=0x7f85ec0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8608800640 1 --2- 172.21.15.106:0/3247702608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f85ec0047c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f85f8009a90 tx=0x7f85ec0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8606d9d640 1 -- 172.21.15.106:0/3247702608 reap_dead start 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8608800640 1 -- 172.21.15.106:0/3247702608 shutdown_connections 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8608800640 1 -- 172.21.15.106:0/3247702608 >> 172.21.15.106:0/3247702608 conn(0x7f860009c7e0 msgr2=0x7f860008af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8608800640 1 -- 172.21.15.106:0/3247702608 shutdown_connections 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.785+0000 7f8608800640 1 -- 172.21.15.106:0/3247702608 wait complete. 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.787+0000 7f8608800640 1 Processor -- start 2023-11-24T08:08:57.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.787+0000 7f8608800640 1 -- start start 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f8608800640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f860015f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f8608800640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86001f66c0 0x7f860014e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f8608800640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86001f6b20 0x7f860014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f8608800640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86001f7640 con 0x7f85ec003ef0 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f8608800640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86001f7080 con 0x7f86001f66c0 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f8608800640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86001f7360 con 0x7f86001f6b20 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f860659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f860015f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f860659c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f860015f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34532/0 (socket says 172.21.15.106:34532) 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.788+0000 7f860659c640 1 -- 172.21.15.106:0/3025694065 learned_addr learned my addr 172.21.15.106:0/3025694065 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86001f66c0 0x7f860014e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f8606d9d640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86001f6b20 0x7f860014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f860659c640 1 -- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86001f6b20 msgr2=0x7f860014ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f860659c640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86001f6b20 0x7f860014ec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f860659c640 1 -- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86001f66c0 msgr2=0x7f860014e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f860659c640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86001f66c0 0x7f860014e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f860659c640 1 -- 172.21.15.106:0/3025694065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000c7c90 con 0x7f85ec003ef0 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f860659c640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f860015f280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85f400e7c0 tx=0x7f85f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400eed0 con 0x7f85ec003ef0 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f400c940 con 0x7f85ec003ef0 2023-11-24T08:08:57.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f400ccd0 con 0x7f85ec003ef0 2023-11-24T08:08:57.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86000c7f70 con 0x7f85ec003ef0 2023-11-24T08:08:57.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.789+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f860015f9a0 con 0x7f85ec003ef0 2023-11-24T08:08:57.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.790+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8600067b80 con 0x7f85ec003ef0 2023-11-24T08:08:57.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.791+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f85f4013470 con 0x7f85ec003ef0 2023-11-24T08:08:57.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.792+0000 7f85f2ffd640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f85e406ae70 0x7f85e406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:57.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.792+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85f4089440 con 0x7f85ec003ef0 2023-11-24T08:08:57.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.792+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f85e406ae70 0x7f85e406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:57.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.793+0000 7f8605d9b640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f85e406ae70 0x7f85e406d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85f8036000 tx=0x7f85f8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:57.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.796+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f85f40544e0 con 0x7f85ec003ef0 2023-11-24T08:08:57.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.969+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f86000ca130 con 0x7f85e406ae70 2023-11-24T08:08:57.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.991+0000 7f85f2ffd640 1 -- 172.21.15.106:0/3025694065 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264263 (secure 0 0 0) 0x7f86000ca130 con 0x7f85e406ae70 2023-11-24T08:08:57.995 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:08:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f85e406ae70 msgr2=0x7f85e406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f85e406ae70 0x7f85e406d330 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f85f8036000 tx=0x7f85f8030000 comp rx=0 tx=0).stop 2023-11-24T08:08:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 msgr2=0x7f860015f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f860015f280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f85f400e7c0 tx=0x7f85f400ec80 comp rx=0 tx=0).stop 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 shutdown_connections 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f85e406ae70 0x7f85e406d330 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86001f6b20 0x7f860014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f86001f66c0 0x7f860014e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 --2- 172.21.15.106:0/3025694065 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f85ec003ef0 0x7f860015f280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 >> 172.21.15.106:0/3025694065 conn(0x7f860009c7e0 msgr2=0x7f86000b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 shutdown_connections 2023-11-24T08:08:58.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:57.997+0000 7f8608800640 1 -- 172.21.15.106:0/3025694065 wait complete. 2023-11-24T08:08:58.018 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T08:08:57.277112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235936,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738784,"statfs":{"total":773094113280,"available":772852514816,"internally_reserved":0,"allocated":3055616,"data_stored":1816564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.159591"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354084+0000","last_change":"2023-11-24T08:08:48.155148+0000","last_active":"2023-11-24T08:08:49.354084+0000","last_peered":"2023-11-24T08:08:49.354084+0000","last_clean":"2023-11-24T08:08:49.354084+0000","last_became_active":"2023-11-24T08:08:48.148725+0000","last_became_peered":"2023-11-24T08:08:48.148725+0000","last_unstale":"2023-11-24T08:08:49.354084+0000","last_undegraded":"2023-11-24T08:08:49.354084+0000","last_fullsized":"2023-11-24T08:08:49.354084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:46.218050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126326+0000","last_change":"2023-11-24T08:08:48.153462+0000","last_active":"2023-11-24T08:08:49.126326+0000","last_peered":"2023-11-24T08:08:49.126326+0000","last_clean":"2023-11-24T08:08:49.126326+0000","last_became_active":"2023-11-24T08:08:48.153009+0000","last_became_peered":"2023-11-24T08:08:48.153009+0000","last_unstale":"2023-11-24T08:08:49.126326+0000","last_undegraded":"2023-11-24T08:08:49.126326+0000","last_fullsized":"2023-11-24T08:08:49.126326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552096+0000","last_change":"2023-11-24T08:08:48.147649+0000","last_active":"2023-11-24T08:08:50.552096+0000","last_peered":"2023-11-24T08:08:50.552096+0000","last_clean":"2023-11-24T08:08:50.552096+0000","last_became_active":"2023-11-24T08:08:48.147280+0000","last_became_peered":"2023-11-24T08:08:48.147280+0000","last_unstale":"2023-11-24T08:08:50.552096+0000","last_undegraded":"2023-11-24T08:08:50.552096+0000","last_fullsized":"2023-11-24T08:08:50.552096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126469+0000","last_change":"2023-11-24T08:08:48.158446+0000","last_active":"2023-11-24T08:08:49.126469+0000","last_peered":"2023-11-24T08:08:49.126469+0000","last_clean":"2023-11-24T08:08:49.126469+0000","last_became_active":"2023-11-24T08:08:48.158196+0000","last_became_peered":"2023-11-24T08:08:48.158196+0000","last_unstale":"2023-11-24T08:08:49.126469+0000","last_undegraded":"2023-11-24T08:08:49.126469+0000","last_fullsized":"2023-11-24T08:08:49.126469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:58.752582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353187+0000","last_change":"2023-11-24T08:08:48.148128+0000","last_active":"2023-11-24T08:08:49.353187+0000","last_peered":"2023-11-24T08:08:49.353187+0000","last_clean":"2023-11-24T08:08:49.353187+0000","last_became_active":"2023-11-24T08:08:48.147789+0000","last_became_peered":"2023-11-24T08:08:48.147789+0000","last_unstale":"2023-11-24T08:08:49.353187+0000","last_undegraded":"2023-11-24T08:08:49.353187+0000","last_fullsized":"2023-11-24T08:08:49.353187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:17.852217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126840+0000","last_change":"2023-11-24T08:08:48.160813+0000","last_active":"2023-11-24T08:08:49.126840+0000","last_peered":"2023-11-24T08:08:49.126840+0000","last_clean":"2023-11-24T08:08:49.126840+0000","last_became_active":"2023-11-24T08:08:48.159970+0000","last_became_peered":"2023-11-24T08:08:48.159970+0000","last_unstale":"2023-11-24T08:08:49.126840+0000","last_undegraded":"2023-11-24T08:08:49.126840+0000","last_fullsized":"2023-11-24T08:08:49.126840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:43.997384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352936+0000","last_change":"2023-11-24T08:08:48.147161+0000","last_active":"2023-11-24T08:08:49.352936+0000","last_peered":"2023-11-24T08:08:49.352936+0000","last_clean":"2023-11-24T08:08:49.352936+0000","last_became_active":"2023-11-24T08:08:48.146867+0000","last_became_peered":"2023-11-24T08:08:48.146867+0000","last_unstale":"2023-11-24T08:08:49.352936+0000","last_undegraded":"2023-11-24T08:08:49.352936+0000","last_fullsized":"2023-11-24T08:08:49.352936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:22:50.592771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447578+0000","last_change":"2023-11-24T08:08:48.151256+0000","last_active":"2023-11-24T08:08:50.447578+0000","last_peered":"2023-11-24T08:08:50.447578+0000","last_clean":"2023-11-24T08:08:50.447578+0000","last_became_active":"2023-11-24T08:08:48.150977+0000","last_became_peered":"2023-11-24T08:08:48.150977+0000","last_unstale":"2023-11-24T08:08:50.447578+0000","last_undegraded":"2023-11-24T08:08:50.447578+0000","last_fullsized":"2023-11-24T08:08:50.447578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354145+0000","last_change":"2023-11-24T08:08:48.165291+0000","last_active":"2023-11-24T08:08:49.354145+0000","last_peered":"2023-11-24T08:08:49.354145+0000","last_clean":"2023-11-24T08:08:49.354145+0000","last_became_active":"2023-11-24T08:08:48.164956+0000","last_became_peered":"2023-11-24T08:08:48.164956+0000","last_unstale":"2023-11-24T08:08:49.354145+0000","last_undegraded":"2023-11-24T08:08:49.354145+0000","last_fullsized":"2023-11-24T08:08:49.354145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:32.727731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125355+0000","last_change":"2023-11-24T08:08:48.159627+0000","last_active":"2023-11-24T08:08:49.125355+0000","last_peered":"2023-11-24T08:08:49.125355+0000","last_clean":"2023-11-24T08:08:49.125355+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:49.125355+0000","last_undegraded":"2023-11-24T08:08:49.125355+0000","last_fullsized":"2023-11-24T08:08:49.125355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125146+0000","last_change":"2023-11-24T08:08:48.157396+0000","last_active":"2023-11-24T08:08:49.125146+0000","last_peered":"2023-11-24T08:08:49.125146+0000","last_clean":"2023-11-24T08:08:49.125146+0000","last_became_active":"2023-11-24T08:08:48.155190+0000","last_became_peered":"2023-11-24T08:08:48.155190+0000","last_unstale":"2023-11-24T08:08:49.125146+0000","last_undegraded":"2023-11-24T08:08:49.125146+0000","last_fullsized":"2023-11-24T08:08:49.125146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107513+0000","last_change":"2023-11-24T08:08:48.139798+0000","last_active":"2023-11-24T08:08:50.107513+0000","last_peered":"2023-11-24T08:08:50.107513+0000","last_clean":"2023-11-24T08:08:50.107513+0000","last_became_active":"2023-11-24T08:08:48.139186+0000","last_became_peered":"2023-11-24T08:08:48.139186+0000","last_unstale":"2023-11-24T08:08:50.107513+0000","last_undegraded":"2023-11-24T08:08:50.107513+0000","last_fullsized":"2023-11-24T08:08:50.107513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448422+0000","last_change":"2023-11-24T08:08:48.141667+0000","last_active":"2023-11-24T08:08:50.448422+0000","last_peered":"2023-11-24T08:08:50.448422+0000","last_clean":"2023-11-24T08:08:50.448422+0000","last_became_active":"2023-11-24T08:08:48.141410+0000","last_became_peered":"2023-11-24T08:08:48.141410+0000","last_unstale":"2023-11-24T08:08:50.448422+0000","last_undegraded":"2023-11-24T08:08:50.448422+0000","last_fullsized":"2023-11-24T08:08:50.448422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:54.347050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551315+0000","last_change":"2023-11-24T08:08:48.154158+0000","last_active":"2023-11-24T08:08:50.551315+0000","last_peered":"2023-11-24T08:08:50.551315+0000","last_clean":"2023-11-24T08:08:50.551315+0000","last_became_active":"2023-11-24T08:08:48.153833+0000","last_became_peered":"2023-11-24T08:08:48.153833+0000","last_unstale":"2023-11-24T08:08:50.551315+0000","last_undegraded":"2023-11-24T08:08:50.551315+0000","last_fullsized":"2023-11-24T08:08:50.551315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:00.726435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125503+0000","last_change":"2023-11-24T08:08:48.130056+0000","last_active":"2023-11-24T08:08:49.125503+0000","last_peered":"2023-11-24T08:08:49.125503+0000","last_clean":"2023-11-24T08:08:49.125503+0000","last_became_active":"2023-11-24T08:08:48.129739+0000","last_became_peered":"2023-11-24T08:08:48.129739+0000","last_unstale":"2023-11-24T08:08:49.125503+0000","last_undegraded":"2023-11-24T08:08:49.125503+0000","last_fullsized":"2023-11-24T08:08:49.125503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125661+0000","last_change":"2023-11-24T08:08:48.157064+0000","last_active":"2023-11-24T08:08:49.125661+0000","last_peered":"2023-11-24T08:08:49.125661+0000","last_clean":"2023-11-24T08:08:49.125661+0000","last_became_active":"2023-11-24T08:08:48.153014+0000","last_became_peered":"2023-11-24T08:08:48.153014+0000","last_unstale":"2023-11-24T08:08:49.125661+0000","last_undegraded":"2023-11-24T08:08:49.125661+0000","last_fullsized":"2023-11-24T08:08:49.125661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:53.228655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128174+0000","last_change":"2023-11-24T08:08:48.168538+0000","last_active":"2023-11-24T08:08:49.128174+0000","last_peered":"2023-11-24T08:08:49.128174+0000","last_clean":"2023-11-24T08:08:49.128174+0000","last_became_active":"2023-11-24T08:08:48.168251+0000","last_became_peered":"2023-11-24T08:08:48.168251+0000","last_unstale":"2023-11-24T08:08:49.128174+0000","last_undegraded":"2023-11-24T08:08:49.128174+0000","last_fullsized":"2023-11-24T08:08:49.128174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:34.330436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:51.205453+0000","last_change":"2023-11-24T08:08:48.135832+0000","last_active":"2023-11-24T08:08:51.205453+0000","last_peered":"2023-11-24T08:08:51.205453+0000","last_clean":"2023-11-24T08:08:51.205453+0000","last_became_active":"2023-11-24T08:08:48.135419+0000","last_became_peered":"2023-11-24T08:08:48.135419+0000","last_unstale":"2023-11-24T08:08:51.205453+0000","last_undegraded":"2023-11-24T08:08:51.205453+0000","last_fullsized":"2023-11-24T08:08:51.205453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353897+0000","last_change":"2023-11-24T08:08:48.148026+0000","last_active":"2023-11-24T08:08:49.353897+0000","last_peered":"2023-11-24T08:08:49.353897+0000","last_clean":"2023-11-24T08:08:49.353897+0000","last_became_active":"2023-11-24T08:08:48.147722+0000","last_became_peered":"2023-11-24T08:08:48.147722+0000","last_unstale":"2023-11-24T08:08:49.353897+0000","last_undegraded":"2023-11-24T08:08:49.353897+0000","last_fullsized":"2023-11-24T08:08:49.353897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:15:57.275173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128304+0000","last_change":"2023-11-24T08:08:48.145897+0000","last_active":"2023-11-24T08:08:49.128304+0000","last_peered":"2023-11-24T08:08:49.128304+0000","last_clean":"2023-11-24T08:08:49.128304+0000","last_became_active":"2023-11-24T08:08:48.145620+0000","last_became_peered":"2023-11-24T08:08:48.145620+0000","last_unstale":"2023-11-24T08:08:49.128304+0000","last_undegraded":"2023-11-24T08:08:49.128304+0000","last_fullsized":"2023-11-24T08:08:49.128304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551787+0000","last_change":"2023-11-24T08:08:48.164711+0000","last_active":"2023-11-24T08:08:50.551787+0000","last_peered":"2023-11-24T08:08:50.551787+0000","last_clean":"2023-11-24T08:08:50.551787+0000","last_became_active":"2023-11-24T08:08:48.164331+0000","last_became_peered":"2023-11-24T08:08:48.164331+0000","last_unstale":"2023-11-24T08:08:50.551787+0000","last_undegraded":"2023-11-24T08:08:50.551787+0000","last_fullsized":"2023-11-24T08:08:50.551787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127224+0000","last_change":"2023-11-24T08:08:48.148716+0000","last_active":"2023-11-24T08:08:49.127224+0000","last_peered":"2023-11-24T08:08:49.127224+0000","last_clean":"2023-11-24T08:08:49.127224+0000","last_became_active":"2023-11-24T08:08:48.148271+0000","last_became_peered":"2023-11-24T08:08:48.148271+0000","last_unstale":"2023-11-24T08:08:49.127224+0000","last_undegraded":"2023-11-24T08:08:49.127224+0000","last_fullsized":"2023-11-24T08:08:49.127224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:14.407973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447763+0000","last_change":"2023-11-24T08:08:48.156383+0000","last_active":"2023-11-24T08:08:50.447763+0000","last_peered":"2023-11-24T08:08:50.447763+0000","last_clean":"2023-11-24T08:08:50.447763+0000","last_became_active":"2023-11-24T08:08:48.156083+0000","last_became_peered":"2023-11-24T08:08:48.156083+0000","last_unstale":"2023-11-24T08:08:50.447763+0000","last_undegraded":"2023-11-24T08:08:50.447763+0000","last_fullsized":"2023-11-24T08:08:50.447763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126030+0000","last_change":"2023-11-24T08:08:48.157282+0000","last_active":"2023-11-24T08:08:49.126030+0000","last_peered":"2023-11-24T08:08:49.126030+0000","last_clean":"2023-11-24T08:08:49.126030+0000","last_became_active":"2023-11-24T08:08:48.155003+0000","last_became_peered":"2023-11-24T08:08:48.155003+0000","last_unstale":"2023-11-24T08:08:49.126030+0000","last_undegraded":"2023-11-24T08:08:49.126030+0000","last_fullsized":"2023-11-24T08:08:49.126030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:02.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448748+0000","last_change":"2023-11-24T08:08:48.146014+0000","last_active":"2023-11-24T08:08:50.448748+0000","last_peered":"2023-11-24T08:08:50.448748+0000","last_clean":"2023-11-24T08:08:50.448748+0000","last_became_active":"2023-11-24T08:08:48.144834+0000","last_became_peered":"2023-11-24T08:08:48.144834+0000","last_unstale":"2023-11-24T08:08:50.448748+0000","last_undegraded":"2023-11-24T08:08:50.448748+0000","last_fullsized":"2023-11-24T08:08:50.448748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:22:50.592771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125567+0000","last_change":"2023-11-24T08:08:48.142247+0000","last_active":"2023-11-24T08:08:49.125567+0000","last_peered":"2023-11-24T08:08:49.125567+0000","last_clean":"2023-11-24T08:08:49.125567+0000","last_became_active":"2023-11-24T08:08:48.141969+0000","last_became_peered":"2023-11-24T08:08:48.141969+0000","last_unstale":"2023-11-24T08:08:49.125567+0000","last_undegraded":"2023-11-24T08:08:49.125567+0000","last_fullsized":"2023-11-24T08:08:49.125567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126146+0000","last_change":"2023-11-24T08:08:48.134993+0000","last_active":"2023-11-24T08:08:49.126146+0000","last_peered":"2023-11-24T08:08:49.126146+0000","last_clean":"2023-11-24T08:08:49.126146+0000","last_became_active":"2023-11-24T08:08:48.134752+0000","last_became_peered":"2023-11-24T08:08:48.134752+0000","last_unstale":"2023-11-24T08:08:49.126146+0000","last_undegraded":"2023-11-24T08:08:49.126146+0000","last_fullsized":"2023-11-24T08:08:49.126146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448933+0000","last_change":"2023-11-24T08:08:48.147386+0000","last_active":"2023-11-24T08:08:50.448933+0000","last_peered":"2023-11-24T08:08:50.448933+0000","last_clean":"2023-11-24T08:08:50.448933+0000","last_became_active":"2023-11-24T08:08:48.144941+0000","last_became_peered":"2023-11-24T08:08:48.144941+0000","last_unstale":"2023-11-24T08:08:50.448933+0000","last_undegraded":"2023-11-24T08:08:50.448933+0000","last_fullsized":"2023-11-24T08:08:50.448933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:17.852217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552015+0000","last_change":"2023-11-24T08:08:48.153360+0000","last_active":"2023-11-24T08:08:50.552015+0000","last_peered":"2023-11-24T08:08:50.552015+0000","last_clean":"2023-11-24T08:08:50.552015+0000","last_became_active":"2023-11-24T08:08:48.153099+0000","last_became_peered":"2023-11-24T08:08:48.153099+0000","last_unstale":"2023-11-24T08:08:50.552015+0000","last_undegraded":"2023-11-24T08:08:50.552015+0000","last_fullsized":"2023-11-24T08:08:50.552015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127574+0000","last_change":"2023-11-24T08:08:48.156712+0000","last_active":"2023-11-24T08:08:49.127574+0000","last_peered":"2023-11-24T08:08:49.127574+0000","last_clean":"2023-11-24T08:08:49.127574+0000","last_became_active":"2023-11-24T08:08:48.151074+0000","last_became_peered":"2023-11-24T08:08:48.151074+0000","last_unstale":"2023-11-24T08:08:49.127574+0000","last_undegraded":"2023-11-24T08:08:49.127574+0000","last_fullsized":"2023-11-24T08:08:49.127574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.079373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:51.205305+0000","last_change":"2023-11-24T08:08:48.149394+0000","last_active":"2023-11-24T08:08:51.205305+0000","last_peered":"2023-11-24T08:08:51.205305+0000","last_clean":"2023-11-24T08:08:51.205305+0000","last_became_active":"2023-11-24T08:08:48.149158+0000","last_became_peered":"2023-11-24T08:08:48.149158+0000","last_unstale":"2023-11-24T08:08:51.205305+0000","last_undegraded":"2023-11-24T08:08:51.205305+0000","last_fullsized":"2023-11-24T08:08:51.205305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124630+0000","last_change":"2023-11-24T08:08:48.155627+0000","last_active":"2023-11-24T08:08:49.124630+0000","last_peered":"2023-11-24T08:08:49.124630+0000","last_clean":"2023-11-24T08:08:49.124630+0000","last_became_active":"2023-11-24T08:08:48.155343+0000","last_became_peered":"2023-11-24T08:08:48.155343+0000","last_unstale":"2023-11-24T08:08:49.124630+0000","last_undegraded":"2023-11-24T08:08:49.124630+0000","last_fullsized":"2023-11-24T08:08:49.124630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:00.726435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107026+0000","last_change":"2023-11-24T08:08:48.141457+0000","last_active":"2023-11-24T08:08:50.107026+0000","last_peered":"2023-11-24T08:08:50.107026+0000","last_clean":"2023-11-24T08:08:50.107026+0000","last_became_active":"2023-11-24T08:08:48.139546+0000","last_became_peered":"2023-11-24T08:08:48.139546+0000","last_unstale":"2023-11-24T08:08:50.107026+0000","last_undegraded":"2023-11-24T08:08:50.107026+0000","last_fullsized":"2023-11-24T08:08:50.107026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.345213+0000","last_change":"2023-11-24T08:08:47.144970+0000","last_active":"2023-11-24T08:08:50.345213+0000","last_peered":"2023-11-24T08:08:50.345213+0000","last_clean":"2023-11-24T08:08:50.345213+0000","last_became_active":"2023-11-24T08:08:47.144667+0000","last_became_peered":"2023-11-24T08:08:47.144667+0000","last_unstale":"2023-11-24T08:08:50.345213+0000","last_undegraded":"2023-11-24T08:08:50.345213+0000","last_fullsized":"2023-11-24T08:08:50.345213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:59.483267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552207+0000","last_change":"2023-11-24T08:08:48.158059+0000","last_active":"2023-11-24T08:08:50.552207+0000","last_peered":"2023-11-24T08:08:50.552207+0000","last_clean":"2023-11-24T08:08:50.552207+0000","last_became_active":"2023-11-24T08:08:48.157743+0000","last_became_peered":"2023-11-24T08:08:48.157743+0000","last_unstale":"2023-11-24T08:08:50.552207+0000","last_undegraded":"2023-11-24T08:08:50.552207+0000","last_fullsized":"2023-11-24T08:08:50.552207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:54.347050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125078+0000","last_change":"2023-11-24T08:08:47.135247+0000","last_active":"2023-11-24T08:08:49.125078+0000","last_peered":"2023-11-24T08:08:49.125078+0000","last_clean":"2023-11-24T08:08:49.125078+0000","last_became_active":"2023-11-24T08:08:47.134923+0000","last_became_peered":"2023-11-24T08:08:47.134923+0000","last_unstale":"2023-11-24T08:08:49.125078+0000","last_undegraded":"2023-11-24T08:08:49.125078+0000","last_fullsized":"2023-11-24T08:08:49.125078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.345062+0000","last_change":"2023-11-24T08:08:47.151833+0000","last_active":"2023-11-24T08:08:50.345062+0000","last_peered":"2023-11-24T08:08:50.345062+0000","last_clean":"2023-11-24T08:08:50.345062+0000","last_became_active":"2023-11-24T08:08:47.151517+0000","last_became_peered":"2023-11-24T08:08:47.151517+0000","last_unstale":"2023-11-24T08:08:50.345062+0000","last_undegraded":"2023-11-24T08:08:50.345062+0000","last_fullsized":"2023-11-24T08:08:50.345062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447758+0000","last_change":"2023-11-24T08:08:48.148262+0000","last_active":"2023-11-24T08:08:50.447758+0000","last_peered":"2023-11-24T08:08:50.447758+0000","last_clean":"2023-11-24T08:08:50.447758+0000","last_became_active":"2023-11-24T08:08:48.147874+0000","last_became_peered":"2023-11-24T08:08:48.147874+0000","last_unstale":"2023-11-24T08:08:50.447758+0000","last_undegraded":"2023-11-24T08:08:50.447758+0000","last_fullsized":"2023-11-24T08:08:50.447758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126262+0000","last_change":"2023-11-24T08:08:47.144280+0000","last_active":"2023-11-24T08:08:49.126262+0000","last_peered":"2023-11-24T08:08:49.126262+0000","last_clean":"2023-11-24T08:08:49.126262+0000","last_became_active":"2023-11-24T08:08:47.143933+0000","last_became_peered":"2023-11-24T08:08:47.143933+0000","last_unstale":"2023-11-24T08:08:49.126262+0000","last_undegraded":"2023-11-24T08:08:49.126262+0000","last_fullsized":"2023-11-24T08:08:49.126262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:28.820828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125379+0000","last_change":"2023-11-24T08:08:48.151024+0000","last_active":"2023-11-24T08:08:49.125379+0000","last_peered":"2023-11-24T08:08:49.125379+0000","last_clean":"2023-11-24T08:08:49.125379+0000","last_became_active":"2023-11-24T08:08:48.150727+0000","last_became_peered":"2023-11-24T08:08:48.150727+0000","last_unstale":"2023-11-24T08:08:49.125379+0000","last_undegraded":"2023-11-24T08:08:49.125379+0000","last_fullsized":"2023-11-24T08:08:49.125379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127936+0000","last_change":"2023-11-24T08:08:47.158190+0000","last_active":"2023-11-24T08:08:49.127936+0000","last_peered":"2023-11-24T08:08:49.127936+0000","last_clean":"2023-11-24T08:08:49.127936+0000","last_became_active":"2023-11-24T08:08:47.157894+0000","last_became_peered":"2023-11-24T08:08:47.157894+0000","last_unstale":"2023-11-24T08:08:49.127936+0000","last_undegraded":"2023-11-24T08:08:49.127936+0000","last_fullsized":"2023-11-24T08:08:49.127936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:18.739564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127729+0000","last_change":"2023-11-24T08:08:48.152122+0000","last_active":"2023-11-24T08:08:49.127729+0000","last_peered":"2023-11-24T08:08:49.127729+0000","last_clean":"2023-11-24T08:08:49.127729+0000","last_became_active":"2023-11-24T08:08:48.151509+0000","last_became_peered":"2023-11-24T08:08:48.151509+0000","last_unstale":"2023-11-24T08:08:49.127729+0000","last_undegraded":"2023-11-24T08:08:49.127729+0000","last_fullsized":"2023-11-24T08:08:49.127729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:17.382456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127836+0000","last_change":"2023-11-24T08:08:47.154457+0000","last_active":"2023-11-24T08:08:49.127836+0000","last_peered":"2023-11-24T08:08:49.127836+0000","last_clean":"2023-11-24T08:08:49.127836+0000","last_became_active":"2023-11-24T08:08:47.154137+0000","last_became_peered":"2023-11-24T08:08:47.154137+0000","last_unstale":"2023-11-24T08:08:49.127836+0000","last_undegraded":"2023-11-24T08:08:49.127836+0000","last_fullsized":"2023-11-24T08:08:49.127836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:19.309577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552125+0000","last_change":"2023-11-24T08:08:48.154269+0000","last_active":"2023-11-24T08:08:50.552125+0000","last_peered":"2023-11-24T08:08:50.552125+0000","last_clean":"2023-11-24T08:08:50.552125+0000","last_became_active":"2023-11-24T08:08:48.153873+0000","last_became_peered":"2023-11-24T08:08:48.153873+0000","last_unstale":"2023-11-24T08:08:50.552125+0000","last_undegraded":"2023-11-24T08:08:50.552125+0000","last_fullsized":"2023-11-24T08:08:50.552125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:30.453859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126169+0000","last_change":"2023-11-24T08:08:47.156467+0000","last_active":"2023-11-24T08:08:49.126169+0000","last_peered":"2023-11-24T08:08:49.126169+0000","last_clean":"2023-11-24T08:08:49.126169+0000","last_became_active":"2023-11-24T08:08:47.155999+0000","last_became_peered":"2023-11-24T08:08:47.155999+0000","last_unstale":"2023-11-24T08:08:49.126169+0000","last_undegraded":"2023-11-24T08:08:49.126169+0000","last_fullsized":"2023-11-24T08:08:49.126169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107764+0000","last_change":"2023-11-24T08:08:48.142662+0000","last_active":"2023-11-24T08:08:50.107764+0000","last_peered":"2023-11-24T08:08:50.107764+0000","last_clean":"2023-11-24T08:08:50.107764+0000","last_became_active":"2023-11-24T08:08:48.139505+0000","last_became_peered":"2023-11-24T08:08:48.139505+0000","last_unstale":"2023-11-24T08:08:50.107764+0000","last_undegraded":"2023-11-24T08:08:50.107764+0000","last_fullsized":"2023-11-24T08:08:50.107764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:01.498497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125359+0000","last_change":"2023-11-24T08:08:47.148708+0000","last_active":"2023-11-24T08:08:49.125359+0000","last_peered":"2023-11-24T08:08:49.125359+0000","last_clean":"2023-11-24T08:08:49.125359+0000","last_became_active":"2023-11-24T08:08:47.148386+0000","last_became_peered":"2023-11-24T08:08:47.148386+0000","last_unstale":"2023-11-24T08:08:49.125359+0000","last_undegraded":"2023-11-24T08:08:49.125359+0000","last_fullsized":"2023-11-24T08:08:49.125359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353438+0000","last_change":"2023-11-24T08:08:48.147343+0000","last_active":"2023-11-24T08:08:49.353438+0000","last_peered":"2023-11-24T08:08:49.353438+0000","last_clean":"2023-11-24T08:08:49.353438+0000","last_became_active":"2023-11-24T08:08:48.146625+0000","last_became_peered":"2023-11-24T08:08:48.146625+0000","last_unstale":"2023-11-24T08:08:49.353438+0000","last_undegraded":"2023-11-24T08:08:49.353438+0000","last_fullsized":"2023-11-24T08:08:49.353438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.269500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.342547+0000","last_change":"2023-11-24T08:08:47.141638+0000","last_active":"2023-11-24T08:08:50.342547+0000","last_peered":"2023-11-24T08:08:50.342547+0000","last_clean":"2023-11-24T08:08:50.342547+0000","last_became_active":"2023-11-24T08:08:47.141294+0000","last_became_peered":"2023-11-24T08:08:47.141294+0000","last_unstale":"2023-11-24T08:08:50.342547+0000","last_undegraded":"2023-11-24T08:08:50.342547+0000","last_fullsized":"2023-11-24T08:08:50.342547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551206+0000","last_change":"2023-11-24T08:08:48.159790+0000","last_active":"2023-11-24T08:08:50.551206+0000","last_peered":"2023-11-24T08:08:50.551206+0000","last_clean":"2023-11-24T08:08:50.551206+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:50.551206+0000","last_undegraded":"2023-11-24T08:08:50.551206+0000","last_fullsized":"2023-11-24T08:08:50.551206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:19.982732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127841+0000","last_change":"2023-11-24T08:08:47.157623+0000","last_active":"2023-11-24T08:08:49.127841+0000","last_peered":"2023-11-24T08:08:49.127841+0000","last_clean":"2023-11-24T08:08:49.127841+0000","last_became_active":"2023-11-24T08:08:47.156256+0000","last_became_peered":"2023-11-24T08:08:47.156256+0000","last_unstale":"2023-11-24T08:08:49.127841+0000","last_undegraded":"2023-11-24T08:08:49.127841+0000","last_fullsized":"2023-11-24T08:08:49.127841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:59.304396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353649+0000","last_change":"2023-11-24T08:08:48.152586+0000","last_active":"2023-11-24T08:08:49.353649+0000","last_peered":"2023-11-24T08:08:49.353649+0000","last_clean":"2023-11-24T08:08:49.353649+0000","last_became_active":"2023-11-24T08:08:48.152296+0000","last_became_peered":"2023-11-24T08:08:48.152296+0000","last_unstale":"2023-11-24T08:08:49.353649+0000","last_undegraded":"2023-11-24T08:08:49.353649+0000","last_fullsized":"2023-11-24T08:08:49.353649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:23.916532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124531+0000","last_change":"2023-11-24T08:08:47.136838+0000","last_active":"2023-11-24T08:08:49.124531+0000","last_peered":"2023-11-24T08:08:49.124531+0000","last_clean":"2023-11-24T08:08:49.124531+0000","last_became_active":"2023-11-24T08:08:47.136397+0000","last_became_peered":"2023-11-24T08:08:47.136397+0000","last_unstale":"2023-11-24T08:08:49.124531+0000","last_undegraded":"2023-11-24T08:08:49.124531+0000","last_fullsized":"2023-11-24T08:08:49.124531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125520+0000","last_change":"2023-11-24T08:08:48.153870+0000","last_active":"2023-11-24T08:08:49.125520+0000","last_peered":"2023-11-24T08:08:49.125520+0000","last_clean":"2023-11-24T08:08:49.125520+0000","last_became_active":"2023-11-24T08:08:48.153566+0000","last_became_peered":"2023-11-24T08:08:48.153566+0000","last_unstale":"2023-11-24T08:08:49.125520+0000","last_undegraded":"2023-11-24T08:08:49.125520+0000","last_fullsized":"2023-11-24T08:08:49.125520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351287+0000","last_change":"2023-11-24T08:08:47.133661+0000","last_active":"2023-11-24T08:08:50.351287+0000","last_peered":"2023-11-24T08:08:50.351287+0000","last_clean":"2023-11-24T08:08:50.351287+0000","last_became_active":"2023-11-24T08:08:47.133327+0000","last_became_peered":"2023-11-24T08:08:47.133327+0000","last_unstale":"2023-11-24T08:08:50.351287+0000","last_undegraded":"2023-11-24T08:08:50.351287+0000","last_fullsized":"2023-11-24T08:08:50.351287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:36.361920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352992+0000","last_change":"2023-11-24T08:08:48.159918+0000","last_active":"2023-11-24T08:08:49.352992+0000","last_peered":"2023-11-24T08:08:49.352992+0000","last_clean":"2023-11-24T08:08:49.352992+0000","last_became_active":"2023-11-24T08:08:48.159604+0000","last_became_peered":"2023-11-24T08:08:48.159604+0000","last_unstale":"2023-11-24T08:08:49.352992+0000","last_undegraded":"2023-11-24T08:08:49.352992+0000","last_fullsized":"2023-11-24T08:08:49.352992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:19.645688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107838+0000","last_change":"2023-11-24T08:08:47.139479+0000","last_active":"2023-11-24T08:08:50.107838+0000","last_peered":"2023-11-24T08:08:50.107838+0000","last_clean":"2023-11-24T08:08:50.107838+0000","last_became_active":"2023-11-24T08:08:47.139162+0000","last_became_peered":"2023-11-24T08:08:47.139162+0000","last_unstale":"2023-11-24T08:08:50.107838+0000","last_undegraded":"2023-11-24T08:08:50.107838+0000","last_fullsized":"2023-11-24T08:08:50.107838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:51.916232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353918+0000","last_change":"2023-11-24T08:08:48.160119+0000","last_active":"2023-11-24T08:08:49.353918+0000","last_peered":"2023-11-24T08:08:49.353918+0000","last_clean":"2023-11-24T08:08:49.353918+0000","last_became_active":"2023-11-24T08:08:48.159806+0000","last_became_peered":"2023-11-24T08:08:48.159806+0000","last_unstale":"2023-11-24T08:08:49.353918+0000","last_undegraded":"2023-11-24T08:08:49.353918+0000","last_fullsized":"2023-11-24T08:08:49.353918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:19.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351440+0000","last_change":"2023-11-24T08:08:47.139000+0000","last_active":"2023-11-24T08:08:50.351440+0000","last_peered":"2023-11-24T08:08:50.351440+0000","last_clean":"2023-11-24T08:08:50.351440+0000","last_became_active":"2023-11-24T08:08:47.138477+0000","last_became_peered":"2023-11-24T08:08:47.138477+0000","last_unstale":"2023-11-24T08:08:50.351440+0000","last_undegraded":"2023-11-24T08:08:50.351440+0000","last_fullsized":"2023-11-24T08:08:50.351440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:49.285847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354329+0000","last_change":"2023-11-24T08:08:48.162632+0000","last_active":"2023-11-24T08:08:49.354329+0000","last_peered":"2023-11-24T08:08:49.354329+0000","last_clean":"2023-11-24T08:08:49.354329+0000","last_became_active":"2023-11-24T08:08:48.160274+0000","last_became_peered":"2023-11-24T08:08:48.160274+0000","last_unstale":"2023-11-24T08:08:49.354329+0000","last_undegraded":"2023-11-24T08:08:49.354329+0000","last_fullsized":"2023-11-24T08:08:49.354329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:23.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126343+0000","last_change":"2023-11-24T08:08:47.133083+0000","last_active":"2023-11-24T08:08:49.126343+0000","last_peered":"2023-11-24T08:08:49.126343+0000","last_clean":"2023-11-24T08:08:49.126343+0000","last_became_active":"2023-11-24T08:08:47.132746+0000","last_became_peered":"2023-11-24T08:08:47.132746+0000","last_unstale":"2023-11-24T08:08:49.126343+0000","last_undegraded":"2023-11-24T08:08:49.126343+0000","last_fullsized":"2023-11-24T08:08:49.126343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:25.146390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126883+0000","last_change":"2023-11-24T08:08:48.151332+0000","last_active":"2023-11-24T08:08:49.126883+0000","last_peered":"2023-11-24T08:08:49.126883+0000","last_clean":"2023-11-24T08:08:49.126883+0000","last_became_active":"2023-11-24T08:08:48.151045+0000","last_became_peered":"2023-11-24T08:08:48.151045+0000","last_unstale":"2023-11-24T08:08:49.126883+0000","last_undegraded":"2023-11-24T08:08:49.126883+0000","last_fullsized":"2023-11-24T08:08:49.126883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:15.646317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351397+0000","last_change":"2023-11-24T08:08:47.150523+0000","last_active":"2023-11-24T08:08:50.351397+0000","last_peered":"2023-11-24T08:08:50.351397+0000","last_clean":"2023-11-24T08:08:50.351397+0000","last_became_active":"2023-11-24T08:08:47.150208+0000","last_became_peered":"2023-11-24T08:08:47.150208+0000","last_unstale":"2023-11-24T08:08:50.351397+0000","last_undegraded":"2023-11-24T08:08:50.351397+0000","last_fullsized":"2023-11-24T08:08:50.351397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353456+0000","last_change":"2023-11-24T08:08:48.156319+0000","last_active":"2023-11-24T08:08:49.353456+0000","last_peered":"2023-11-24T08:08:49.353456+0000","last_clean":"2023-11-24T08:08:49.353456+0000","last_became_active":"2023-11-24T08:08:48.154550+0000","last_became_peered":"2023-11-24T08:08:48.154550+0000","last_unstale":"2023-11-24T08:08:49.353456+0000","last_undegraded":"2023-11-24T08:08:49.353456+0000","last_fullsized":"2023-11-24T08:08:49.353456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:17.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.135471+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:08:48.135471+0000","last_peered":"2023-11-24T08:08:48.135471+0000","last_clean":"2023-11-24T08:08:48.135471+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:08:48.135471+0000","last_undegraded":"2023-11-24T08:08:48.135471+0000","last_fullsized":"2023-11-24T08:08:48.135471+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125945+0000","last_change":"2023-11-24T08:08:47.157035+0000","last_active":"2023-11-24T08:08:49.125945+0000","last_peered":"2023-11-24T08:08:49.125945+0000","last_clean":"2023-11-24T08:08:49.125945+0000","last_became_active":"2023-11-24T08:08:47.155189+0000","last_became_peered":"2023-11-24T08:08:47.155189+0000","last_unstale":"2023-11-24T08:08:49.125945+0000","last_undegraded":"2023-11-24T08:08:49.125945+0000","last_fullsized":"2023-11-24T08:08:49.125945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:54.597005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448128+0000","last_change":"2023-11-24T08:08:48.161239+0000","last_active":"2023-11-24T08:08:50.448128+0000","last_peered":"2023-11-24T08:08:50.448128+0000","last_clean":"2023-11-24T08:08:50.448128+0000","last_became_active":"2023-11-24T08:08:48.160530+0000","last_became_peered":"2023-11-24T08:08:48.160530+0000","last_unstale":"2023-11-24T08:08:50.448128+0000","last_undegraded":"2023-11-24T08:08:50.448128+0000","last_fullsized":"2023-11-24T08:08:50.448128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:30.453859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353366+0000","last_change":"2023-11-24T08:08:47.153886+0000","last_active":"2023-11-24T08:08:49.353366+0000","last_peered":"2023-11-24T08:08:49.353366+0000","last_clean":"2023-11-24T08:08:49.353366+0000","last_became_active":"2023-11-24T08:08:47.153576+0000","last_became_peered":"2023-11-24T08:08:47.153576+0000","last_unstale":"2023-11-24T08:08:49.353366+0000","last_undegraded":"2023-11-24T08:08:49.353366+0000","last_fullsized":"2023-11-24T08:08:49.353366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551438+0000","last_change":"2023-11-24T08:08:48.157955+0000","last_active":"2023-11-24T08:08:50.551438+0000","last_peered":"2023-11-24T08:08:50.551438+0000","last_clean":"2023-11-24T08:08:50.551438+0000","last_became_active":"2023-11-24T08:08:48.157636+0000","last_became_peered":"2023-11-24T08:08:48.157636+0000","last_unstale":"2023-11-24T08:08:50.551438+0000","last_undegraded":"2023-11-24T08:08:50.551438+0000","last_fullsized":"2023-11-24T08:08:50.551438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351375+0000","last_change":"2023-11-24T08:08:47.143826+0000","last_active":"2023-11-24T08:08:50.351375+0000","last_peered":"2023-11-24T08:08:50.351375+0000","last_clean":"2023-11-24T08:08:50.351375+0000","last_became_active":"2023-11-24T08:08:47.142226+0000","last_became_peered":"2023-11-24T08:08:47.142226+0000","last_unstale":"2023-11-24T08:08:50.351375+0000","last_undegraded":"2023-11-24T08:08:50.351375+0000","last_fullsized":"2023-11-24T08:08:50.351375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125232+0000","last_change":"2023-11-24T08:08:48.133248+0000","last_active":"2023-11-24T08:08:49.125232+0000","last_peered":"2023-11-24T08:08:49.125232+0000","last_clean":"2023-11-24T08:08:49.125232+0000","last_became_active":"2023-11-24T08:08:48.133014+0000","last_became_peered":"2023-11-24T08:08:48.133014+0000","last_unstale":"2023-11-24T08:08:49.125232+0000","last_undegraded":"2023-11-24T08:08:49.125232+0000","last_fullsized":"2023-11-24T08:08:49.125232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127822+0000","last_change":"2023-11-24T08:08:47.154380+0000","last_active":"2023-11-24T08:08:49.127822+0000","last_peered":"2023-11-24T08:08:49.127822+0000","last_clean":"2023-11-24T08:08:49.127822+0000","last_became_active":"2023-11-24T08:08:47.154066+0000","last_became_peered":"2023-11-24T08:08:47.154066+0000","last_unstale":"2023-11-24T08:08:49.127822+0000","last_undegraded":"2023-11-24T08:08:49.127822+0000","last_fullsized":"2023-11-24T08:08:49.127822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:48:29.123228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107253+0000","last_change":"2023-11-24T08:08:48.155559+0000","last_active":"2023-11-24T08:08:50.107253+0000","last_peered":"2023-11-24T08:08:50.107253+0000","last_clean":"2023-11-24T08:08:50.107253+0000","last_became_active":"2023-11-24T08:08:48.155311+0000","last_became_peered":"2023-11-24T08:08:48.155311+0000","last_unstale":"2023-11-24T08:08:50.107253+0000","last_undegraded":"2023-11-24T08:08:50.107253+0000","last_fullsized":"2023-11-24T08:08:50.107253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125803+0000","last_change":"2023-11-24T08:08:47.137292+0000","last_active":"2023-11-24T08:08:49.125803+0000","last_peered":"2023-11-24T08:08:49.125803+0000","last_clean":"2023-11-24T08:08:49.125803+0000","last_became_active":"2023-11-24T08:08:47.136872+0000","last_became_peered":"2023-11-24T08:08:47.136872+0000","last_unstale":"2023-11-24T08:08:49.125803+0000","last_undegraded":"2023-11-24T08:08:49.125803+0000","last_fullsized":"2023-11-24T08:08:49.125803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127383+0000","last_change":"2023-11-24T08:08:48.161415+0000","last_active":"2023-11-24T08:08:49.127383+0000","last_peered":"2023-11-24T08:08:49.127383+0000","last_clean":"2023-11-24T08:08:49.127383+0000","last_became_active":"2023-11-24T08:08:48.158160+0000","last_became_peered":"2023-11-24T08:08:48.158160+0000","last_unstale":"2023-11-24T08:08:49.127383+0000","last_undegraded":"2023-11-24T08:08:49.127383+0000","last_fullsized":"2023-11-24T08:08:49.127383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:24.063260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354005+0000","last_change":"2023-11-24T08:08:47.153377+0000","last_active":"2023-11-24T08:08:49.354005+0000","last_peered":"2023-11-24T08:08:49.354005+0000","last_clean":"2023-11-24T08:08:49.354005+0000","last_became_active":"2023-11-24T08:08:47.153112+0000","last_became_peered":"2023-11-24T08:08:47.153112+0000","last_unstale":"2023-11-24T08:08:49.354005+0000","last_undegraded":"2023-11-24T08:08:49.354005+0000","last_fullsized":"2023-11-24T08:08:49.354005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:39.232399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125805+0000","last_change":"2023-11-24T08:08:48.134378+0000","last_active":"2023-11-24T08:08:49.125805+0000","last_peered":"2023-11-24T08:08:49.125805+0000","last_clean":"2023-11-24T08:08:49.125805+0000","last_became_active":"2023-11-24T08:08:48.134043+0000","last_became_peered":"2023-11-24T08:08:48.134043+0000","last_unstale":"2023-11-24T08:08:49.125805+0000","last_undegraded":"2023-11-24T08:08:49.125805+0000","last_fullsized":"2023-11-24T08:08:49.125805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.352131+0000","last_change":"2023-11-24T08:08:47.148275+0000","last_active":"2023-11-24T08:08:50.352131+0000","last_peered":"2023-11-24T08:08:50.352131+0000","last_clean":"2023-11-24T08:08:50.352131+0000","last_became_active":"2023-11-24T08:08:47.148025+0000","last_became_peered":"2023-11-24T08:08:47.148025+0000","last_unstale":"2023-11-24T08:08:50.352131+0000","last_undegraded":"2023-11-24T08:08:50.352131+0000","last_fullsized":"2023-11-24T08:08:50.352131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.449026+0000","last_change":"2023-11-24T08:08:48.146544+0000","last_active":"2023-11-24T08:08:50.449026+0000","last_peered":"2023-11-24T08:08:50.449026+0000","last_clean":"2023-11-24T08:08:50.449026+0000","last_became_active":"2023-11-24T08:08:48.146269+0000","last_became_peered":"2023-11-24T08:08:48.146269+0000","last_unstale":"2023-11-24T08:08:50.449026+0000","last_undegraded":"2023-11-24T08:08:50.449026+0000","last_fullsized":"2023-11-24T08:08:50.449026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.269500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128129+0000","last_change":"2023-11-24T08:08:47.152442+0000","last_active":"2023-11-24T08:08:49.128129+0000","last_peered":"2023-11-24T08:08:49.128129+0000","last_clean":"2023-11-24T08:08:49.128129+0000","last_became_active":"2023-11-24T08:08:47.151763+0000","last_became_peered":"2023-11-24T08:08:47.151763+0000","last_unstale":"2023-11-24T08:08:49.128129+0000","last_undegraded":"2023-11-24T08:08:49.128129+0000","last_fullsized":"2023-11-24T08:08:49.128129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551563+0000","last_change":"2023-11-24T08:08:48.152228+0000","last_active":"2023-11-24T08:08:50.551563+0000","last_peered":"2023-11-24T08:08:50.551563+0000","last_clean":"2023-11-24T08:08:50.551563+0000","last_became_active":"2023-11-24T08:08:48.151904+0000","last_became_peered":"2023-11-24T08:08:48.151904+0000","last_unstale":"2023-11-24T08:08:50.551563+0000","last_undegraded":"2023-11-24T08:08:50.551563+0000","last_fullsized":"2023-11-24T08:08:50.551563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352672+0000","last_change":"2023-11-24T08:08:47.150060+0000","last_active":"2023-11-24T08:08:49.352672+0000","last_peered":"2023-11-24T08:08:49.352672+0000","last_clean":"2023-11-24T08:08:49.352672+0000","last_became_active":"2023-11-24T08:08:47.149756+0000","last_became_peered":"2023-11-24T08:08:47.149756+0000","last_unstale":"2023-11-24T08:08:49.352672+0000","last_undegraded":"2023-11-24T08:08:49.352672+0000","last_fullsized":"2023-11-24T08:08:49.352672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107214+0000","last_change":"2023-11-24T08:08:48.150454+0000","last_active":"2023-11-24T08:08:50.107214+0000","last_peered":"2023-11-24T08:08:50.107214+0000","last_clean":"2023-11-24T08:08:50.107214+0000","last_became_active":"2023-11-24T08:08:48.139831+0000","last_became_peered":"2023-11-24T08:08:48.139831+0000","last_unstale":"2023-11-24T08:08:50.107214+0000","last_undegraded":"2023-11-24T08:08:50.107214+0000","last_fullsized":"2023-11-24T08:08:50.107214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125621+0000","last_change":"2023-11-24T08:08:47.141901+0000","last_active":"2023-11-24T08:08:49.125621+0000","last_peered":"2023-11-24T08:08:49.125621+0000","last_clean":"2023-11-24T08:08:49.125621+0000","last_became_active":"2023-11-24T08:08:47.140760+0000","last_became_peered":"2023-11-24T08:08:47.140760+0000","last_unstale":"2023-11-24T08:08:49.125621+0000","last_undegraded":"2023-11-24T08:08:49.125621+0000","last_fullsized":"2023-11-24T08:08:49.125621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:12.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447992+0000","last_change":"2023-11-24T08:08:48.141100+0000","last_active":"2023-11-24T08:08:50.447992+0000","last_peered":"2023-11-24T08:08:50.447992+0000","last_clean":"2023-11-24T08:08:50.447992+0000","last_became_active":"2023-11-24T08:08:48.138053+0000","last_became_peered":"2023-11-24T08:08:48.138053+0000","last_unstale":"2023-11-24T08:08:50.447992+0000","last_undegraded":"2023-11-24T08:08:50.447992+0000","last_fullsized":"2023-11-24T08:08:50.447992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125743+0000","last_change":"2023-11-24T08:08:47.138812+0000","last_active":"2023-11-24T08:08:49.125743+0000","last_peered":"2023-11-24T08:08:49.125743+0000","last_clean":"2023-11-24T08:08:49.125743+0000","last_became_active":"2023-11-24T08:08:47.138512+0000","last_became_peered":"2023-11-24T08:08:47.138512+0000","last_unstale":"2023-11-24T08:08:49.125743+0000","last_undegraded":"2023-11-24T08:08:49.125743+0000","last_fullsized":"2023-11-24T08:08:49.125743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448493+0000","last_change":"2023-11-24T08:08:48.155403+0000","last_active":"2023-11-24T08:08:50.448493+0000","last_peered":"2023-11-24T08:08:50.448493+0000","last_clean":"2023-11-24T08:08:50.448493+0000","last_became_active":"2023-11-24T08:08:48.154517+0000","last_became_peered":"2023-11-24T08:08:48.154517+0000","last_unstale":"2023-11-24T08:08:50.448493+0000","last_undegraded":"2023-11-24T08:08:50.448493+0000","last_fullsized":"2023-11-24T08:08:50.448493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:16.993204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353934+0000","last_change":"2023-11-24T08:08:47.152677+0000","last_active":"2023-11-24T08:08:49.353934+0000","last_peered":"2023-11-24T08:08:49.353934+0000","last_clean":"2023-11-24T08:08:49.353934+0000","last_became_active":"2023-11-24T08:08:47.151375+0000","last_became_peered":"2023-11-24T08:08:47.151375+0000","last_unstale":"2023-11-24T08:08:49.353934+0000","last_undegraded":"2023-11-24T08:08:49.353934+0000","last_fullsized":"2023-11-24T08:08:49.353934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:56.046702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125623+0000","last_change":"2023-11-24T08:08:48.162261+0000","last_active":"2023-11-24T08:08:49.125623+0000","last_peered":"2023-11-24T08:08:49.125623+0000","last_clean":"2023-11-24T08:08:49.125623+0000","last_became_active":"2023-11-24T08:08:48.156627+0000","last_became_peered":"2023-11-24T08:08:48.156627+0000","last_unstale":"2023-11-24T08:08:49.125623+0000","last_undegraded":"2023-11-24T08:08:49.125623+0000","last_fullsized":"2023-11-24T08:08:49.125623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127263+0000","last_change":"2023-11-24T08:08:48.162034+0000","last_active":"2023-11-24T08:08:49.127263+0000","last_peered":"2023-11-24T08:08:49.127263+0000","last_clean":"2023-11-24T08:08:49.127263+0000","last_became_active":"2023-11-24T08:08:48.161663+0000","last_became_peered":"2023-11-24T08:08:48.161663+0000","last_unstale":"2023-11-24T08:08:49.127263+0000","last_undegraded":"2023-11-24T08:08:49.127263+0000","last_fullsized":"2023-11-24T08:08:49.127263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:56.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353243+0000","last_change":"2023-11-24T08:08:47.156960+0000","last_active":"2023-11-24T08:08:49.353243+0000","last_peered":"2023-11-24T08:08:49.353243+0000","last_clean":"2023-11-24T08:08:49.353243+0000","last_became_active":"2023-11-24T08:08:47.156401+0000","last_became_peered":"2023-11-24T08:08:47.156401+0000","last_unstale":"2023-11-24T08:08:49.353243+0000","last_undegraded":"2023-11-24T08:08:49.353243+0000","last_fullsized":"2023-11-24T08:08:49.353243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:04.035911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126566+0000","last_change":"2023-11-24T08:08:48.159506+0000","last_active":"2023-11-24T08:08:49.126566+0000","last_peered":"2023-11-24T08:08:49.126566+0000","last_clean":"2023-11-24T08:08:49.126566+0000","last_became_active":"2023-11-24T08:08:48.159220+0000","last_became_peered":"2023-11-24T08:08:48.159220+0000","last_unstale":"2023-11-24T08:08:49.126566+0000","last_undegraded":"2023-11-24T08:08:49.126566+0000","last_fullsized":"2023-11-24T08:08:49.126566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:14.900691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353690+0000","last_change":"2023-11-24T08:08:47.155268+0000","last_active":"2023-11-24T08:08:49.353690+0000","last_peered":"2023-11-24T08:08:49.353690+0000","last_clean":"2023-11-24T08:08:49.353690+0000","last_became_active":"2023-11-24T08:08:47.154139+0000","last_became_peered":"2023-11-24T08:08:47.154139+0000","last_unstale":"2023-11-24T08:08:49.353690+0000","last_undegraded":"2023-11-24T08:08:49.353690+0000","last_fullsized":"2023-11-24T08:08:49.353690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551743+0000","last_change":"2023-11-24T08:08:48.157139+0000","last_active":"2023-11-24T08:08:50.551743+0000","last_peered":"2023-11-24T08:08:50.551743+0000","last_clean":"2023-11-24T08:08:50.551743+0000","last_became_active":"2023-11-24T08:08:48.156875+0000","last_became_peered":"2023-11-24T08:08:48.156875+0000","last_unstale":"2023-11-24T08:08:50.551743+0000","last_undegraded":"2023-11-24T08:08:50.551743+0000","last_fullsized":"2023-11-24T08:08:50.551743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.345355+0000","last_change":"2023-11-24T08:08:47.143443+0000","last_active":"2023-11-24T08:08:50.345355+0000","last_peered":"2023-11-24T08:08:50.345355+0000","last_clean":"2023-11-24T08:08:50.345355+0000","last_became_active":"2023-11-24T08:08:47.143156+0000","last_became_peered":"2023-11-24T08:08:47.143156+0000","last_unstale":"2023-11-24T08:08:50.345355+0000","last_undegraded":"2023-11-24T08:08:50.345355+0000","last_fullsized":"2023-11-24T08:08:50.345355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448871+0000","last_change":"2023-11-24T08:08:48.141063+0000","last_active":"2023-11-24T08:08:50.448871+0000","last_peered":"2023-11-24T08:08:50.448871+0000","last_clean":"2023-11-24T08:08:50.448871+0000","last_became_active":"2023-11-24T08:08:48.140809+0000","last_became_peered":"2023-11-24T08:08:48.140809+0000","last_unstale":"2023-11-24T08:08:50.448871+0000","last_undegraded":"2023-11-24T08:08:50.448871+0000","last_fullsized":"2023-11-24T08:08:50.448871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:19.645688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.342687+0000","last_change":"2023-11-24T08:08:47.150905+0000","last_active":"2023-11-24T08:08:50.342687+0000","last_peered":"2023-11-24T08:08:50.342687+0000","last_clean":"2023-11-24T08:08:50.342687+0000","last_became_active":"2023-11-24T08:08:47.150670+0000","last_became_peered":"2023-11-24T08:08:47.150670+0000","last_unstale":"2023-11-24T08:08:50.342687+0000","last_undegraded":"2023-11-24T08:08:50.342687+0000","last_fullsized":"2023-11-24T08:08:50.342687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.944}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.952}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.218}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.978}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1339999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4300000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0659999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.907}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":831488,"data_stored":669882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:58.022 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:08:58.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.266+0000 7fd715aa3640 1 Processor -- start 2023-11-24T08:08:58.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.266+0000 7fd715aa3640 1 -- start start 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd715aa3640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd710144b30 0x7fd71015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd715aa3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd71015c410 0x7fd7100b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd715aa3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd7100b1de0 0x7fd7100b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd715aa3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100b27b0 con 0x7fd7100b1de0 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd715aa3640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71015bf80 con 0x7fd710144b30 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd715aa3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100b1560 con 0x7fd71015c410 2023-11-24T08:08:58.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd7100b1de0 0x7fd7100b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd7100b1de0 0x7fd7100b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34538/0 (socket says 172.21.15.106:34538) 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 -- 172.21.15.106:0/773075337 learned_addr learned my addr 172.21.15.106:0/773075337 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd714aa1640 1 --2- 172.21.15.106:0/773075337 >> v2:172.21.15.156:3300/0 conn(0x7fd710144b30 0x7fd71015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd70ffff640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd71015c410 0x7fd7100b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 -- 172.21.15.106:0/773075337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd71015c410 msgr2=0x7fd7100b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd71015c410 0x7fd7100b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 -- 172.21.15.106:0/773075337 >> v2:172.21.15.156:3300/0 conn(0x7fd710144b30 msgr2=0x7fd71015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 --2- 172.21.15.106:0/773075337 >> v2:172.21.15.156:3300/0 conn(0x7fd710144b30 0x7fd71015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd7152a2640 1 -- 172.21.15.106:0/773075337 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100b0fb0 con 0x7fd7100b1de0 2023-11-24T08:08:58.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.268+0000 7fd70ffff640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd71015c410 0x7fd7100b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:58.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.269+0000 7fd7152a2640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7100b1de0 0x7fd7100b2270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd71006f690 tx=0x7fd70802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560a39d29a8ccd33 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd70f7fe640 1 -- 172.21.15.106:0/773075337 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70802ca70 con 0x7fd7100b1de0 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd70f7fe640 1 -- 172.21.15.106:0/773075337 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70802cbd0 con 0x7fd7100b1de0 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd70f7fe640 1 -- 172.21.15.106:0/773075337 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7080343f0 con 0x7fd7100b1de0 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 -- 172.21.15.106:0/773075337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7100b1de0 msgr2=0x7fd7100b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7100b1de0 0x7fd7100b2270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fd71006f690 tx=0x7fd70802bed0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 -- 172.21.15.106:0/773075337 shutdown_connections 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7100b1de0 0x7fd7100b2270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 --2- 172.21.15.106:0/773075337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd71015c410 0x7fd7100b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 --2- 172.21.15.106:0/773075337 >> v2:172.21.15.156:3300/0 conn(0x7fd710144b30 0x7fd71015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.270+0000 7fd715aa3640 1 -- 172.21.15.106:0/773075337 >> 172.21.15.106:0/773075337 conn(0x7fd7100cb830 msgr2=0x7fd7100cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.271+0000 7fd715aa3640 1 -- 172.21.15.106:0/773075337 shutdown_connections 2023-11-24T08:08:58.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.271+0000 7fd715aa3640 1 -- 172.21.15.106:0/773075337 wait complete. 2023-11-24T08:08:58.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.272+0000 7fd715aa3640 1 Processor -- start 2023-11-24T08:08:58.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.272+0000 7fd715aa3640 1 -- start start 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd715aa3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7100b1de0 0x7fd7101f24c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd715aa3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd710144b30 0x7fd710136300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd715aa3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 0x7fd710136840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd715aa3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100712e0 con 0x7fd71015c410 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd715aa3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100715c0 con 0x7fd7100b1de0 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd715aa3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100718a0 con 0x7fd710144b30 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd7152a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 0x7fd710136840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd7152a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 0x7fd710136840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34552/0 (socket says 172.21.15.106:34552) 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.273+0000 7fd7152a2640 1 -- 172.21.15.106:0/2731705951 learned_addr learned my addr 172.21.15.106:0/2731705951 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd714aa1640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7100b1de0 0x7fd7101f24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd7152a2640 1 -- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd710144b30 msgr2=0x7fd710136300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd7152a2640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd710144b30 0x7fd710136300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd7152a2640 1 -- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7100b1de0 msgr2=0x7fd7101f24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd7152a2640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7100b1de0 0x7fd7101f24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd7152a2640 1 -- 172.21.15.106:0/2731705951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd708009b30 con 0x7fd71015c410 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd714aa1640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7100b1de0 0x7fd7101f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd7152a2640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 0x7fd710136840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd708036040 tx=0x7fd708004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd708009510 con 0x7fd71015c410 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70803e450 con 0x7fd71015c410 2023-11-24T08:08:58.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd710071b80 con 0x7fd71015c410 2023-11-24T08:08:58.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.274+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70803e7e0 con 0x7fd71015c410 2023-11-24T08:08:58.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.275+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7101f2a00 con 0x7fd71015c410 2023-11-24T08:08:58.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.276+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd71014c420 con 0x7fd71015c410 2023-11-24T08:08:58.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.276+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd70803e940 con 0x7fd71015c410 2023-11-24T08:08:58.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.278+0000 7fd70d7fa640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd6f406ae40 0x7fd6f406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.278+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7080ac430 con 0x7fd71015c410 2023-11-24T08:08:58.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.278+0000 7fd714aa1640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd6f406ae40 0x7fd6f406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.278+0000 7fd714aa1640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd6f406ae40 0x7fd6f406d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.282+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd71014c420 con 0x7fd71015c410 2023-11-24T08:08:58.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.459+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd71015bc10 con 0x7fd6f406ae40 2023-11-24T08:08:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.467+0000 7fd70d7fa640 1 -- 172.21.15.106:0/2731705951 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264263 (secure 0 0 0) 0x7fd704009d90 con 0x7fd6f406ae40 2023-11-24T08:08:58.473 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:58.474 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd6f406ae40 msgr2=0x7fd6f406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd6f406ae40 0x7fd6f406d300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd704006fd0 tx=0x7fd704008040 comp rx=0 tx=0).stop 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 msgr2=0x7fd710136840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 0x7fd710136840 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd708036040 tx=0x7fd708004a40 comp rx=0 tx=0).stop 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 shutdown_connections 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd71015c410 0x7fd710136840 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd6f406ae40 0x7fd6f406d300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd710144b30 0x7fd710136300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 --2- 172.21.15.106:0/2731705951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd7100b1de0 0x7fd7101f24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 >> 172.21.15.106:0/2731705951 conn(0x7fd7100cb830 msgr2=0x7fd71015c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 shutdown_connections 2023-11-24T08:08:58.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.472+0000 7fd715aa3640 1 -- 172.21.15.106:0/2731705951 wait complete. 2023-11-24T08:08:58.493 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-24T08:08:57.277112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235936,"kb_used_data":2984,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738784,"statfs":{"total":773094113280,"available":772852514816,"internally_reserved":0,"allocated":3055616,"data_stored":1816564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":59,"num_write_kb":276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.159591"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354084+0000","last_change":"2023-11-24T08:08:48.155148+0000","last_active":"2023-11-24T08:08:49.354084+0000","last_peered":"2023-11-24T08:08:49.354084+0000","last_clean":"2023-11-24T08:08:49.354084+0000","last_became_active":"2023-11-24T08:08:48.148725+0000","last_became_peered":"2023-11-24T08:08:48.148725+0000","last_unstale":"2023-11-24T08:08:49.354084+0000","last_undegraded":"2023-11-24T08:08:49.354084+0000","last_fullsized":"2023-11-24T08:08:49.354084+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:04:46.218050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126326+0000","last_change":"2023-11-24T08:08:48.153462+0000","last_active":"2023-11-24T08:08:49.126326+0000","last_peered":"2023-11-24T08:08:49.126326+0000","last_clean":"2023-11-24T08:08:49.126326+0000","last_became_active":"2023-11-24T08:08:48.153009+0000","last_became_peered":"2023-11-24T08:08:48.153009+0000","last_unstale":"2023-11-24T08:08:49.126326+0000","last_undegraded":"2023-11-24T08:08:49.126326+0000","last_fullsized":"2023-11-24T08:08:49.126326+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552096+0000","last_change":"2023-11-24T08:08:48.147649+0000","last_active":"2023-11-24T08:08:50.552096+0000","last_peered":"2023-11-24T08:08:50.552096+0000","last_clean":"2023-11-24T08:08:50.552096+0000","last_became_active":"2023-11-24T08:08:48.147280+0000","last_became_peered":"2023-11-24T08:08:48.147280+0000","last_unstale":"2023-11-24T08:08:50.552096+0000","last_undegraded":"2023-11-24T08:08:50.552096+0000","last_fullsized":"2023-11-24T08:08:50.552096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126469+0000","last_change":"2023-11-24T08:08:48.158446+0000","last_active":"2023-11-24T08:08:49.126469+0000","last_peered":"2023-11-24T08:08:49.126469+0000","last_clean":"2023-11-24T08:08:49.126469+0000","last_became_active":"2023-11-24T08:08:48.158196+0000","last_became_peered":"2023-11-24T08:08:48.158196+0000","last_unstale":"2023-11-24T08:08:49.126469+0000","last_undegraded":"2023-11-24T08:08:49.126469+0000","last_fullsized":"2023-11-24T08:08:49.126469+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:58.752582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353187+0000","last_change":"2023-11-24T08:08:48.148128+0000","last_active":"2023-11-24T08:08:49.353187+0000","last_peered":"2023-11-24T08:08:49.353187+0000","last_clean":"2023-11-24T08:08:49.353187+0000","last_became_active":"2023-11-24T08:08:48.147789+0000","last_became_peered":"2023-11-24T08:08:48.147789+0000","last_unstale":"2023-11-24T08:08:49.353187+0000","last_undegraded":"2023-11-24T08:08:49.353187+0000","last_fullsized":"2023-11-24T08:08:49.353187+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:17.852217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126840+0000","last_change":"2023-11-24T08:08:48.160813+0000","last_active":"2023-11-24T08:08:49.126840+0000","last_peered":"2023-11-24T08:08:49.126840+0000","last_clean":"2023-11-24T08:08:49.126840+0000","last_became_active":"2023-11-24T08:08:48.159970+0000","last_became_peered":"2023-11-24T08:08:48.159970+0000","last_unstale":"2023-11-24T08:08:49.126840+0000","last_undegraded":"2023-11-24T08:08:49.126840+0000","last_fullsized":"2023-11-24T08:08:49.126840+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:29:43.997384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352936+0000","last_change":"2023-11-24T08:08:48.147161+0000","last_active":"2023-11-24T08:08:49.352936+0000","last_peered":"2023-11-24T08:08:49.352936+0000","last_clean":"2023-11-24T08:08:49.352936+0000","last_became_active":"2023-11-24T08:08:48.146867+0000","last_became_peered":"2023-11-24T08:08:48.146867+0000","last_unstale":"2023-11-24T08:08:49.352936+0000","last_undegraded":"2023-11-24T08:08:49.352936+0000","last_fullsized":"2023-11-24T08:08:49.352936+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:22:50.592771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447578+0000","last_change":"2023-11-24T08:08:48.151256+0000","last_active":"2023-11-24T08:08:50.447578+0000","last_peered":"2023-11-24T08:08:50.447578+0000","last_clean":"2023-11-24T08:08:50.447578+0000","last_became_active":"2023-11-24T08:08:48.150977+0000","last_became_peered":"2023-11-24T08:08:48.150977+0000","last_unstale":"2023-11-24T08:08:50.447578+0000","last_undegraded":"2023-11-24T08:08:50.447578+0000","last_fullsized":"2023-11-24T08:08:50.447578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354145+0000","last_change":"2023-11-24T08:08:48.165291+0000","last_active":"2023-11-24T08:08:49.354145+0000","last_peered":"2023-11-24T08:08:49.354145+0000","last_clean":"2023-11-24T08:08:49.354145+0000","last_became_active":"2023-11-24T08:08:48.164956+0000","last_became_peered":"2023-11-24T08:08:48.164956+0000","last_unstale":"2023-11-24T08:08:49.354145+0000","last_undegraded":"2023-11-24T08:08:49.354145+0000","last_fullsized":"2023-11-24T08:08:49.354145+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:32.727731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125355+0000","last_change":"2023-11-24T08:08:48.159627+0000","last_active":"2023-11-24T08:08:49.125355+0000","last_peered":"2023-11-24T08:08:49.125355+0000","last_clean":"2023-11-24T08:08:49.125355+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:49.125355+0000","last_undegraded":"2023-11-24T08:08:49.125355+0000","last_fullsized":"2023-11-24T08:08:49.125355+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125146+0000","last_change":"2023-11-24T08:08:48.157396+0000","last_active":"2023-11-24T08:08:49.125146+0000","last_peered":"2023-11-24T08:08:49.125146+0000","last_clean":"2023-11-24T08:08:49.125146+0000","last_became_active":"2023-11-24T08:08:48.155190+0000","last_became_peered":"2023-11-24T08:08:48.155190+0000","last_unstale":"2023-11-24T08:08:49.125146+0000","last_undegraded":"2023-11-24T08:08:49.125146+0000","last_fullsized":"2023-11-24T08:08:49.125146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107513+0000","last_change":"2023-11-24T08:08:48.139798+0000","last_active":"2023-11-24T08:08:50.107513+0000","last_peered":"2023-11-24T08:08:50.107513+0000","last_clean":"2023-11-24T08:08:50.107513+0000","last_became_active":"2023-11-24T08:08:48.139186+0000","last_became_peered":"2023-11-24T08:08:48.139186+0000","last_unstale":"2023-11-24T08:08:50.107513+0000","last_undegraded":"2023-11-24T08:08:50.107513+0000","last_fullsized":"2023-11-24T08:08:50.107513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:26.389558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448422+0000","last_change":"2023-11-24T08:08:48.141667+0000","last_active":"2023-11-24T08:08:50.448422+0000","last_peered":"2023-11-24T08:08:50.448422+0000","last_clean":"2023-11-24T08:08:50.448422+0000","last_became_active":"2023-11-24T08:08:48.141410+0000","last_became_peered":"2023-11-24T08:08:48.141410+0000","last_unstale":"2023-11-24T08:08:50.448422+0000","last_undegraded":"2023-11-24T08:08:50.448422+0000","last_fullsized":"2023-11-24T08:08:50.448422+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:54.347050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551315+0000","last_change":"2023-11-24T08:08:48.154158+0000","last_active":"2023-11-24T08:08:50.551315+0000","last_peered":"2023-11-24T08:08:50.551315+0000","last_clean":"2023-11-24T08:08:50.551315+0000","last_became_active":"2023-11-24T08:08:48.153833+0000","last_became_peered":"2023-11-24T08:08:48.153833+0000","last_unstale":"2023-11-24T08:08:50.551315+0000","last_undegraded":"2023-11-24T08:08:50.551315+0000","last_fullsized":"2023-11-24T08:08:50.551315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:00.726435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125503+0000","last_change":"2023-11-24T08:08:48.130056+0000","last_active":"2023-11-24T08:08:49.125503+0000","last_peered":"2023-11-24T08:08:49.125503+0000","last_clean":"2023-11-24T08:08:49.125503+0000","last_became_active":"2023-11-24T08:08:48.129739+0000","last_became_peered":"2023-11-24T08:08:48.129739+0000","last_unstale":"2023-11-24T08:08:49.125503+0000","last_undegraded":"2023-11-24T08:08:49.125503+0000","last_fullsized":"2023-11-24T08:08:49.125503+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125661+0000","last_change":"2023-11-24T08:08:48.157064+0000","last_active":"2023-11-24T08:08:49.125661+0000","last_peered":"2023-11-24T08:08:49.125661+0000","last_clean":"2023-11-24T08:08:49.125661+0000","last_became_active":"2023-11-24T08:08:48.153014+0000","last_became_peered":"2023-11-24T08:08:48.153014+0000","last_unstale":"2023-11-24T08:08:49.125661+0000","last_undegraded":"2023-11-24T08:08:49.125661+0000","last_fullsized":"2023-11-24T08:08:49.125661+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:53.228655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128174+0000","last_change":"2023-11-24T08:08:48.168538+0000","last_active":"2023-11-24T08:08:49.128174+0000","last_peered":"2023-11-24T08:08:49.128174+0000","last_clean":"2023-11-24T08:08:49.128174+0000","last_became_active":"2023-11-24T08:08:48.168251+0000","last_became_peered":"2023-11-24T08:08:48.168251+0000","last_unstale":"2023-11-24T08:08:49.128174+0000","last_undegraded":"2023-11-24T08:08:49.128174+0000","last_fullsized":"2023-11-24T08:08:49.128174+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:10:34.330436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:51.205453+0000","last_change":"2023-11-24T08:08:48.135832+0000","last_active":"2023-11-24T08:08:51.205453+0000","last_peered":"2023-11-24T08:08:51.205453+0000","last_clean":"2023-11-24T08:08:51.205453+0000","last_became_active":"2023-11-24T08:08:48.135419+0000","last_became_peered":"2023-11-24T08:08:48.135419+0000","last_unstale":"2023-11-24T08:08:51.205453+0000","last_undegraded":"2023-11-24T08:08:51.205453+0000","last_fullsized":"2023-11-24T08:08:51.205453+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353897+0000","last_change":"2023-11-24T08:08:48.148026+0000","last_active":"2023-11-24T08:08:49.353897+0000","last_peered":"2023-11-24T08:08:49.353897+0000","last_clean":"2023-11-24T08:08:49.353897+0000","last_became_active":"2023-11-24T08:08:48.147722+0000","last_became_peered":"2023-11-24T08:08:48.147722+0000","last_unstale":"2023-11-24T08:08:49.353897+0000","last_undegraded":"2023-11-24T08:08:49.353897+0000","last_fullsized":"2023-11-24T08:08:49.353897+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:15:57.275173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128304+0000","last_change":"2023-11-24T08:08:48.145897+0000","last_active":"2023-11-24T08:08:49.128304+0000","last_peered":"2023-11-24T08:08:49.128304+0000","last_clean":"2023-11-24T08:08:49.128304+0000","last_became_active":"2023-11-24T08:08:48.145620+0000","last_became_peered":"2023-11-24T08:08:48.145620+0000","last_unstale":"2023-11-24T08:08:49.128304+0000","last_undegraded":"2023-11-24T08:08:49.128304+0000","last_fullsized":"2023-11-24T08:08:49.128304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551787+0000","last_change":"2023-11-24T08:08:48.164711+0000","last_active":"2023-11-24T08:08:50.551787+0000","last_peered":"2023-11-24T08:08:50.551787+0000","last_clean":"2023-11-24T08:08:50.551787+0000","last_became_active":"2023-11-24T08:08:48.164331+0000","last_became_peered":"2023-11-24T08:08:48.164331+0000","last_unstale":"2023-11-24T08:08:50.551787+0000","last_undegraded":"2023-11-24T08:08:50.551787+0000","last_fullsized":"2023-11-24T08:08:50.551787+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127224+0000","last_change":"2023-11-24T08:08:48.148716+0000","last_active":"2023-11-24T08:08:49.127224+0000","last_peered":"2023-11-24T08:08:49.127224+0000","last_clean":"2023-11-24T08:08:49.127224+0000","last_became_active":"2023-11-24T08:08:48.148271+0000","last_became_peered":"2023-11-24T08:08:48.148271+0000","last_unstale":"2023-11-24T08:08:49.127224+0000","last_undegraded":"2023-11-24T08:08:49.127224+0000","last_fullsized":"2023-11-24T08:08:49.127224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:14.407973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447763+0000","last_change":"2023-11-24T08:08:48.156383+0000","last_active":"2023-11-24T08:08:50.447763+0000","last_peered":"2023-11-24T08:08:50.447763+0000","last_clean":"2023-11-24T08:08:50.447763+0000","last_became_active":"2023-11-24T08:08:48.156083+0000","last_became_peered":"2023-11-24T08:08:48.156083+0000","last_unstale":"2023-11-24T08:08:50.447763+0000","last_undegraded":"2023-11-24T08:08:50.447763+0000","last_fullsized":"2023-11-24T08:08:50.447763+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126030+0000","last_change":"2023-11-24T08:08:48.157282+0000","last_active":"2023-11-24T08:08:49.126030+0000","last_peered":"2023-11-24T08:08:49.126030+0000","last_clean":"2023-11-24T08:08:49.126030+0000","last_became_active":"2023-11-24T08:08:48.155003+0000","last_became_peered":"2023-11-24T08:08:48.155003+0000","last_unstale":"2023-11-24T08:08:49.126030+0000","last_undegraded":"2023-11-24T08:08:49.126030+0000","last_fullsized":"2023-11-24T08:08:49.126030+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:07:02.540136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448748+0000","last_change":"2023-11-24T08:08:48.146014+0000","last_active":"2023-11-24T08:08:50.448748+0000","last_peered":"2023-11-24T08:08:50.448748+0000","last_clean":"2023-11-24T08:08:50.448748+0000","last_became_active":"2023-11-24T08:08:48.144834+0000","last_became_peered":"2023-11-24T08:08:48.144834+0000","last_unstale":"2023-11-24T08:08:50.448748+0000","last_undegraded":"2023-11-24T08:08:50.448748+0000","last_fullsized":"2023-11-24T08:08:50.448748+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:22:50.592771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125567+0000","last_change":"2023-11-24T08:08:48.142247+0000","last_active":"2023-11-24T08:08:49.125567+0000","last_peered":"2023-11-24T08:08:49.125567+0000","last_clean":"2023-11-24T08:08:49.125567+0000","last_became_active":"2023-11-24T08:08:48.141969+0000","last_became_peered":"2023-11-24T08:08:48.141969+0000","last_unstale":"2023-11-24T08:08:49.125567+0000","last_undegraded":"2023-11-24T08:08:49.125567+0000","last_fullsized":"2023-11-24T08:08:49.125567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126146+0000","last_change":"2023-11-24T08:08:48.134993+0000","last_active":"2023-11-24T08:08:49.126146+0000","last_peered":"2023-11-24T08:08:49.126146+0000","last_clean":"2023-11-24T08:08:49.126146+0000","last_became_active":"2023-11-24T08:08:48.134752+0000","last_became_peered":"2023-11-24T08:08:48.134752+0000","last_unstale":"2023-11-24T08:08:49.126146+0000","last_undegraded":"2023-11-24T08:08:49.126146+0000","last_fullsized":"2023-11-24T08:08:49.126146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448933+0000","last_change":"2023-11-24T08:08:48.147386+0000","last_active":"2023-11-24T08:08:50.448933+0000","last_peered":"2023-11-24T08:08:50.448933+0000","last_clean":"2023-11-24T08:08:50.448933+0000","last_became_active":"2023-11-24T08:08:48.144941+0000","last_became_peered":"2023-11-24T08:08:48.144941+0000","last_unstale":"2023-11-24T08:08:50.448933+0000","last_undegraded":"2023-11-24T08:08:50.448933+0000","last_fullsized":"2023-11-24T08:08:50.448933+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:17.852217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552015+0000","last_change":"2023-11-24T08:08:48.153360+0000","last_active":"2023-11-24T08:08:50.552015+0000","last_peered":"2023-11-24T08:08:50.552015+0000","last_clean":"2023-11-24T08:08:50.552015+0000","last_became_active":"2023-11-24T08:08:48.153099+0000","last_became_peered":"2023-11-24T08:08:48.153099+0000","last_unstale":"2023-11-24T08:08:50.552015+0000","last_undegraded":"2023-11-24T08:08:50.552015+0000","last_fullsized":"2023-11-24T08:08:50.552015+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127574+0000","last_change":"2023-11-24T08:08:48.156712+0000","last_active":"2023-11-24T08:08:49.127574+0000","last_peered":"2023-11-24T08:08:49.127574+0000","last_clean":"2023-11-24T08:08:49.127574+0000","last_became_active":"2023-11-24T08:08:48.151074+0000","last_became_peered":"2023-11-24T08:08:48.151074+0000","last_unstale":"2023-11-24T08:08:49.127574+0000","last_undegraded":"2023-11-24T08:08:49.127574+0000","last_fullsized":"2023-11-24T08:08:49.127574+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:37:46.079373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:51.205305+0000","last_change":"2023-11-24T08:08:48.149394+0000","last_active":"2023-11-24T08:08:51.205305+0000","last_peered":"2023-11-24T08:08:51.205305+0000","last_clean":"2023-11-24T08:08:51.205305+0000","last_became_active":"2023-11-24T08:08:48.149158+0000","last_became_peered":"2023-11-24T08:08:48.149158+0000","last_unstale":"2023-11-24T08:08:51.205305+0000","last_undegraded":"2023-11-24T08:08:51.205305+0000","last_fullsized":"2023-11-24T08:08:51.205305+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124630+0000","last_change":"2023-11-24T08:08:48.155627+0000","last_active":"2023-11-24T08:08:49.124630+0000","last_peered":"2023-11-24T08:08:49.124630+0000","last_clean":"2023-11-24T08:08:49.124630+0000","last_became_active":"2023-11-24T08:08:48.155343+0000","last_became_peered":"2023-11-24T08:08:48.155343+0000","last_unstale":"2023-11-24T08:08:49.124630+0000","last_undegraded":"2023-11-24T08:08:49.124630+0000","last_fullsized":"2023-11-24T08:08:49.124630+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:30:00.726435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107026+0000","last_change":"2023-11-24T08:08:48.141457+0000","last_active":"2023-11-24T08:08:50.107026+0000","last_peered":"2023-11-24T08:08:50.107026+0000","last_clean":"2023-11-24T08:08:50.107026+0000","last_became_active":"2023-11-24T08:08:48.139546+0000","last_became_peered":"2023-11-24T08:08:48.139546+0000","last_unstale":"2023-11-24T08:08:50.107026+0000","last_undegraded":"2023-11-24T08:08:50.107026+0000","last_fullsized":"2023-11-24T08:08:50.107026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:46.574140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.345213+0000","last_change":"2023-11-24T08:08:47.144970+0000","last_active":"2023-11-24T08:08:50.345213+0000","last_peered":"2023-11-24T08:08:50.345213+0000","last_clean":"2023-11-24T08:08:50.345213+0000","last_became_active":"2023-11-24T08:08:47.144667+0000","last_became_peered":"2023-11-24T08:08:47.144667+0000","last_unstale":"2023-11-24T08:08:50.345213+0000","last_undegraded":"2023-11-24T08:08:50.345213+0000","last_fullsized":"2023-11-24T08:08:50.345213+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:59.483267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552207+0000","last_change":"2023-11-24T08:08:48.158059+0000","last_active":"2023-11-24T08:08:50.552207+0000","last_peered":"2023-11-24T08:08:50.552207+0000","last_clean":"2023-11-24T08:08:50.552207+0000","last_became_active":"2023-11-24T08:08:48.157743+0000","last_became_peered":"2023-11-24T08:08:48.157743+0000","last_unstale":"2023-11-24T08:08:50.552207+0000","last_undegraded":"2023-11-24T08:08:50.552207+0000","last_fullsized":"2023-11-24T08:08:50.552207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:08:54.347050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125078+0000","last_change":"2023-11-24T08:08:47.135247+0000","last_active":"2023-11-24T08:08:49.125078+0000","last_peered":"2023-11-24T08:08:49.125078+0000","last_clean":"2023-11-24T08:08:49.125078+0000","last_became_active":"2023-11-24T08:08:47.134923+0000","last_became_peered":"2023-11-24T08:08:47.134923+0000","last_unstale":"2023-11-24T08:08:49.125078+0000","last_undegraded":"2023-11-24T08:08:49.125078+0000","last_fullsized":"2023-11-24T08:08:49.125078+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.345062+0000","last_change":"2023-11-24T08:08:47.151833+0000","last_active":"2023-11-24T08:08:50.345062+0000","last_peered":"2023-11-24T08:08:50.345062+0000","last_clean":"2023-11-24T08:08:50.345062+0000","last_became_active":"2023-11-24T08:08:47.151517+0000","last_became_peered":"2023-11-24T08:08:47.151517+0000","last_unstale":"2023-11-24T08:08:50.345062+0000","last_undegraded":"2023-11-24T08:08:50.345062+0000","last_fullsized":"2023-11-24T08:08:50.345062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447758+0000","last_change":"2023-11-24T08:08:48.148262+0000","last_active":"2023-11-24T08:08:50.447758+0000","last_peered":"2023-11-24T08:08:50.447758+0000","last_clean":"2023-11-24T08:08:50.447758+0000","last_became_active":"2023-11-24T08:08:48.147874+0000","last_became_peered":"2023-11-24T08:08:48.147874+0000","last_unstale":"2023-11-24T08:08:50.447758+0000","last_undegraded":"2023-11-24T08:08:50.447758+0000","last_fullsized":"2023-11-24T08:08:50.447758+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126262+0000","last_change":"2023-11-24T08:08:47.144280+0000","last_active":"2023-11-24T08:08:49.126262+0000","last_peered":"2023-11-24T08:08:49.126262+0000","last_clean":"2023-11-24T08:08:49.126262+0000","last_became_active":"2023-11-24T08:08:47.143933+0000","last_became_peered":"2023-11-24T08:08:47.143933+0000","last_unstale":"2023-11-24T08:08:49.126262+0000","last_undegraded":"2023-11-24T08:08:49.126262+0000","last_fullsized":"2023-11-24T08:08:49.126262+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:17:28.820828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125379+0000","last_change":"2023-11-24T08:08:48.151024+0000","last_active":"2023-11-24T08:08:49.125379+0000","last_peered":"2023-11-24T08:08:49.125379+0000","last_clean":"2023-11-24T08:08:49.125379+0000","last_became_active":"2023-11-24T08:08:48.150727+0000","last_became_peered":"2023-11-24T08:08:48.150727+0000","last_unstale":"2023-11-24T08:08:49.125379+0000","last_undegraded":"2023-11-24T08:08:49.125379+0000","last_fullsized":"2023-11-24T08:08:49.125379+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127936+0000","last_change":"2023-11-24T08:08:47.158190+0000","last_active":"2023-11-24T08:08:49.127936+0000","last_peered":"2023-11-24T08:08:49.127936+0000","last_clean":"2023-11-24T08:08:49.127936+0000","last_became_active":"2023-11-24T08:08:47.157894+0000","last_became_peered":"2023-11-24T08:08:47.157894+0000","last_unstale":"2023-11-24T08:08:49.127936+0000","last_undegraded":"2023-11-24T08:08:49.127936+0000","last_fullsized":"2023-11-24T08:08:49.127936+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:18.739564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127729+0000","last_change":"2023-11-24T08:08:48.152122+0000","last_active":"2023-11-24T08:08:49.127729+0000","last_peered":"2023-11-24T08:08:49.127729+0000","last_clean":"2023-11-24T08:08:49.127729+0000","last_became_active":"2023-11-24T08:08:48.151509+0000","last_became_peered":"2023-11-24T08:08:48.151509+0000","last_unstale":"2023-11-24T08:08:49.127729+0000","last_undegraded":"2023-11-24T08:08:49.127729+0000","last_fullsized":"2023-11-24T08:08:49.127729+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:17.382456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127836+0000","last_change":"2023-11-24T08:08:47.154457+0000","last_active":"2023-11-24T08:08:49.127836+0000","last_peered":"2023-11-24T08:08:49.127836+0000","last_clean":"2023-11-24T08:08:49.127836+0000","last_became_active":"2023-11-24T08:08:47.154137+0000","last_became_peered":"2023-11-24T08:08:47.154137+0000","last_unstale":"2023-11-24T08:08:49.127836+0000","last_undegraded":"2023-11-24T08:08:49.127836+0000","last_fullsized":"2023-11-24T08:08:49.127836+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:29:19.309577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.552125+0000","last_change":"2023-11-24T08:08:48.154269+0000","last_active":"2023-11-24T08:08:50.552125+0000","last_peered":"2023-11-24T08:08:50.552125+0000","last_clean":"2023-11-24T08:08:50.552125+0000","last_became_active":"2023-11-24T08:08:48.153873+0000","last_became_peered":"2023-11-24T08:08:48.153873+0000","last_unstale":"2023-11-24T08:08:50.552125+0000","last_undegraded":"2023-11-24T08:08:50.552125+0000","last_fullsized":"2023-11-24T08:08:50.552125+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:30.453859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"21'3","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126169+0000","last_change":"2023-11-24T08:08:47.156467+0000","last_active":"2023-11-24T08:08:49.126169+0000","last_peered":"2023-11-24T08:08:49.126169+0000","last_clean":"2023-11-24T08:08:49.126169+0000","last_became_active":"2023-11-24T08:08:47.155999+0000","last_became_peered":"2023-11-24T08:08:47.155999+0000","last_unstale":"2023-11-24T08:08:49.126169+0000","last_undegraded":"2023-11-24T08:08:49.126169+0000","last_fullsized":"2023-11-24T08:08:49.126169+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107764+0000","last_change":"2023-11-24T08:08:48.142662+0000","last_active":"2023-11-24T08:08:50.107764+0000","last_peered":"2023-11-24T08:08:50.107764+0000","last_clean":"2023-11-24T08:08:50.107764+0000","last_became_active":"2023-11-24T08:08:48.139505+0000","last_became_peered":"2023-11-24T08:08:48.139505+0000","last_unstale":"2023-11-24T08:08:50.107764+0000","last_undegraded":"2023-11-24T08:08:50.107764+0000","last_fullsized":"2023-11-24T08:08:50.107764+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:01.498497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"21'6","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125359+0000","last_change":"2023-11-24T08:08:47.148708+0000","last_active":"2023-11-24T08:08:49.125359+0000","last_peered":"2023-11-24T08:08:49.125359+0000","last_clean":"2023-11-24T08:08:49.125359+0000","last_became_active":"2023-11-24T08:08:47.148386+0000","last_became_peered":"2023-11-24T08:08:47.148386+0000","last_unstale":"2023-11-24T08:08:49.125359+0000","last_undegraded":"2023-11-24T08:08:49.125359+0000","last_fullsized":"2023-11-24T08:08:49.125359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353438+0000","last_change":"2023-11-24T08:08:48.147343+0000","last_active":"2023-11-24T08:08:49.353438+0000","last_peered":"2023-11-24T08:08:49.353438+0000","last_clean":"2023-11-24T08:08:49.353438+0000","last_became_active":"2023-11-24T08:08:48.146625+0000","last_became_peered":"2023-11-24T08:08:48.146625+0000","last_unstale":"2023-11-24T08:08:49.353438+0000","last_undegraded":"2023-11-24T08:08:49.353438+0000","last_fullsized":"2023-11-24T08:08:49.353438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.269500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.342547+0000","last_change":"2023-11-24T08:08:47.141638+0000","last_active":"2023-11-24T08:08:50.342547+0000","last_peered":"2023-11-24T08:08:50.342547+0000","last_clean":"2023-11-24T08:08:50.342547+0000","last_became_active":"2023-11-24T08:08:47.141294+0000","last_became_peered":"2023-11-24T08:08:47.141294+0000","last_unstale":"2023-11-24T08:08:50.342547+0000","last_undegraded":"2023-11-24T08:08:50.342547+0000","last_fullsized":"2023-11-24T08:08:50.342547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551206+0000","last_change":"2023-11-24T08:08:48.159790+0000","last_active":"2023-11-24T08:08:50.551206+0000","last_peered":"2023-11-24T08:08:50.551206+0000","last_clean":"2023-11-24T08:08:50.551206+0000","last_became_active":"2023-11-24T08:08:48.159381+0000","last_became_peered":"2023-11-24T08:08:48.159381+0000","last_unstale":"2023-11-24T08:08:50.551206+0000","last_undegraded":"2023-11-24T08:08:50.551206+0000","last_fullsized":"2023-11-24T08:08:50.551206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:19.982732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127841+0000","last_change":"2023-11-24T08:08:47.157623+0000","last_active":"2023-11-24T08:08:49.127841+0000","last_peered":"2023-11-24T08:08:49.127841+0000","last_clean":"2023-11-24T08:08:49.127841+0000","last_became_active":"2023-11-24T08:08:47.156256+0000","last_became_peered":"2023-11-24T08:08:47.156256+0000","last_unstale":"2023-11-24T08:08:49.127841+0000","last_undegraded":"2023-11-24T08:08:49.127841+0000","last_fullsized":"2023-11-24T08:08:49.127841+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:59.304396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353649+0000","last_change":"2023-11-24T08:08:48.152586+0000","last_active":"2023-11-24T08:08:49.353649+0000","last_peered":"2023-11-24T08:08:49.353649+0000","last_clean":"2023-11-24T08:08:49.353649+0000","last_became_active":"2023-11-24T08:08:48.152296+0000","last_became_peered":"2023-11-24T08:08:48.152296+0000","last_unstale":"2023-11-24T08:08:49.353649+0000","last_undegraded":"2023-11-24T08:08:49.353649+0000","last_fullsized":"2023-11-24T08:08:49.353649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:26:23.916532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.124531+0000","last_change":"2023-11-24T08:08:47.136838+0000","last_active":"2023-11-24T08:08:49.124531+0000","last_peered":"2023-11-24T08:08:49.124531+0000","last_clean":"2023-11-24T08:08:49.124531+0000","last_became_active":"2023-11-24T08:08:47.136397+0000","last_became_peered":"2023-11-24T08:08:47.136397+0000","last_unstale":"2023-11-24T08:08:49.124531+0000","last_undegraded":"2023-11-24T08:08:49.124531+0000","last_fullsized":"2023-11-24T08:08:49.124531+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125520+0000","last_change":"2023-11-24T08:08:48.153870+0000","last_active":"2023-11-24T08:08:49.125520+0000","last_peered":"2023-11-24T08:08:49.125520+0000","last_clean":"2023-11-24T08:08:49.125520+0000","last_became_active":"2023-11-24T08:08:48.153566+0000","last_became_peered":"2023-11-24T08:08:48.153566+0000","last_unstale":"2023-11-24T08:08:49.125520+0000","last_undegraded":"2023-11-24T08:08:49.125520+0000","last_fullsized":"2023-11-24T08:08:49.125520+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"21'8","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351287+0000","last_change":"2023-11-24T08:08:47.133661+0000","last_active":"2023-11-24T08:08:50.351287+0000","last_peered":"2023-11-24T08:08:50.351287+0000","last_clean":"2023-11-24T08:08:50.351287+0000","last_became_active":"2023-11-24T08:08:47.133327+0000","last_became_peered":"2023-11-24T08:08:47.133327+0000","last_unstale":"2023-11-24T08:08:50.351287+0000","last_undegraded":"2023-11-24T08:08:50.351287+0000","last_fullsized":"2023-11-24T08:08:50.351287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:29:36.361920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352992+0000","last_change":"2023-11-24T08:08:48.159918+0000","last_active":"2023-11-24T08:08:49.352992+0000","last_peered":"2023-11-24T08:08:49.352992+0000","last_clean":"2023-11-24T08:08:49.352992+0000","last_became_active":"2023-11-24T08:08:48.159604+0000","last_became_peered":"2023-11-24T08:08:48.159604+0000","last_unstale":"2023-11-24T08:08:49.352992+0000","last_undegraded":"2023-11-24T08:08:49.352992+0000","last_fullsized":"2023-11-24T08:08:49.352992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:19.645688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107838+0000","last_change":"2023-11-24T08:08:47.139479+0000","last_active":"2023-11-24T08:08:50.107838+0000","last_peered":"2023-11-24T08:08:50.107838+0000","last_clean":"2023-11-24T08:08:50.107838+0000","last_became_active":"2023-11-24T08:08:47.139162+0000","last_became_peered":"2023-11-24T08:08:47.139162+0000","last_unstale":"2023-11-24T08:08:50.107838+0000","last_undegraded":"2023-11-24T08:08:50.107838+0000","last_fullsized":"2023-11-24T08:08:50.107838+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:51.916232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353918+0000","last_change":"2023-11-24T08:08:48.160119+0000","last_active":"2023-11-24T08:08:49.353918+0000","last_peered":"2023-11-24T08:08:49.353918+0000","last_clean":"2023-11-24T08:08:49.353918+0000","last_became_active":"2023-11-24T08:08:48.159806+0000","last_became_peered":"2023-11-24T08:08:48.159806+0000","last_unstale":"2023-11-24T08:08:49.353918+0000","last_undegraded":"2023-11-24T08:08:49.353918+0000","last_fullsized":"2023-11-24T08:08:49.353918+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:19.651759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"21'2","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351440+0000","last_change":"2023-11-24T08:08:47.139000+0000","last_active":"2023-11-24T08:08:50.351440+0000","last_peered":"2023-11-24T08:08:50.351440+0000","last_clean":"2023-11-24T08:08:50.351440+0000","last_became_active":"2023-11-24T08:08:47.138477+0000","last_became_peered":"2023-11-24T08:08:47.138477+0000","last_unstale":"2023-11-24T08:08:50.351440+0000","last_undegraded":"2023-11-24T08:08:50.351440+0000","last_fullsized":"2023-11-24T08:08:50.351440+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:49.285847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354329+0000","last_change":"2023-11-24T08:08:48.162632+0000","last_active":"2023-11-24T08:08:49.354329+0000","last_peered":"2023-11-24T08:08:49.354329+0000","last_clean":"2023-11-24T08:08:49.354329+0000","last_became_active":"2023-11-24T08:08:48.160274+0000","last_became_peered":"2023-11-24T08:08:48.160274+0000","last_unstale":"2023-11-24T08:08:49.354329+0000","last_undegraded":"2023-11-24T08:08:49.354329+0000","last_fullsized":"2023-11-24T08:08:49.354329+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:23.085689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"21'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126343+0000","last_change":"2023-11-24T08:08:47.133083+0000","last_active":"2023-11-24T08:08:49.126343+0000","last_peered":"2023-11-24T08:08:49.126343+0000","last_clean":"2023-11-24T08:08:49.126343+0000","last_became_active":"2023-11-24T08:08:47.132746+0000","last_became_peered":"2023-11-24T08:08:47.132746+0000","last_unstale":"2023-11-24T08:08:49.126343+0000","last_undegraded":"2023-11-24T08:08:49.126343+0000","last_fullsized":"2023-11-24T08:08:49.126343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:25.146390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126883+0000","last_change":"2023-11-24T08:08:48.151332+0000","last_active":"2023-11-24T08:08:49.126883+0000","last_peered":"2023-11-24T08:08:49.126883+0000","last_clean":"2023-11-24T08:08:49.126883+0000","last_became_active":"2023-11-24T08:08:48.151045+0000","last_became_peered":"2023-11-24T08:08:48.151045+0000","last_unstale":"2023-11-24T08:08:49.126883+0000","last_undegraded":"2023-11-24T08:08:49.126883+0000","last_fullsized":"2023-11-24T08:08:49.126883+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:22:15.646317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351397+0000","last_change":"2023-11-24T08:08:47.150523+0000","last_active":"2023-11-24T08:08:50.351397+0000","last_peered":"2023-11-24T08:08:50.351397+0000","last_clean":"2023-11-24T08:08:50.351397+0000","last_became_active":"2023-11-24T08:08:47.150208+0000","last_became_peered":"2023-11-24T08:08:47.150208+0000","last_unstale":"2023-11-24T08:08:50.351397+0000","last_undegraded":"2023-11-24T08:08:50.351397+0000","last_fullsized":"2023-11-24T08:08:50.351397+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353456+0000","last_change":"2023-11-24T08:08:48.156319+0000","last_active":"2023-11-24T08:08:49.353456+0000","last_peered":"2023-11-24T08:08:49.353456+0000","last_clean":"2023-11-24T08:08:49.353456+0000","last_became_active":"2023-11-24T08:08:48.154550+0000","last_became_peered":"2023-11-24T08:08:48.154550+0000","last_unstale":"2023-11-24T08:08:49.353456+0000","last_undegraded":"2023-11-24T08:08:49.353456+0000","last_fullsized":"2023-11-24T08:08:49.353456+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:38:17.806898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:48.135471+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:08:48.135471+0000","last_peered":"2023-11-24T08:08:48.135471+0000","last_clean":"2023-11-24T08:08:48.135471+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:08:48.135471+0000","last_undegraded":"2023-11-24T08:08:48.135471+0000","last_fullsized":"2023-11-24T08:08:48.135471+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125945+0000","last_change":"2023-11-24T08:08:47.157035+0000","last_active":"2023-11-24T08:08:49.125945+0000","last_peered":"2023-11-24T08:08:49.125945+0000","last_clean":"2023-11-24T08:08:49.125945+0000","last_became_active":"2023-11-24T08:08:47.155189+0000","last_became_peered":"2023-11-24T08:08:47.155189+0000","last_unstale":"2023-11-24T08:08:49.125945+0000","last_undegraded":"2023-11-24T08:08:49.125945+0000","last_fullsized":"2023-11-24T08:08:49.125945+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:54.597005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448128+0000","last_change":"2023-11-24T08:08:48.161239+0000","last_active":"2023-11-24T08:08:50.448128+0000","last_peered":"2023-11-24T08:08:50.448128+0000","last_clean":"2023-11-24T08:08:50.448128+0000","last_became_active":"2023-11-24T08:08:48.160530+0000","last_became_peered":"2023-11-24T08:08:48.160530+0000","last_unstale":"2023-11-24T08:08:50.448128+0000","last_undegraded":"2023-11-24T08:08:50.448128+0000","last_fullsized":"2023-11-24T08:08:50.448128+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:12:30.453859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353366+0000","last_change":"2023-11-24T08:08:47.153886+0000","last_active":"2023-11-24T08:08:49.353366+0000","last_peered":"2023-11-24T08:08:49.353366+0000","last_clean":"2023-11-24T08:08:49.353366+0000","last_became_active":"2023-11-24T08:08:47.153576+0000","last_became_peered":"2023-11-24T08:08:47.153576+0000","last_unstale":"2023-11-24T08:08:49.353366+0000","last_undegraded":"2023-11-24T08:08:49.353366+0000","last_fullsized":"2023-11-24T08:08:49.353366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551438+0000","last_change":"2023-11-24T08:08:48.157955+0000","last_active":"2023-11-24T08:08:50.551438+0000","last_peered":"2023-11-24T08:08:50.551438+0000","last_clean":"2023-11-24T08:08:50.551438+0000","last_became_active":"2023-11-24T08:08:48.157636+0000","last_became_peered":"2023-11-24T08:08:48.157636+0000","last_unstale":"2023-11-24T08:08:50.551438+0000","last_undegraded":"2023-11-24T08:08:50.551438+0000","last_fullsized":"2023-11-24T08:08:50.551438+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.351375+0000","last_change":"2023-11-24T08:08:47.143826+0000","last_active":"2023-11-24T08:08:50.351375+0000","last_peered":"2023-11-24T08:08:50.351375+0000","last_clean":"2023-11-24T08:08:50.351375+0000","last_became_active":"2023-11-24T08:08:47.142226+0000","last_became_peered":"2023-11-24T08:08:47.142226+0000","last_unstale":"2023-11-24T08:08:50.351375+0000","last_undegraded":"2023-11-24T08:08:50.351375+0000","last_fullsized":"2023-11-24T08:08:50.351375+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125232+0000","last_change":"2023-11-24T08:08:48.133248+0000","last_active":"2023-11-24T08:08:49.125232+0000","last_peered":"2023-11-24T08:08:49.125232+0000","last_clean":"2023-11-24T08:08:49.125232+0000","last_became_active":"2023-11-24T08:08:48.133014+0000","last_became_peered":"2023-11-24T08:08:48.133014+0000","last_unstale":"2023-11-24T08:08:49.125232+0000","last_undegraded":"2023-11-24T08:08:49.125232+0000","last_fullsized":"2023-11-24T08:08:49.125232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:58:28.630116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127822+0000","last_change":"2023-11-24T08:08:47.154380+0000","last_active":"2023-11-24T08:08:49.127822+0000","last_peered":"2023-11-24T08:08:49.127822+0000","last_clean":"2023-11-24T08:08:49.127822+0000","last_became_active":"2023-11-24T08:08:47.154066+0000","last_became_peered":"2023-11-24T08:08:47.154066+0000","last_unstale":"2023-11-24T08:08:49.127822+0000","last_undegraded":"2023-11-24T08:08:49.127822+0000","last_fullsized":"2023-11-24T08:08:49.127822+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:48:29.123228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107253+0000","last_change":"2023-11-24T08:08:48.155559+0000","last_active":"2023-11-24T08:08:50.107253+0000","last_peered":"2023-11-24T08:08:50.107253+0000","last_clean":"2023-11-24T08:08:50.107253+0000","last_became_active":"2023-11-24T08:08:48.155311+0000","last_became_peered":"2023-11-24T08:08:48.155311+0000","last_unstale":"2023-11-24T08:08:50.107253+0000","last_undegraded":"2023-11-24T08:08:50.107253+0000","last_fullsized":"2023-11-24T08:08:50.107253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:14.185680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125803+0000","last_change":"2023-11-24T08:08:47.137292+0000","last_active":"2023-11-24T08:08:49.125803+0000","last_peered":"2023-11-24T08:08:49.125803+0000","last_clean":"2023-11-24T08:08:49.125803+0000","last_became_active":"2023-11-24T08:08:47.136872+0000","last_became_peered":"2023-11-24T08:08:47.136872+0000","last_unstale":"2023-11-24T08:08:49.125803+0000","last_undegraded":"2023-11-24T08:08:49.125803+0000","last_fullsized":"2023-11-24T08:08:49.125803+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127383+0000","last_change":"2023-11-24T08:08:48.161415+0000","last_active":"2023-11-24T08:08:49.127383+0000","last_peered":"2023-11-24T08:08:49.127383+0000","last_clean":"2023-11-24T08:08:49.127383+0000","last_became_active":"2023-11-24T08:08:48.158160+0000","last_became_peered":"2023-11-24T08:08:48.158160+0000","last_unstale":"2023-11-24T08:08:49.127383+0000","last_undegraded":"2023-11-24T08:08:49.127383+0000","last_fullsized":"2023-11-24T08:08:49.127383+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:24.063260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.354005+0000","last_change":"2023-11-24T08:08:47.153377+0000","last_active":"2023-11-24T08:08:49.354005+0000","last_peered":"2023-11-24T08:08:49.354005+0000","last_clean":"2023-11-24T08:08:49.354005+0000","last_became_active":"2023-11-24T08:08:47.153112+0000","last_became_peered":"2023-11-24T08:08:47.153112+0000","last_unstale":"2023-11-24T08:08:49.354005+0000","last_undegraded":"2023-11-24T08:08:49.354005+0000","last_fullsized":"2023-11-24T08:08:49.354005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:39.232399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125805+0000","last_change":"2023-11-24T08:08:48.134378+0000","last_active":"2023-11-24T08:08:49.125805+0000","last_peered":"2023-11-24T08:08:49.125805+0000","last_clean":"2023-11-24T08:08:49.125805+0000","last_became_active":"2023-11-24T08:08:48.134043+0000","last_became_peered":"2023-11-24T08:08:48.134043+0000","last_unstale":"2023-11-24T08:08:49.125805+0000","last_undegraded":"2023-11-24T08:08:49.125805+0000","last_fullsized":"2023-11-24T08:08:49.125805+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:05.279079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'1","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.352131+0000","last_change":"2023-11-24T08:08:47.148275+0000","last_active":"2023-11-24T08:08:50.352131+0000","last_peered":"2023-11-24T08:08:50.352131+0000","last_clean":"2023-11-24T08:08:50.352131+0000","last_became_active":"2023-11-24T08:08:47.148025+0000","last_became_peered":"2023-11-24T08:08:47.148025+0000","last_unstale":"2023-11-24T08:08:50.352131+0000","last_undegraded":"2023-11-24T08:08:50.352131+0000","last_fullsized":"2023-11-24T08:08:50.352131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:37.989439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.449026+0000","last_change":"2023-11-24T08:08:48.146544+0000","last_active":"2023-11-24T08:08:50.449026+0000","last_peered":"2023-11-24T08:08:50.449026+0000","last_clean":"2023-11-24T08:08:50.449026+0000","last_became_active":"2023-11-24T08:08:48.146269+0000","last_became_peered":"2023-11-24T08:08:48.146269+0000","last_unstale":"2023-11-24T08:08:50.449026+0000","last_undegraded":"2023-11-24T08:08:50.449026+0000","last_fullsized":"2023-11-24T08:08:50.449026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:20.269500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.128129+0000","last_change":"2023-11-24T08:08:47.152442+0000","last_active":"2023-11-24T08:08:49.128129+0000","last_peered":"2023-11-24T08:08:49.128129+0000","last_clean":"2023-11-24T08:08:49.128129+0000","last_became_active":"2023-11-24T08:08:47.151763+0000","last_became_peered":"2023-11-24T08:08:47.151763+0000","last_unstale":"2023-11-24T08:08:49.128129+0000","last_undegraded":"2023-11-24T08:08:49.128129+0000","last_fullsized":"2023-11-24T08:08:49.128129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551563+0000","last_change":"2023-11-24T08:08:48.152228+0000","last_active":"2023-11-24T08:08:50.551563+0000","last_peered":"2023-11-24T08:08:50.551563+0000","last_clean":"2023-11-24T08:08:50.551563+0000","last_became_active":"2023-11-24T08:08:48.151904+0000","last_became_peered":"2023-11-24T08:08:48.151904+0000","last_unstale":"2023-11-24T08:08:50.551563+0000","last_undegraded":"2023-11-24T08:08:50.551563+0000","last_fullsized":"2023-11-24T08:08:50.551563+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:00:40.414582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.352672+0000","last_change":"2023-11-24T08:08:47.150060+0000","last_active":"2023-11-24T08:08:49.352672+0000","last_peered":"2023-11-24T08:08:49.352672+0000","last_clean":"2023-11-24T08:08:49.352672+0000","last_became_active":"2023-11-24T08:08:47.149756+0000","last_became_peered":"2023-11-24T08:08:47.149756+0000","last_unstale":"2023-11-24T08:08:49.352672+0000","last_undegraded":"2023-11-24T08:08:49.352672+0000","last_fullsized":"2023-11-24T08:08:49.352672+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:03:45.330972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.107214+0000","last_change":"2023-11-24T08:08:48.150454+0000","last_active":"2023-11-24T08:08:50.107214+0000","last_peered":"2023-11-24T08:08:50.107214+0000","last_clean":"2023-11-24T08:08:50.107214+0000","last_became_active":"2023-11-24T08:08:48.139831+0000","last_became_peered":"2023-11-24T08:08:48.139831+0000","last_unstale":"2023-11-24T08:08:50.107214+0000","last_undegraded":"2023-11-24T08:08:50.107214+0000","last_fullsized":"2023-11-24T08:08:50.107214+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:39.232607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125621+0000","last_change":"2023-11-24T08:08:47.141901+0000","last_active":"2023-11-24T08:08:49.125621+0000","last_peered":"2023-11-24T08:08:49.125621+0000","last_clean":"2023-11-24T08:08:49.125621+0000","last_became_active":"2023-11-24T08:08:47.140760+0000","last_became_peered":"2023-11-24T08:08:47.140760+0000","last_unstale":"2023-11-24T08:08:49.125621+0000","last_undegraded":"2023-11-24T08:08:49.125621+0000","last_fullsized":"2023-11-24T08:08:49.125621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:12:12.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.447992+0000","last_change":"2023-11-24T08:08:48.141100+0000","last_active":"2023-11-24T08:08:50.447992+0000","last_peered":"2023-11-24T08:08:50.447992+0000","last_clean":"2023-11-24T08:08:50.447992+0000","last_became_active":"2023-11-24T08:08:48.138053+0000","last_became_peered":"2023-11-24T08:08:48.138053+0000","last_unstale":"2023-11-24T08:08:50.447992+0000","last_undegraded":"2023-11-24T08:08:50.447992+0000","last_fullsized":"2023-11-24T08:08:50.447992+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:57.289870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125743+0000","last_change":"2023-11-24T08:08:47.138812+0000","last_active":"2023-11-24T08:08:49.125743+0000","last_peered":"2023-11-24T08:08:49.125743+0000","last_clean":"2023-11-24T08:08:49.125743+0000","last_became_active":"2023-11-24T08:08:47.138512+0000","last_became_peered":"2023-11-24T08:08:47.138512+0000","last_unstale":"2023-11-24T08:08:49.125743+0000","last_undegraded":"2023-11-24T08:08:49.125743+0000","last_fullsized":"2023-11-24T08:08:49.125743+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:01:36.953502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448493+0000","last_change":"2023-11-24T08:08:48.155403+0000","last_active":"2023-11-24T08:08:50.448493+0000","last_peered":"2023-11-24T08:08:50.448493+0000","last_clean":"2023-11-24T08:08:50.448493+0000","last_became_active":"2023-11-24T08:08:48.154517+0000","last_became_peered":"2023-11-24T08:08:48.154517+0000","last_unstale":"2023-11-24T08:08:50.448493+0000","last_undegraded":"2023-11-24T08:08:50.448493+0000","last_fullsized":"2023-11-24T08:08:50.448493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:16.993204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353934+0000","last_change":"2023-11-24T08:08:47.152677+0000","last_active":"2023-11-24T08:08:49.353934+0000","last_peered":"2023-11-24T08:08:49.353934+0000","last_clean":"2023-11-24T08:08:49.353934+0000","last_became_active":"2023-11-24T08:08:47.151375+0000","last_became_peered":"2023-11-24T08:08:47.151375+0000","last_unstale":"2023-11-24T08:08:49.353934+0000","last_undegraded":"2023-11-24T08:08:49.353934+0000","last_fullsized":"2023-11-24T08:08:49.353934+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:04:56.046702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.125623+0000","last_change":"2023-11-24T08:08:48.162261+0000","last_active":"2023-11-24T08:08:49.125623+0000","last_peered":"2023-11-24T08:08:49.125623+0000","last_clean":"2023-11-24T08:08:49.125623+0000","last_became_active":"2023-11-24T08:08:48.156627+0000","last_became_peered":"2023-11-24T08:08:48.156627+0000","last_unstale":"2023-11-24T08:08:49.125623+0000","last_undegraded":"2023-11-24T08:08:49.125623+0000","last_fullsized":"2023-11-24T08:08:49.125623+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:10:40.475566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.127263+0000","last_change":"2023-11-24T08:08:48.162034+0000","last_active":"2023-11-24T08:08:49.127263+0000","last_peered":"2023-11-24T08:08:49.127263+0000","last_clean":"2023-11-24T08:08:49.127263+0000","last_became_active":"2023-11-24T08:08:48.161663+0000","last_became_peered":"2023-11-24T08:08:48.161663+0000","last_unstale":"2023-11-24T08:08:49.127263+0000","last_undegraded":"2023-11-24T08:08:49.127263+0000","last_fullsized":"2023-11-24T08:08:49.127263+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:49:56.039005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353243+0000","last_change":"2023-11-24T08:08:47.156960+0000","last_active":"2023-11-24T08:08:49.353243+0000","last_peered":"2023-11-24T08:08:49.353243+0000","last_clean":"2023-11-24T08:08:49.353243+0000","last_became_active":"2023-11-24T08:08:47.156401+0000","last_became_peered":"2023-11-24T08:08:47.156401+0000","last_unstale":"2023-11-24T08:08:49.353243+0000","last_undegraded":"2023-11-24T08:08:49.353243+0000","last_fullsized":"2023-11-24T08:08:49.353243+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:09:04.035911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.126566+0000","last_change":"2023-11-24T08:08:48.159506+0000","last_active":"2023-11-24T08:08:49.126566+0000","last_peered":"2023-11-24T08:08:49.126566+0000","last_clean":"2023-11-24T08:08:49.126566+0000","last_became_active":"2023-11-24T08:08:48.159220+0000","last_became_peered":"2023-11-24T08:08:48.159220+0000","last_unstale":"2023-11-24T08:08:49.126566+0000","last_undegraded":"2023-11-24T08:08:49.126566+0000","last_fullsized":"2023-11-24T08:08:49.126566+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:33:14.900691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"21'2","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:49.353690+0000","last_change":"2023-11-24T08:08:47.155268+0000","last_active":"2023-11-24T08:08:49.353690+0000","last_peered":"2023-11-24T08:08:49.353690+0000","last_clean":"2023-11-24T08:08:49.353690+0000","last_became_active":"2023-11-24T08:08:47.154139+0000","last_became_peered":"2023-11-24T08:08:47.154139+0000","last_unstale":"2023-11-24T08:08:49.353690+0000","last_undegraded":"2023-11-24T08:08:49.353690+0000","last_fullsized":"2023-11-24T08:08:49.353690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:20.169115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.551743+0000","last_change":"2023-11-24T08:08:48.157139+0000","last_active":"2023-11-24T08:08:50.551743+0000","last_peered":"2023-11-24T08:08:50.551743+0000","last_clean":"2023-11-24T08:08:50.551743+0000","last_became_active":"2023-11-24T08:08:48.156875+0000","last_became_peered":"2023-11-24T08:08:48.156875+0000","last_unstale":"2023-11-24T08:08:50.551743+0000","last_undegraded":"2023-11-24T08:08:50.551743+0000","last_fullsized":"2023-11-24T08:08:50.551743+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:32:17.630995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"21'1","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.345355+0000","last_change":"2023-11-24T08:08:47.143443+0000","last_active":"2023-11-24T08:08:50.345355+0000","last_peered":"2023-11-24T08:08:50.345355+0000","last_clean":"2023-11-24T08:08:50.345355+0000","last_became_active":"2023-11-24T08:08:47.143156+0000","last_became_peered":"2023-11-24T08:08:47.143156+0000","last_unstale":"2023-11-24T08:08:50.345355+0000","last_undegraded":"2023-11-24T08:08:50.345355+0000","last_fullsized":"2023-11-24T08:08:50.345355+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:07:19.724311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.448871+0000","last_change":"2023-11-24T08:08:48.141063+0000","last_active":"2023-11-24T08:08:50.448871+0000","last_peered":"2023-11-24T08:08:50.448871+0000","last_clean":"2023-11-24T08:08:50.448871+0000","last_became_active":"2023-11-24T08:08:48.140809+0000","last_became_peered":"2023-11-24T08:08:48.140809+0000","last_unstale":"2023-11-24T08:08:50.448871+0000","last_undegraded":"2023-11-24T08:08:50.448871+0000","last_fullsized":"2023-11-24T08:08:50.448871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:47.095064+0000","last_clean_scrub_stamp":"2023-11-24T08:08:47.095064+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:19.645688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"21'4","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-24T08:08:50.342687+0000","last_change":"2023-11-24T08:08:47.150905+0000","last_active":"2023-11-24T08:08:50.342687+0000","last_peered":"2023-11-24T08:08:50.342687+0000","last_clean":"2023-11-24T08:08:50.342687+0000","last_became_active":"2023-11-24T08:08:47.150670+0000","last_became_peered":"2023-11-24T08:08:47.150670+0000","last_unstale":"2023-11-24T08:08:50.342687+0000","last_undegraded":"2023-11-24T08:08:50.342687+0000","last_fullsized":"2023-11-24T08:08:50.342687+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:45.851896+0000","last_clean_scrub_stamp":"2023-11-24T08:08:45.851896+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:00.255329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32120,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339720,"statfs":{"total":96636764160,"available":96603873280,"internally_reserved":0,"allocated":823296,"data_stored":669826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.046}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4910000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8420000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.944}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31420,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340420,"statfs":{"total":96636764160,"available":96604590080,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4970000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.99}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":229376,"data_stored":78966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1240000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2290000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.952}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.736}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.627}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.909}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.218}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31416,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340424,"statfs":{"total":96636764160,"available":96604594176,"internally_reserved":0,"allocated":237568,"data_stored":79002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.24}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6240000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7909999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":237568,"data_stored":80488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.978}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.351}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1339999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4300000000000002}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":225280,"data_stored":78876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4219999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2010000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.492}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":79036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0920000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0659999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.907}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.143}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2210000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.423}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":831488,"data_stored":669882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82099999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.151}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.355}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5569999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.671}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.774}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:08:58.497 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T08:08:58.497 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-24T08:08:58.497 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-24T08:08:58.497 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-24T08:08:58.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.731+0000 7f48d45de640 1 Processor -- start 2023-11-24T08:08:58.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.731+0000 7f48d45de640 1 -- start start 2023-11-24T08:08:58.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d45de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48cc0cf450 0x7f48cc0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d45de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc0cece0 con 0x7f48cc15c7f0 2023-11-24T08:08:58.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d45de640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48cc06d040 con 0x7f48cc0cdfb0 2023-11-24T08:08:58.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d45de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0cfe40 con 0x7f48cc0cf450 2023-11-24T08:08:58.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d237a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f48cc15c7f0 0x7f48cc06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32864/0 (socket says 172.21.15.106:32864) 2023-11-24T08:08:58.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d237a640 1 -- 172.21.15.106:0/2939953532 learned_addr learned my addr 172.21.15.106:0/2939953532 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:58.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d1b79640 1 --2- 172.21.15.106:0/2939953532 >> v2:172.21.15.106:3300/0 conn(0x7f48cc0cf450 0x7f48cc0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.733+0000 7f48d1378640 1 -- 172.21.15.106:0/2939953532 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185344891 0 0) 0x7f48cc0cece0 con 0x7f48cc15c7f0 2023-11-24T08:08:58.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.734+0000 7f48d1378640 1 -- 172.21.15.106:0/2939953532 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8003520 con 0x7f48cc15c7f0 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.734+0000 7f48d1b79640 1 -- 172.21.15.106:0/2939953532 >> v1:172.21.15.156:6789/0 conn(0x7f48cc0cdfb0 legacy=0x7f48cc0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.734+0000 7f48d1b79640 1 -- 172.21.15.106:0/2939953532 >> v1:172.21.15.106:6789/0 conn(0x7f48cc15c7f0 legacy=0x7f48cc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.734+0000 7f48d1b79640 1 -- 172.21.15.106:0/2939953532 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc06d320 con 0x7f48cc0cf450 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.735+0000 7f48d1b79640 1 --2- 172.21.15.106:0/2939953532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc0cf900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0097a0 tx=0x7f48bc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6287aca4e891c25c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.735+0000 7f48d1378640 1 -- 172.21.15.106:0/2939953532 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc031830 con 0x7f48cc0cf450 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.735+0000 7f48d1378640 1 -- 172.21.15.106:0/2939953532 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc031990 con 0x7f48cc0cf450 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.735+0000 7f48d1378640 1 -- 172.21.15.106:0/2939953532 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc031d20 con 0x7f48cc0cf450 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.735+0000 7f48d45de640 1 -- 172.21.15.106:0/2939953532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 msgr2=0x7f48cc0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.735+0000 7f48d45de640 1 --2- 172.21.15.106:0/2939953532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc0cf900 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48bc0097a0 tx=0x7f48bc02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.736+0000 7f48d45de640 1 -- 172.21.15.106:0/2939953532 shutdown_connections 2023-11-24T08:08:58.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.736+0000 7f48d45de640 1 --2- 172.21.15.106:0/2939953532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc0cf900 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.736+0000 7f48d45de640 1 -- 172.21.15.106:0/2939953532 >> 172.21.15.106:0/2939953532 conn(0x7f48cc06a920 msgr2=0x7f48cc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.736+0000 7f48d45de640 1 -- 172.21.15.106:0/2939953532 shutdown_connections 2023-11-24T08:08:58.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.736+0000 7f48d45de640 1 -- 172.21.15.106:0/2939953532 wait complete. 2023-11-24T08:08:58.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.737+0000 7f48d45de640 1 Processor -- start 2023-11-24T08:08:58.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.737+0000 7f48d45de640 1 -- start start 2023-11-24T08:08:58.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d45de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0cdfb0 0x7f48cc098990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d45de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc145a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d45de640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15c7f0 0x7f48cc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d45de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc098ed0 con 0x7f48cc0cf450 2023-11-24T08:08:58.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d45de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc1439f0 con 0x7f48cc0cdfb0 2023-11-24T08:08:58.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d45de640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc145fb0 con 0x7f48cc15c7f0 2023-11-24T08:08:58.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d1b79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc145a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d1b79640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc145a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34584/0 (socket says 172.21.15.106:34584) 2023-11-24T08:08:58.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.738+0000 7f48d1b79640 1 -- 172.21.15.106:0/1065256429 learned_addr learned my addr 172.21.15.106:0/1065256429 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:58.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d1b79640 1 -- 172.21.15.106:0/1065256429 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15c7f0 msgr2=0x7f48cc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d1b79640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15c7f0 0x7f48cc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d1b79640 1 -- 172.21.15.106:0/1065256429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0cdfb0 msgr2=0x7f48cc098990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d1b79640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0cdfb0 0x7f48cc098990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d1b79640 1 -- 172.21.15.106:0/1065256429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48bc009400 con 0x7f48cc0cf450 2023-11-24T08:08:58.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d1b79640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc145a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48bc02bee0 tx=0x7f48bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc03a6f0 con 0x7f48cc0cf450 2023-11-24T08:08:58.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48bc03a870 con 0x7f48cc0cf450 2023-11-24T08:08:58.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc1430c0 con 0x7f48cc0cf450 2023-11-24T08:08:58.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48bc03ac20 con 0x7f48cc0cf450 2023-11-24T08:08:58.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.739+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc143680 con 0x7f48cc0cf450 2023-11-24T08:08:58.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.740+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc067470 con 0x7f48cc0cf450 2023-11-24T08:08:58.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.741+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f48bc04c3b0 con 0x7f48cc0cf450 2023-11-24T08:08:58.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.742+0000 7f48c2ffd640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006ae40 0x7f48b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:58.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.743+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48bc0ab330 con 0x7f48cc0cf450 2023-11-24T08:08:58.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.743+0000 7f48d237a640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006ae40 0x7f48b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:58.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.743+0000 7f48d237a640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006ae40 0x7f48b006d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48c8001570 tx=0x7f48c8007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:58.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.746+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48bc0763d0 con 0x7f48cc0cf450 2023-11-24T08:08:58.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.955+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f48cc143b30 con 0x7f48cc0cf450 2023-11-24T08:08:58.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.955+0000 7f48c2ffd640 1 -- 172.21.15.106:0/1065256429 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f48bc0799f0 con 0x7f48cc0cf450 2023-11-24T08:08:58.958 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:58.958 INFO:teuthology.orchestra.run.smithi106.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006ae40 msgr2=0x7f48b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006ae40 0x7f48b006d300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f48c8001570 tx=0x7f48c8007930 comp rx=0 tx=0).stop 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 msgr2=0x7f48cc145a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc145a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48bc02bee0 tx=0x7f48bc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 shutdown_connections 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006ae40 0x7f48b006d300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15c7f0 0x7f48cc06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc0cf450 0x7f48cc145a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 --2- 172.21.15.106:0/1065256429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0cdfb0 0x7f48cc098990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:58.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 >> 172.21.15.106:0/1065256429 conn(0x7f48cc06a920 msgr2=0x7f48cc0b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:58.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 shutdown_connections 2023-11-24T08:08:58.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:58.958+0000 7f48d45de640 1 -- 172.21.15.106:0/1065256429 wait complete. 2023-11-24T08:08:58.978 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-24T08:08:58.978 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:08:59.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.179+0000 7f78341fc640 1 Processor -- start 2023-11-24T08:08:59.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.179+0000 7f78341fc640 1 -- start start 2023-11-24T08:08:59.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.180+0000 7f78341fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f782c0cf450 0x7f782c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.180+0000 7f78341fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f782c0cdfb0 0x7f782c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f78341fc640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0cece0 con 0x7f782c15c7f0 2023-11-24T08:08:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f78341fc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782c0cfe40 con 0x7f782c0cf450 2023-11-24T08:08:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f78341fc640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782c156010 con 0x7f782c0cdfb0 2023-11-24T08:08:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f782c0cf450 0x7f782c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f782c0cf450 0x7f782c0cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34600/0 (socket says 172.21.15.106:34600) 2023-11-24T08:08:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 -- 172.21.15.106:0/1473459551 learned_addr learned my addr 172.21.15.106:0/1473459551 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:59.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7832799640 1 --2- 172.21.15.106:0/1473459551 >> v2:172.21.15.156:3300/0 conn(0x7f782c0cdfb0 0x7f782c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 -- 172.21.15.106:0/1473459551 >> v2:172.21.15.156:3300/0 conn(0x7f782c0cdfb0 msgr2=0x7f782c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 --2- 172.21.15.106:0/1473459551 >> v2:172.21.15.156:3300/0 conn(0x7f782c0cdfb0 0x7f782c0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 -- 172.21.15.106:0/1473459551 >> v1:172.21.15.156:6789/0 conn(0x7f782c15c7f0 legacy=0x7f782c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.181+0000 7f7831797640 1 -- 172.21.15.106:0/1473459551 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c06d040 con 0x7f782c0cf450 2023-11-24T08:08:59.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.182+0000 7f7831797640 1 --2- 172.21.15.106:0/1473459551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c0cf900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f782c0d00e0 tx=0x7f781c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e27caa494357078 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f7830f96640 1 -- 172.21.15.106:0/1473459551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c031870 con 0x7f782c0cf450 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f7830f96640 1 -- 172.21.15.106:0/1473459551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c0319d0 con 0x7f782c0cf450 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f7830f96640 1 -- 172.21.15.106:0/1473459551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c031d60 con 0x7f782c0cf450 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 -- 172.21.15.106:0/1473459551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 msgr2=0x7f782c0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 --2- 172.21.15.106:0/1473459551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c0cf900 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f782c0d00e0 tx=0x7f781c02b910 comp rx=0 tx=0).stop 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 -- 172.21.15.106:0/1473459551 shutdown_connections 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 --2- 172.21.15.106:0/1473459551 >> v2:172.21.15.156:3300/0 conn(0x7f782c0cdfb0 0x7f782c0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 --2- 172.21.15.106:0/1473459551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c0cf900 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 -- 172.21.15.106:0/1473459551 >> 172.21.15.106:0/1473459551 conn(0x7f782c06a920 msgr2=0x7f782c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:59.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.183+0000 7f78341fc640 1 -- 172.21.15.106:0/1473459551 shutdown_connections 2023-11-24T08:08:59.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.184+0000 7f78341fc640 1 -- 172.21.15.106:0/1473459551 wait complete. 2023-11-24T08:08:59.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.185+0000 7f78341fc640 1 Processor -- start 2023-11-24T08:08:59.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.185+0000 7f78341fc640 1 -- start start 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f78341fc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f782c0cdfb0 0x7f782c154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f78341fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f78341fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f782c15c7f0 0x7f782c0add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f78341fc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1553d0 con 0x7f782c0cf450 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f78341fc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0c88b0 con 0x7f782c15c7f0 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f78341fc640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c0c8b90 con 0x7f782c0cdfb0 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f7831797640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f7831797640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c1564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34602/0 (socket says 172.21.15.106:34602) 2023-11-24T08:08:59.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.186+0000 7f7831797640 1 -- 172.21.15.106:0/2140348386 learned_addr learned my addr 172.21.15.106:0/2140348386 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:59.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831797640 1 -- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f782c0cdfb0 msgr2=0x7f782c154e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831f98640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f782c0cdfb0 0x7f782c154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7832799640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f782c15c7f0 0x7f782c0add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831797640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f782c0cdfb0 0x7f782c154e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831797640 1 -- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f782c15c7f0 msgr2=0x7f782c0add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831797640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f782c15c7f0 0x7f782c0add40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831797640 1 -- 172.21.15.106:0/2140348386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f781c0094d0 con 0x7f782c0cf450 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831f98640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f782c0cdfb0 0x7f782c154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f7831797640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c1564c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f781c000c00 tx=0x7f781c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.187+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c03abe0 con 0x7f782c0cf450 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.188+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c03ad40 con 0x7f782c0cf450 2023-11-24T08:08:59.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.188+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c044610 con 0x7f782c0cf450 2023-11-24T08:08:59.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.188+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c0c8e70 con 0x7f782c0cf450 2023-11-24T08:08:59.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.188+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c0c9430 con 0x7f782c0cf450 2023-11-24T08:08:59.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.189+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c067470 con 0x7f782c0cf450 2023-11-24T08:08:59.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.190+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f781c04d450 con 0x7f782c0cf450 2023-11-24T08:08:59.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.190+0000 7f78227fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f781006ae40 0x7f781006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.190+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f781c02d040 con 0x7f782c0cf450 2023-11-24T08:08:59.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.191+0000 7f7831f98640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f781006ae40 0x7f781006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.191+0000 7f7831f98640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f781006ae40 0x7f781006d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f782800a810 tx=0x7f7828008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:59.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.195+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f781c076d00 con 0x7f782c0cf450 2023-11-24T08:08:59.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.380+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f782c0cd730 con 0x7f782c0cf450 2023-11-24T08:08:59.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.381+0000 7f78227fc640 1 -- 172.21.15.106:0/2140348386 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f781c035020 con 0x7f782c0cf450 2023-11-24T08:08:59.384 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:08:59.384 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f781006ae40 msgr2=0x7f781006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f781006ae40 0x7f781006d300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f782800a810 tx=0x7f7828008040 comp rx=0 tx=0).stop 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 msgr2=0x7f782c1564c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c1564c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f781c000c00 tx=0x7f781c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 shutdown_connections 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f782c15c7f0 0x7f782c0add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f782c0cf450 0x7f782c1564c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f781006ae40 0x7f781006d300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 --2- 172.21.15.106:0/2140348386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f782c0cdfb0 0x7f782c154e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 >> 172.21.15.106:0/2140348386 conn(0x7f782c06a920 msgr2=0x7f782c0b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:59.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 shutdown_connections 2023-11-24T08:08:59.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.384+0000 7f78341fc640 1 -- 172.21.15.106:0/2140348386 wait complete. 2023-11-24T08:08:59.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:08:59.405 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:08:59.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.595+0000 7f0e32451640 1 Processor -- start 2023-11-24T08:08:59.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.595+0000 7f0e32451640 1 -- start start 2023-11-24T08:08:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.596+0000 7f0e32451640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0e2c06de10 0x7f0e2c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.596+0000 7f0e32451640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c15ec90 con 0x7f0e2c0c49f0 2023-11-24T08:08:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.596+0000 7f0e32451640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c0cd530 con 0x7f0e2c072160 2023-11-24T08:08:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.596+0000 7f0e32451640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c4ed0 con 0x7f0e2c06de10 2023-11-24T08:08:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e309ee640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0e2c0c49f0 0x7f0e2c0705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32870/0 (socket says 172.21.15.106:32870) 2023-11-24T08:08:59.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e309ee640 1 -- 172.21.15.106:0/2492757440 learned_addr learned my addr 172.21.15.106:0/2492757440 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:08:59.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2492757440 >> v2:172.21.15.156:3300/0 conn(0x7f0e2c06de10 0x7f0e2c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2165951278 0 0) 0x7f0e24001110 con 0x7f0e2c0c49f0 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e0c003520 con 0x7f0e2c0c49f0 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257949034 0 0) 0x7f0e0c003520 con 0x7f0e2c0c49f0 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 >> v2:172.21.15.156:3300/0 conn(0x7f0e2c06de10 msgr2=0x7f0e2c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 --2- 172.21.15.106:0/2492757440 >> v2:172.21.15.156:3300/0 conn(0x7f0e2c06de10 0x7f0e2c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 >> v1:172.21.15.156:6789/0 conn(0x7f0e2c072160 legacy=0x7f0e2c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0c51b0 con 0x7f0e2c0c49f0 2023-11-24T08:08:59.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.597+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f0e24001800 con 0x7f0e2c0c49f0 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 >> v1:172.21.15.106:6789/0 conn(0x7f0e2c0c49f0 legacy=0x7f0e2c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e0c0039c0 0x7f0e0c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c005180 0x7f0e0c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c4ed0 con 0x7f0e0c0047c0 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00c080 con 0x7f0e0c005180 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c00c240 con 0x7f0e0c0039c0 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2bfff640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e0c0039c0 0x7f0e0c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e309ee640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c005180 0x7f0e0c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e0c0039c0 msgr2=0x7f0e0c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e0c0039c0 0x7f0e0c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c005180 msgr2=0x7f0e0c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c005180 0x7f0e0c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2492757440 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c003520 con 0x7f0e0c0047c0 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.598+0000 7f0e309ee640 1 -- 172.21.15.106:0/2492757440 reap_dead start 2023-11-24T08:08:59.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.599+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009fc0 tx=0x7f0e0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac2935fedac7cb1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1402cc20 con 0x7f0e0c0047c0 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400f040 con 0x7f0e0c0047c0 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e2affd640 1 -- 172.21.15.106:0/2492757440 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14034440 con 0x7f0e0c0047c0 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 -- 172.21.15.106:0/2492757440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e0c0047c0 msgr2=0x7f0e0c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009fc0 tx=0x7f0e0c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 -- 172.21.15.106:0/2492757440 shutdown_connections 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 --2- 172.21.15.106:0/2492757440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e0c0047c0 0x7f0e0c004c40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 -- 172.21.15.106:0/2492757440 >> 172.21.15.106:0/2492757440 conn(0x7f0e2c0b0c50 msgr2=0x7f0e2c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 -- 172.21.15.106:0/2492757440 shutdown_connections 2023-11-24T08:08:59.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.600+0000 7f0e32451640 1 -- 172.21.15.106:0/2492757440 wait complete. 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.602+0000 7f0e32451640 1 Processor -- start 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.602+0000 7f0e32451640 1 -- start start 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e32451640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c0047c0 0x7f0e2c1594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e32451640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 0x7f0e2c1f6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e32451640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e2c1f6fa0 0x7f0e2c0cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e32451640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c159a10 con 0x7f0e2c159cd0 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e32451640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c1f7a00 con 0x7f0e0c0047c0 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e32451640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0b33f0 con 0x7f0e2c1f6fa0 2023-11-24T08:08:59.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e2b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 0x7f0e2c1f6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e2b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 0x7f0e2c1f6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34622/0 (socket says 172.21.15.106:34622) 2023-11-24T08:08:59.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.603+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2778001310 learned_addr learned my addr 172.21.15.106:0/2778001310 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:08:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2778001310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e2c1f6fa0 msgr2=0x7f0e2c0cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e2c1f6fa0 0x7f0e2c0cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2778001310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c0047c0 msgr2=0x7f0e2c1594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c0047c0 0x7f0e2c1594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e2b7fe640 1 -- 172.21.15.106:0/2778001310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c1f7440 con 0x7f0e2c159cd0 2023-11-24T08:08:59.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e2b7fe640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 0x7f0e2c1f6a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402c690 tx=0x7f0e14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14034c50 con 0x7f0e2c159cd0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14034db0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14003bb0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c1f7720 con 0x7f0e2c159cd0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.604+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c0cb7d0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.605+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c0b2ee0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.606+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0e14040450 con 0x7f0e2c159cd0 2023-11-24T08:08:59.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.607+0000 7f0e28ff9640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0e1006ae40 0x7f0e1006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:08:59.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.607+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e140abe70 con 0x7f0e2c159cd0 2023-11-24T08:08:59.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.608+0000 7f0e2bfff640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0e1006ae40 0x7f0e1006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:08:59.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.608+0000 7f0e2bfff640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0e1006ae40 0x7f0e1006d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0b50b0 tx=0x7f0e1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:08:59.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.612+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0e2c0b2ee0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.799+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0e2c0b2ee0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.800+0000 7f0e28ff9640 1 -- 172.21.15.106:0/2778001310 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0e2c0b2ee0 con 0x7f0e2c159cd0 2023-11-24T08:08:59.802 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:08:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0e1006ae40 msgr2=0x7f0e1006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0e1006ae40 0x7f0e1006d300 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0b50b0 tx=0x7f0e1c009040 comp rx=0 tx=0).stop 2023-11-24T08:08:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 msgr2=0x7f0e2c1f6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:08:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 0x7f0e2c1f6a60 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e1402c690 tx=0x7f0e14009750 comp rx=0 tx=0).stop 2023-11-24T08:08:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 shutdown_connections 2023-11-24T08:08:59.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0e1006ae40 0x7f0e1006d300 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0e2c1f6fa0 0x7f0e2c0cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0e2c159cd0 0x7f0e2c1f6a60 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 --2- 172.21.15.106:0/2778001310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e0c0047c0 0x7f0e2c1594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:08:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 >> 172.21.15.106:0/2778001310 conn(0x7f0e2c0b0c50 msgr2=0x7f0e2c0ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:08:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 shutdown_connections 2023-11-24T08:08:59.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:08:59.802+0000 7f0e32451640 1 -- 172.21.15.106:0/2778001310 wait complete. 2023-11-24T08:08:59.823 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.032+0000 7f26c3147640 1 Processor -- start 2023-11-24T08:09:00.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.032+0000 7f26c3147640 1 -- start start 2023-11-24T08:09:00.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.033+0000 7f26c3147640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f26bc0b0b40 0x7f26bc0b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.033+0000 7f26c3147640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 0x7f26bc06ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.033+0000 7f26c3147640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26bc15ed20 con 0x7f26bc15b260 2023-11-24T08:09:00.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.033+0000 7f26c3147640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc15b640 con 0x7f26bc0b0b40 2023-11-24T08:09:00.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.033+0000 7f26c3147640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0b14f0 con 0x7f26bc06c5b0 2023-11-24T08:09:00.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.034+0000 7f26c2946640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 0x7f26bc06ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.034+0000 7f26c1944640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f26bc0b0b40 0x7f26bc0b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.034+0000 7f26c2946640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 0x7f26bc06ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48612/0 (socket says 172.21.15.106:48612) 2023-11-24T08:09:00.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.034+0000 7f26c1944640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f26bc0b0b40 0x7f26bc0b0fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39878/0 (socket says 172.21.15.106:39878) 2023-11-24T08:09:00.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.034+0000 7f26c2946640 1 -- 172.21.15.106:0/2279422038 learned_addr learned my addr 172.21.15.106:0/2279422038 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:00.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.035+0000 7f26c2946640 1 -- 172.21.15.106:0/2279422038 >> v2:172.21.15.156:3300/0 conn(0x7f26bc0b0b40 msgr2=0x7f26bc0b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.035+0000 7f26c2946640 1 --2- 172.21.15.106:0/2279422038 >> v2:172.21.15.156:3300/0 conn(0x7f26bc0b0b40 0x7f26bc0b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.035+0000 7f26c2946640 1 -- 172.21.15.106:0/2279422038 >> v1:172.21.15.156:6789/0 conn(0x7f26bc15b260 legacy=0x7f26bc06e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.035+0000 7f26c2946640 1 -- 172.21.15.106:0/2279422038 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc15f8a0 con 0x7f26bc06c5b0 2023-11-24T08:09:00.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.036+0000 7f26c2946640 1 --2- 172.21.15.106:0/2279422038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 0x7f26bc06ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009e50 tx=0x7f26b402be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=382fb764a8a46f21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.037+0000 7f26c1143640 1 -- 172.21.15.106:0/2279422038 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b402c9e0 con 0x7f26bc06c5b0 2023-11-24T08:09:00.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.037+0000 7f26c1143640 1 -- 172.21.15.106:0/2279422038 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b402cb40 con 0x7f26bc06c5b0 2023-11-24T08:09:00.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.037+0000 7f26c1143640 1 -- 172.21.15.106:0/2279422038 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b40343f0 con 0x7f26bc06c5b0 2023-11-24T08:09:00.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 -- 172.21.15.106:0/2279422038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 msgr2=0x7f26bc06ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 --2- 172.21.15.106:0/2279422038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 0x7f26bc06ca40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f26b4009e50 tx=0x7f26b402be40 comp rx=0 tx=0).stop 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 -- 172.21.15.106:0/2279422038 shutdown_connections 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 --2- 172.21.15.106:0/2279422038 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc06c5b0 0x7f26bc06ca40 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 --2- 172.21.15.106:0/2279422038 >> v2:172.21.15.156:3300/0 conn(0x7f26bc0b0b40 0x7f26bc0b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 -- 172.21.15.106:0/2279422038 >> 172.21.15.106:0/2279422038 conn(0x7f26bc0cbb40 msgr2=0x7f26bc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 -- 172.21.15.106:0/2279422038 shutdown_connections 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.038+0000 7f26c3147640 1 -- 172.21.15.106:0/2279422038 wait complete. 2023-11-24T08:09:00.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.040+0000 7f26c3147640 1 Processor -- start 2023-11-24T08:09:00.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.040+0000 7f26c3147640 1 -- start start 2023-11-24T08:09:00.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.041+0000 7f26c3147640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26bc06c5b0 0x7f26bc14e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.041+0000 7f26c3147640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc0b0b40 0x7f26bc0c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.041+0000 7f26c3147640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 0x7f26bc0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.041+0000 7f26c3147640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc14e850 con 0x7f26bc15b260 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.041+0000 7f26c3147640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc0c6500 con 0x7f26bc06c5b0 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.041+0000 7f26c3147640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc099c80 con 0x7f26bc0b0b40 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 0x7f26bc0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 0x7f26bc0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34626/0 (socket says 172.21.15.106:34626) 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 -- 172.21.15.106:0/2182148623 learned_addr learned my addr 172.21.15.106:0/2182148623 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c1944640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc0b0b40 0x7f26bc0c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2145640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26bc06c5b0 0x7f26bc14e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 -- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc0b0b40 msgr2=0x7f26bc0c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc0b0b40 0x7f26bc0c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 -- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26bc06c5b0 msgr2=0x7f26bc14e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26bc06c5b0 0x7f26bc14e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.042+0000 7f26c2946640 1 -- 172.21.15.106:0/2182148623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4009b30 con 0x7f26bc15b260 2023-11-24T08:09:00.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.043+0000 7f26c2946640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 0x7f26bc0c9c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26b4036040 tx=0x7f26b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.043+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b4009510 con 0x7f26bc15b260 2023-11-24T08:09:00.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.043+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b403e450 con 0x7f26bc15b260 2023-11-24T08:09:00.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.043+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc099f60 con 0x7f26bc15b260 2023-11-24T08:09:00.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.043+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b403e800 con 0x7f26bc15b260 2023-11-24T08:09:00.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.043+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc14ecc0 con 0x7f26bc15b260 2023-11-24T08:09:00.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.044+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc09a480 con 0x7f26bc15b260 2023-11-24T08:09:00.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.045+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f26b403e960 con 0x7f26bc15b260 2023-11-24T08:09:00.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.047+0000 7f26b2ffd640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f26a006ae40 0x7f26a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.047+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26b40ac340 con 0x7f26bc15b260 2023-11-24T08:09:00.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.047+0000 7f26c2145640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f26a006ae40 0x7f26a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.048+0000 7f26c2145640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f26a006ae40 0x7f26a006d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000f60 tx=0x7f26b8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.053+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26bc09a480 con 0x7f26bc15b260 2023-11-24T08:09:00.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.247+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26bc09a480 con 0x7f26bc15b260 2023-11-24T08:09:00.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.249+0000 7f26b2ffd640 1 -- 172.21.15.106:0/2182148623 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26bc09a480 con 0x7f26bc15b260 2023-11-24T08:09:00.251 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f26a006ae40 msgr2=0x7f26a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f26a006ae40 0x7f26a006d300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f26b8000f60 tx=0x7f26b8009040 comp rx=0 tx=0).stop 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 msgr2=0x7f26bc0c9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 0x7f26bc0c9c50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f26b4036040 tx=0x7f26b4004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 shutdown_connections 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26bc15b260 0x7f26bc0c9c50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f26a006ae40 0x7f26a006d300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26bc0b0b40 0x7f26bc0c9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 --2- 172.21.15.106:0/2182148623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26bc06c5b0 0x7f26bc14e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.251+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 >> 172.21.15.106:0/2182148623 conn(0x7f26bc0cbb40 msgr2=0x7f26bc071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.252+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 shutdown_connections 2023-11-24T08:09:00.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.252+0000 7f26c3147640 1 -- 172.21.15.106:0/2182148623 wait complete. 2023-11-24T08:09:00.272 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.486+0000 7f57387c4640 1 Processor -- start 2023-11-24T08:09:00.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.487+0000 7f57387c4640 1 -- start start 2023-11-24T08:09:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.488+0000 7f57387c4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f573015d720 0x7f57300d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.488+0000 7f57387c4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57300d3870 0x7f57300b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.488+0000 7f57387c4640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573015db00 con 0x7f57300b1400 2023-11-24T08:09:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.488+0000 7f57387c4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57300d3d20 con 0x7f573015d720 2023-11-24T08:09:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.488+0000 7f57387c4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57300d0ce0 con 0x7f57300d3870 2023-11-24T08:09:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5736560640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f573015d720 0x7f57300d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5735d5f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57300d3870 0x7f57300b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5735d5f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57300d3870 0x7f57300b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48632/0 (socket says 172.21.15.106:48632) 2023-11-24T08:09:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5735d5f640 1 -- 172.21.15.106:0/2392724444 learned_addr learned my addr 172.21.15.106:0/2392724444 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5736560640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f573015d720 0x7f57300d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39888/0 (socket says 172.21.15.106:39888) 2023-11-24T08:09:00.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5736560640 1 -- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57300d3870 msgr2=0x7f57300b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5736560640 1 --2- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57300d3870 0x7f57300b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5736560640 1 -- 172.21.15.106:0/2392724444 >> v1:172.21.15.156:6789/0 conn(0x7f57300b1400 legacy=0x7f57300b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.489+0000 7f5736560640 1 -- 172.21.15.106:0/2392724444 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573006d400 con 0x7f573015d720 2023-11-24T08:09:00.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.491+0000 7f5736560640 1 --2- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f573015d720 0x7f57300d3330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f572c0097a0 tx=0x7f572c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c0a43d1fbadc7c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.491+0000 7f573555e640 1 -- 172.21.15.106:0/2392724444 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c031780 con 0x7f573015d720 2023-11-24T08:09:00.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.491+0000 7f573555e640 1 -- 172.21.15.106:0/2392724444 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572c0318e0 con 0x7f573015d720 2023-11-24T08:09:00.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.491+0000 7f573555e640 1 -- 172.21.15.106:0/2392724444 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572c031c70 con 0x7f573015d720 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 -- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f573015d720 msgr2=0x7f57300d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 --2- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f573015d720 0x7f57300d3330 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f572c0097a0 tx=0x7f572c02b820 comp rx=0 tx=0).stop 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 -- 172.21.15.106:0/2392724444 shutdown_connections 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 --2- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f57300d3870 0x7f57300b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 --2- 172.21.15.106:0/2392724444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f573015d720 0x7f57300d3330 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 -- 172.21.15.106:0/2392724444 >> 172.21.15.106:0/2392724444 conn(0x7f573008af60 msgr2=0x7f573009c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 -- 172.21.15.106:0/2392724444 shutdown_connections 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.492+0000 7f57387c4640 1 -- 172.21.15.106:0/2392724444 wait complete. 2023-11-24T08:09:00.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.494+0000 7f57387c4640 1 Processor -- start 2023-11-24T08:09:00.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.494+0000 7f57387c4640 1 -- start start 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f57387c4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57300b1400 0x7f573015f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f57387c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 0x7f573015bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f57387c4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f573015d720 0x7f573015e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f57387c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730160430 con 0x7f57300d3870 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f57387c4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5730160710 con 0x7f57300b1400 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f57387c4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57301609f0 con 0x7f573015d720 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f5735d5f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 0x7f573015bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f5735d5f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 0x7f573015bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34642/0 (socket says 172.21.15.106:34642) 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f5735d5f640 1 -- 172.21.15.106:0/3981600542 learned_addr learned my addr 172.21.15.106:0/3981600542 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f5736560640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57300b1400 0x7f573015f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.495+0000 7f5736d61640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f573015d720 0x7f573015e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5735d5f640 1 -- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f573015d720 msgr2=0x7f573015e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5735d5f640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f573015d720 0x7f573015e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5735d5f640 1 -- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57300b1400 msgr2=0x7f573015f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5735d5f640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57300b1400 0x7f573015f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5735d5f640 1 -- 172.21.15.106:0/3981600542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f572c009400 con 0x7f57300d3870 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5735d5f640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 0x7f573015bc40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5720009fd0 tx=0x7f572000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572000c940 con 0x7f57300d3870 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f572000caa0 con 0x7f57300d3870 2023-11-24T08:09:00.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f572000ce30 con 0x7f57300d3870 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573015dd30 con 0x7f57300d3870 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.496+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5730157d80 con 0x7f57300d3870 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.497+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f573004e8b0 con 0x7f57300d3870 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.498+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5720013470 con 0x7f57300d3870 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.499+0000 7f5726ffd640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f571406ae40 0x7f571406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.499+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57200898c0 con 0x7f57300d3870 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.500+0000 7f5736560640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f571406ae40 0x7f571406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.500+0000 7f5736560640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f571406ae40 0x7f571406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f572c036000 tx=0x7f572c03d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.504+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f57200548d0 con 0x7f57300d3870 2023-11-24T08:09:00.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.694+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f57300ca150 con 0x7f57300d3870 2023-11-24T08:09:00.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.695+0000 7f5726ffd640 1 -- 172.21.15.106:0/3981600542 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f57200170a0 con 0x7f57300d3870 2023-11-24T08:09:00.698 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.697+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f571406ae40 msgr2=0x7f571406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.697+0000 7f57387c4640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f571406ae40 0x7f571406d300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f572c036000 tx=0x7f572c03d040 comp rx=0 tx=0).stop 2023-11-24T08:09:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.697+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 msgr2=0x7f573015bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.697+0000 7f57387c4640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 0x7f573015bc40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5720009fd0 tx=0x7f572000ed40 comp rx=0 tx=0).stop 2023-11-24T08:09:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 shutdown_connections 2023-11-24T08:09:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f571406ae40 0x7f571406d300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f573015d720 0x7f573015e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f57300d3870 0x7f573015bc40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 --2- 172.21.15.106:0/3981600542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57300b1400 0x7f573015f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 >> 172.21.15.106:0/3981600542 conn(0x7f573008af60 msgr2=0x7f57300cf900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 shutdown_connections 2023-11-24T08:09:00.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.698+0000 7f57387c4640 1 -- 172.21.15.106:0/3981600542 wait complete. 2023-11-24T08:09:00.720 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:00.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.927+0000 7f5f8d5d5640 1 Processor -- start 2023-11-24T08:09:00.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.927+0000 7f5f8d5d5640 1 -- start start 2023-11-24T08:09:00.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f8d5d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f880ce6c0 0x7f5f880cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f8d5d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f880cece0 con 0x7f5f8815c7f0 2023-11-24T08:09:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f8d5d5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f8806d040 con 0x7f5f880cf450 2023-11-24T08:09:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f8d5d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f880cfe40 con 0x7f5f880ce6c0 2023-11-24T08:09:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f877fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f880ce6c0 0x7f5f880cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f86ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5f8815c7f0 0x7f5f8806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32878/0 (socket says 172.21.15.106:32878) 2023-11-24T08:09:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.929+0000 7f5f86ffd640 1 -- 172.21.15.106:0/3642601732 learned_addr learned my addr 172.21.15.106:0/3642601732 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.930+0000 7f5f877fe640 1 -- 172.21.15.106:0/3642601732 >> v1:172.21.15.156:6789/0 conn(0x7f5f880cf450 legacy=0x7f5f880cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.930+0000 7f5f877fe640 1 -- 172.21.15.106:0/3642601732 >> v1:172.21.15.106:6789/0 conn(0x7f5f8815c7f0 legacy=0x7f5f8806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.930+0000 7f5f877fe640 1 -- 172.21.15.106:0/3642601732 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8806d320 con 0x7f5f880ce6c0 2023-11-24T08:09:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.930+0000 7f5f867fc640 1 -- 172.21.15.106:0/3642601732 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3832273112 0 0) 0x7f5f880cece0 con 0x7f5f8815c7f0 2023-11-24T08:09:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.931+0000 7f5f877fe640 1 --2- 172.21.15.106:0/3642601732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f880ce6c0 0x7f5f880cf900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009d90 tx=0x7f5f8002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4173d03f324b1a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.931+0000 7f5f867fc640 1 -- 172.21.15.106:0/3642601732 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8002cc60 con 0x7f5f880ce6c0 2023-11-24T08:09:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.931+0000 7f5f867fc640 1 -- 172.21.15.106:0/3642601732 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8002cdc0 con 0x7f5f880ce6c0 2023-11-24T08:09:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.931+0000 7f5f867fc640 1 -- 172.21.15.106:0/3642601732 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f80034660 con 0x7f5f880ce6c0 2023-11-24T08:09:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.931+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/3642601732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f880ce6c0 msgr2=0x7f5f880cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.931+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/3642601732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f880ce6c0 0x7f5f880cf900 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009d90 tx=0x7f5f8002beb0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.932+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/3642601732 shutdown_connections 2023-11-24T08:09:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.932+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/3642601732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f880ce6c0 0x7f5f880cf900 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.932+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/3642601732 >> 172.21.15.106:0/3642601732 conn(0x7f5f8806a920 msgr2=0x7f5f8806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.932+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/3642601732 shutdown_connections 2023-11-24T08:09:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.932+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/3642601732 wait complete. 2023-11-24T08:09:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.933+0000 7f5f8d5d5640 1 Processor -- start 2023-11-24T08:09:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 -- start start 2023-11-24T08:09:00.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f880ce6c0 0x7f5f880b2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f880cf450 0x7f5f881515d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 0x7f5f880af610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f88151b10 con 0x7f5f8815c7f0 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b32a0 con 0x7f5f880ce6c0 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.934+0000 7f5f8d5d5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f880b3580 con 0x7f5f880cf450 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 0x7f5f880af610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 0x7f5f880af610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34662/0 (socket says 172.21.15.106:34662) 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 -- 172.21.15.106:0/4222486 learned_addr learned my addr 172.21.15.106:0/4222486 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f7ffff640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f880cf450 0x7f5f881515d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f86ffd640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f880ce6c0 0x7f5f880b2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 -- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f880cf450 msgr2=0x7f5f881515d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f880cf450 0x7f5f881515d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 -- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f880ce6c0 msgr2=0x7f5f880b2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f880ce6c0 0x7f5f880b2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 -- 172.21.15.106:0/4222486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80009a70 con 0x7f5f8815c7f0 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f7ffff640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f880cf450 0x7f5f881515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f86ffd640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f880ce6c0 0x7f5f880b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.935+0000 7f5f877fe640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 0x7f5f880af610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009d90 tx=0x7f5f80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.936+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8003e610 con 0x7f5f8815c7f0 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.936+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8003e770 con 0x7f5f8815c7f0 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.936+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8003eb60 con 0x7f5f8815c7f0 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.936+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f880b3860 con 0x7f5f8815c7f0 2023-11-24T08:09:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.936+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f880c8400 con 0x7f5f8815c7f0 2023-11-24T08:09:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.937+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f880c85f0 con 0x7f5f8815c7f0 2023-11-24T08:09:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.938+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5f8003b050 con 0x7f5f8815c7f0 2023-11-24T08:09:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.939+0000 7f5f7f7fe640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f6806ae40 0x7f5f6806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.939+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f800abff0 con 0x7f5f8815c7f0 2023-11-24T08:09:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.939+0000 7f5f86ffd640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f6806ae40 0x7f5f6806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.940+0000 7f5f86ffd640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f6806ae40 0x7f5f6806d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000f60 tx=0x7f5f78009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:00.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:00.943+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f80077270 con 0x7f5f8815c7f0 2023-11-24T08:09:01.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.176+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5f88067470 con 0x7f5f8815c7f0 2023-11-24T08:09:01.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.177+0000 7f5f7f7fe640 1 -- 172.21.15.106:0/4222486 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f5f80039090 con 0x7f5f8815c7f0 2023-11-24T08:09:01.179 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:01.180 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:01.180 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.179+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f6806ae40 msgr2=0x7f5f6806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.179+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f6806ae40 0x7f5f6806d300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5f78000f60 tx=0x7f5f78009040 comp rx=0 tx=0).stop 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.179+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 msgr2=0x7f5f880af610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.179+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 0x7f5f880af610 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5f80009d90 tx=0x7f5f80004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 shutdown_connections 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f8815c7f0 0x7f5f880af610 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f6806ae40 0x7f5f6806d300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f880cf450 0x7f5f881515d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 --2- 172.21.15.106:0/4222486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f880ce6c0 0x7f5f880b2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 >> 172.21.15.106:0/4222486 conn(0x7f5f8806a920 msgr2=0x7f5f8806c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 shutdown_connections 2023-11-24T08:09:01.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.180+0000 7f5f8d5d5640 1 -- 172.21.15.106:0/4222486 wait complete. 2023-11-24T08:09:01.201 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.411+0000 7fa39b183640 1 Processor -- start 2023-11-24T08:09:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.411+0000 7fa39b183640 1 -- start start 2023-11-24T08:09:01.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.412+0000 7fa39b183640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa3940b2250 0x7fa39415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.412+0000 7fa39b183640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3940b2630 0x7fa3940ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.412+0000 7fa39b183640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2aa0 0x7fa39415b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.412+0000 7fa39b183640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39415a990 con 0x7fa3940b2630 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.412+0000 7fa39b183640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39415ab80 con 0x7fa3940b2250 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.412+0000 7fa39b183640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa39415e680 con 0x7fa3940b2aa0 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3940b2630 0x7fa3940ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3940b2630 0x7fa3940ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34674/0 (socket says 172.21.15.106:34674) 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 -- 172.21.15.106:0/3397291440 learned_addr learned my addr 172.21.15.106:0/3397291440 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa399720640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2aa0 0x7fa39415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 -- 172.21.15.106:0/3397291440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2aa0 msgr2=0x7fa39415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2aa0 0x7fa39415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 -- 172.21.15.106:0/3397291440 >> v2:172.21.15.156:3300/0 conn(0x7fa3940b2250 msgr2=0x7fa39415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 --2- 172.21.15.106:0/3397291440 >> v2:172.21.15.156:3300/0 conn(0x7fa3940b2250 0x7fa39415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa393fff640 1 -- 172.21.15.106:0/3397291440 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39415e960 con 0x7fa3940b2630 2023-11-24T08:09:01.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.413+0000 7fa399720640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2aa0 0x7fa39415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.414+0000 7fa393fff640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2630 0x7fa3940ccf30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa394072a70 tx=0x7fa38002e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64cd5d323a1101b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.414+0000 7fa3937fe640 1 -- 172.21.15.106:0/3397291440 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38002ee80 con 0x7fa3940b2630 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa3937fe640 1 -- 172.21.15.106:0/3397291440 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa380036440 con 0x7fa3940b2630 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa3937fe640 1 -- 172.21.15.106:0/3397291440 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3800367b0 con 0x7fa3940b2630 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 -- 172.21.15.106:0/3397291440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2630 msgr2=0x7fa3940ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2630 0x7fa3940ccf30 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa394072a70 tx=0x7fa38002e0d0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 -- 172.21.15.106:0/3397291440 shutdown_connections 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2aa0 0x7fa39415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 --2- 172.21.15.106:0/3397291440 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2630 0x7fa3940ccf30 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 --2- 172.21.15.106:0/3397291440 >> v2:172.21.15.156:3300/0 conn(0x7fa3940b2250 0x7fa39415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 -- 172.21.15.106:0/3397291440 >> 172.21.15.106:0/3397291440 conn(0x7fa39406a920 msgr2=0x7fa39406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 -- 172.21.15.106:0/3397291440 shutdown_connections 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.415+0000 7fa39b183640 1 -- 172.21.15.106:0/3397291440 wait complete. 2023-11-24T08:09:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.417+0000 7fa39b183640 1 Processor -- start 2023-11-24T08:09:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.417+0000 7fa39b183640 1 -- start start 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa39b183640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 0x7fa3941463d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa39b183640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2630 0x7fa39415b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa39b183640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3940b2aa0 0x7fa394155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa39b183640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa394151f70 con 0x7fa3940b2250 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa39b183640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3941513a0 con 0x7fa3940b2aa0 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa39b183640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa394151640 con 0x7fa3940b2630 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa398f1f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 0x7fa3941463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa398f1f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 0x7fa3941463d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34684/0 (socket says 172.21.15.106:34684) 2023-11-24T08:09:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.418+0000 7fa398f1f640 1 -- 172.21.15.106:0/3436870950 learned_addr learned my addr 172.21.15.106:0/3436870950 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa398f1f640 1 -- 172.21.15.106:0/3436870950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2630 msgr2=0x7fa39415b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa398f1f640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2630 0x7fa39415b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa398f1f640 1 -- 172.21.15.106:0/3436870950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3940b2aa0 msgr2=0x7fa394155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa398f1f640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3940b2aa0 0x7fa394155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa398f1f640 1 -- 172.21.15.106:0/3436870950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38000b770 con 0x7fa3940b2250 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa398f1f640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 0x7fa3941463d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b4a0 tx=0x7fa38800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388010610 con 0x7fa3940b2250 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa388010770 con 0x7fa3940b2250 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa388010b20 con 0x7fa3940b2250 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa394151920 con 0x7fa3940b2250 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.419+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa394146910 con 0x7fa3940b2250 2023-11-24T08:09:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.420+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa394067470 con 0x7fa3940b2250 2023-11-24T08:09:01.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.421+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa388010c80 con 0x7fa3940b2250 2023-11-24T08:09:01.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.422+0000 7fa3917fa640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa37806ae70 0x7fa37806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.422+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3880892c0 con 0x7fa3940b2250 2023-11-24T08:09:01.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.422+0000 7fa393fff640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa37806ae70 0x7fa37806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.423+0000 7fa393fff640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa37806ae70 0x7fa37806d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa380038000 tx=0x7fa380030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.426+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3880542d0 con 0x7fa3940b2250 2023-11-24T08:09:01.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.615+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa3940cea00 con 0x7fa3940b2250 2023-11-24T08:09:01.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.618+0000 7fa3917fa640 1 -- 172.21.15.106:0/3436870950 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fa3880578f0 con 0x7fa3940b2250 2023-11-24T08:09:01.622 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:01.622 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa37806ae70 msgr2=0x7fa37806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa37806ae70 0x7fa37806d330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fa380038000 tx=0x7fa380030000 comp rx=0 tx=0).stop 2023-11-24T08:09:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 msgr2=0x7fa3941463d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 0x7fa3941463d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa38800b4a0 tx=0x7fa38800b960 comp rx=0 tx=0).stop 2023-11-24T08:09:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 shutdown_connections 2023-11-24T08:09:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3940b2aa0 0x7fa394155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa37806ae70 0x7fa37806d330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3940b2630 0x7fa39415b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 --2- 172.21.15.106:0/3436870950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3940b2250 0x7fa3941463d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 >> 172.21.15.106:0/3436870950 conn(0x7fa39406a920 msgr2=0x7fa3941597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 shutdown_connections 2023-11-24T08:09:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.621+0000 7fa39b183640 1 -- 172.21.15.106:0/3436870950 wait complete. 2023-11-24T08:09:01.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:01.642 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:01.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.843+0000 7fa89527c640 1 Processor -- start 2023-11-24T08:09:01.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.843+0000 7fa89527c640 1 -- start start 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.844+0000 7fa89527c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa89015b440 0x7fa8900cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.844+0000 7fa89527c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8900c59b0 0x7fa89006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.844+0000 7fa89527c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8900c5dc0 con 0x7fa8900cbf20 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa89527c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8900cc3d0 con 0x7fa8900c59b0 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa89527c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89015b820 con 0x7fa89015b440 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8900c59b0 0x7fa89006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88e7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa8900cbf20 0x7fa8900c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32880/0 (socket says 172.21.15.106:32880) 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88e7fc640 1 -- 172.21.15.106:0/393121738 learned_addr learned my addr 172.21.15.106:0/393121738 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:01.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88effd640 1 --2- 172.21.15.106:0/393121738 >> v2:172.21.15.156:3300/0 conn(0x7fa89015b440 0x7fa8900cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88f7fe640 1 -- 172.21.15.106:0/393121738 >> v2:172.21.15.156:3300/0 conn(0x7fa89015b440 msgr2=0x7fa8900cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:01.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88f7fe640 1 --2- 172.21.15.106:0/393121738 >> v2:172.21.15.156:3300/0 conn(0x7fa89015b440 0x7fa8900cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88f7fe640 1 -- 172.21.15.106:0/393121738 >> v1:172.21.15.106:6789/0 conn(0x7fa8900cbf20 legacy=0x7fa8900c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88dffb640 1 -- 172.21.15.106:0/393121738 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123940928 0 0) 0x7fa8900c5dc0 con 0x7fa8900cbf20 2023-11-24T08:09:01.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.845+0000 7fa88f7fe640 1 -- 172.21.15.106:0/393121738 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89015eea0 con 0x7fa8900c59b0 2023-11-24T08:09:01.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.847+0000 7fa88f7fe640 1 --2- 172.21.15.106:0/393121738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa89006c5f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa89006e770 tx=0x7fa88802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6203e62dd3eb7e6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.847+0000 7fa88dffb640 1 -- 172.21.15.106:0/393121738 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88802cbf0 con 0x7fa8900c59b0 2023-11-24T08:09:01.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.847+0000 7fa88dffb640 1 -- 172.21.15.106:0/393121738 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88802cd50 con 0x7fa8900c59b0 2023-11-24T08:09:01.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.847+0000 7fa88dffb640 1 -- 172.21.15.106:0/393121738 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888034660 con 0x7fa8900c59b0 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.847+0000 7fa89527c640 1 -- 172.21.15.106:0/393121738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 msgr2=0x7fa89006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.847+0000 7fa89527c640 1 --2- 172.21.15.106:0/393121738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa89006c5f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa89006e770 tx=0x7fa88802be40 comp rx=0 tx=0).stop 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.848+0000 7fa89527c640 1 -- 172.21.15.106:0/393121738 shutdown_connections 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.848+0000 7fa89527c640 1 --2- 172.21.15.106:0/393121738 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa89006c5f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.848+0000 7fa89527c640 1 --2- 172.21.15.106:0/393121738 >> v2:172.21.15.156:3300/0 conn(0x7fa89015b440 0x7fa8900cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.848+0000 7fa89527c640 1 -- 172.21.15.106:0/393121738 >> 172.21.15.106:0/393121738 conn(0x7fa8900b0c50 msgr2=0x7fa8900b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.848+0000 7fa89527c640 1 -- 172.21.15.106:0/393121738 shutdown_connections 2023-11-24T08:09:01.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.848+0000 7fa89527c640 1 -- 172.21.15.106:0/393121738 wait complete. 2023-11-24T08:09:01.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.849+0000 7fa89527c640 1 Processor -- start 2023-11-24T08:09:01.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.850+0000 7fa89527c640 1 -- start start 2023-11-24T08:09:01.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa89527c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa890154c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa89527c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8900cbf20 0x7fa89015a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa89527c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa89015b440 0x7fa890156500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa89527c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa890155150 con 0x7fa8900c59b0 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa89527c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8900cda30 con 0x7fa8900cbf20 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa89527c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89015abb0 con 0x7fa89015b440 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa890154c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa890154c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34692/0 (socket says 172.21.15.106:34692) 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 -- 172.21.15.106:0/4181812555 learned_addr learned my addr 172.21.15.106:0/4181812555 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 -- 172.21.15.106:0/4181812555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa89015b440 msgr2=0x7fa890156500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa89015b440 0x7fa890156500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 -- 172.21.15.106:0/4181812555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8900cbf20 msgr2=0x7fa89015a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:01.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8900cbf20 0x7fa89015a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:01.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 -- 172.21.15.106:0/4181812555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888009a70 con 0x7fa8900c59b0 2023-11-24T08:09:01.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.851+0000 7fa88effd640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa890154c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa88000d5f0 tx=0x7fa88000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.852+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800125f0 con 0x7fa8900c59b0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.852+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880012750 con 0x7fa8900c59b0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.852+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa880012b00 con 0x7fa8900c59b0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.852+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8900cd190 con 0x7fa8900c59b0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.852+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8900cd750 con 0x7fa8900c59b0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.853+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8900ca980 con 0x7fa8900c59b0 2023-11-24T08:09:01.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.854+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa880012c60 con 0x7fa8900c59b0 2023-11-24T08:09:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.855+0000 7fa8777fe640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa87c06ae40 0x7fa87c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:01.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.855+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa880089b50 con 0x7fa8900c59b0 2023-11-24T08:09:01.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.855+0000 7fa88e7fc640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa87c06ae40 0x7fa87c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:01.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.856+0000 7fa88e7fc640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa87c06ae40 0x7fa87c06d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa884007fd0 tx=0x7fa884007920 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:01.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:01.860+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa880054b60 con 0x7fa8900c59b0 2023-11-24T08:09:02.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.037+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa89004e8b0 con 0x7fa8900c59b0 2023-11-24T08:09:02.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.037+0000 7fa8777fe640 1 -- 172.21.15.106:0/4181812555 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa880058180 con 0x7fa8900c59b0 2023-11-24T08:09:02.040 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.039+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa87c06ae40 msgr2=0x7fa87c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.039+0000 7fa89527c640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa87c06ae40 0x7fa87c06d300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa884007fd0 tx=0x7fa884007920 comp rx=0 tx=0).stop 2023-11-24T08:09:02.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 msgr2=0x7fa890154c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa890154c10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa88000d5f0 tx=0x7fa88000dab0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 shutdown_connections 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa87c06ae40 0x7fa87c06d300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa89015b440 0x7fa890156500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8900cbf20 0x7fa89015a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 --2- 172.21.15.106:0/4181812555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8900c59b0 0x7fa890154c10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 >> 172.21.15.106:0/4181812555 conn(0x7fa8900b0c50 msgr2=0x7fa8900718e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 shutdown_connections 2023-11-24T08:09:02.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.040+0000 7fa89527c640 1 -- 172.21.15.106:0/4181812555 wait complete. 2023-11-24T08:09:02.061 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.274+0000 7fcad1dba640 1 Processor -- start 2023-11-24T08:09:02.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.274+0000 7fcad1dba640 1 -- start start 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad1dba640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcacc15a1c0 0x7fcacc06d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad1dba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcacc0b0b40 0x7fcacc0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad1dba640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcacc06d430 con 0x7fcacc15ded0 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad1dba640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc15ea70 con 0x7fcacc0b0b40 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad1dba640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcacc15ec50 con 0x7fcacc15a1c0 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcacbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcacc0b0b40 0x7fcacc0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad15b9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcacc15ded0 0x7fcacc15e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32896/0 (socket says 172.21.15.106:32896) 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad15b9640 1 -- 172.21.15.106:0/1164851480 learned_addr learned my addr 172.21.15.106:0/1164851480 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.276+0000 7fcad0db8640 1 --2- 172.21.15.106:0/1164851480 >> v2:172.21.15.156:3300/0 conn(0x7fcacc15a1c0 0x7fcacc06d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.277+0000 7fcacbfff640 1 -- 172.21.15.106:0/1164851480 >> v2:172.21.15.156:3300/0 conn(0x7fcacc15a1c0 msgr2=0x7fcacc06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.277+0000 7fcacbfff640 1 --2- 172.21.15.106:0/1164851480 >> v2:172.21.15.156:3300/0 conn(0x7fcacc15a1c0 0x7fcacc06d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.277+0000 7fcacbfff640 1 -- 172.21.15.106:0/1164851480 >> v1:172.21.15.106:6789/0 conn(0x7fcacc15ded0 legacy=0x7fcacc15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.277+0000 7fcacbfff640 1 -- 172.21.15.106:0/1164851480 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcacc0b14e0 con 0x7fcacc0b0b40 2023-11-24T08:09:02.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.277+0000 7fcacb7fe640 1 -- 172.21.15.106:0/1164851480 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 211064399 0 0) 0x7fcacc06d430 con 0x7fcacc15ded0 2023-11-24T08:09:02.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.278+0000 7fcacbfff640 1 --2- 172.21.15.106:0/1164851480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc0b0b40 0x7fcacc0b0fa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0cca80 tx=0x7fcab802e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfa79e17cc5d606e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.278+0000 7fcacb7fe640 1 -- 172.21.15.106:0/1164851480 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab802ee50 con 0x7fcacc0b0b40 2023-11-24T08:09:02.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.278+0000 7fcacb7fe640 1 -- 172.21.15.106:0/1164851480 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab8036440 con 0x7fcacc0b0b40 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.278+0000 7fcacb7fe640 1 -- 172.21.15.106:0/1164851480 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab80367d0 con 0x7fcacc0b0b40 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 -- 172.21.15.106:0/1164851480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc0b0b40 msgr2=0x7fcacc0b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 --2- 172.21.15.106:0/1164851480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc0b0b40 0x7fcacc0b0fa0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0cca80 tx=0x7fcab802e0a0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 -- 172.21.15.106:0/1164851480 shutdown_connections 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 --2- 172.21.15.106:0/1164851480 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc0b0b40 0x7fcacc0b0fa0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 --2- 172.21.15.106:0/1164851480 >> v2:172.21.15.156:3300/0 conn(0x7fcacc15a1c0 0x7fcacc06d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 -- 172.21.15.106:0/1164851480 >> 172.21.15.106:0/1164851480 conn(0x7fcacc0cbb40 msgr2=0x7fcacc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 -- 172.21.15.106:0/1164851480 shutdown_connections 2023-11-24T08:09:02.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.279+0000 7fcad1dba640 1 -- 172.21.15.106:0/1164851480 wait complete. 2023-11-24T08:09:02.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.281+0000 7fcad1dba640 1 Processor -- start 2023-11-24T08:09:02.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.281+0000 7fcad1dba640 1 -- start start 2023-11-24T08:09:02.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad1dba640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcacc0b0b40 0x7fcacc152590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad1dba640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcacc15a1c0 0x7fcacc0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad1dba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 0x7fcacc15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad1dba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b3150 con 0x7fcacc15ded0 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad1dba640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b24d0 con 0x7fcacc15a1c0 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad1dba640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc0b27b0 con 0x7fcacc0b0b40 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 0x7fcacc15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 0x7fcacc15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34704/0 (socket says 172.21.15.106:34704) 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 -- 172.21.15.106:0/2155146153 learned_addr learned my addr 172.21.15.106:0/2155146153 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcacbfff640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcacc15a1c0 0x7fcacc0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad0db8640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcacc0b0b40 0x7fcacc152590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 -- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcacc0b0b40 msgr2=0x7fcacc152590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcacc0b0b40 0x7fcacc152590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 -- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcacc15a1c0 msgr2=0x7fcacc0b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcacc15a1c0 0x7fcacc0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.282+0000 7fcad15b9640 1 -- 172.21.15.106:0/2155146153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab800b6a0 con 0x7fcacc15ded0 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.283+0000 7fcad15b9640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 0x7fcacc15c380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcac400c700 tx=0x7fcac400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.283+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400ce10 con 0x7fcacc15ded0 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.283+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcac400a640 con 0x7fcacc15ded0 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.283+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400aa70 con 0x7fcacc15ded0 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.283+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc0b2a90 con 0x7fcacc15ded0 2023-11-24T08:09:02.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.283+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc152ad0 con 0x7fcacc15ded0 2023-11-24T08:09:02.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.284+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc0b3290 con 0x7fcacc15ded0 2023-11-24T08:09:02.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.285+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcac400abd0 con 0x7fcacc15ded0 2023-11-24T08:09:02.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.286+0000 7fcac97fa640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcab006ae40 0x7fcab006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.286+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcac4088b60 con 0x7fcacc15ded0 2023-11-24T08:09:02.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.286+0000 7fcad0db8640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcab006ae40 0x7fcab006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.287+0000 7fcad0db8640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcab006ae40 0x7fcab006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006fd0 tx=0x7fcac0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.290+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcacc0b3290 con 0x7fcacc15ded0 2023-11-24T08:09:02.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.482+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcacc0b3290 con 0x7fcacc15ded0 2023-11-24T08:09:02.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.483+0000 7fcac97fa640 1 -- 172.21.15.106:0/2155146153 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcacc0b3290 con 0x7fcacc15ded0 2023-11-24T08:09:02.485 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:02.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.485+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcab006ae40 msgr2=0x7fcab006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.485+0000 7fcad1dba640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcab006ae40 0x7fcab006d300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcac0006fd0 tx=0x7fcac0008040 comp rx=0 tx=0).stop 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 msgr2=0x7fcacc15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 0x7fcacc15c380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcac400c700 tx=0x7fcac400cbc0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 shutdown_connections 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcacc15ded0 0x7fcacc15c380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcacc15a1c0 0x7fcacc0b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcab006ae40 0x7fcab006d300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 --2- 172.21.15.106:0/2155146153 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcacc0b0b40 0x7fcacc152590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 >> 172.21.15.106:0/2155146153 conn(0x7fcacc0cbb40 msgr2=0x7fcacc0cd500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 shutdown_connections 2023-11-24T08:09:02.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.486+0000 7fcad1dba640 1 -- 172.21.15.106:0/2155146153 wait complete. 2023-11-24T08:09:02.508 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.710+0000 7f43d8c5c640 1 Processor -- start 2023-11-24T08:09:02.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.711+0000 7f43d8c5c640 1 -- start start 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.712+0000 7f43d8c5c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43d415b440 0x7f43d40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.712+0000 7f43d8c5c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43d40cbf20 0x7f43d40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.712+0000 7f43d8c5c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d415eea0 con 0x7f43d40c57e0 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.712+0000 7f43d8c5c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d40cc3d0 con 0x7f43d415b440 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.712+0000 7f43d8c5c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d415b820 con 0x7f43d40cbf20 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d259c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43d415b440 0x7f43d40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f43d40c57e0 0x7f43d40c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32898/0 (socket says 172.21.15.106:32898) 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d2d9d640 1 -- 172.21.15.106:0/2044121170 learned_addr learned my addr 172.21.15.106:0/2044121170 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:02.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d1d9b640 1 --2- 172.21.15.106:0/2044121170 >> v2:172.21.15.156:3300/0 conn(0x7f43d40cbf20 0x7f43d40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d259c640 1 -- 172.21.15.106:0/2044121170 >> v2:172.21.15.156:3300/0 conn(0x7f43d40cbf20 msgr2=0x7f43d40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:02.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d259c640 1 --2- 172.21.15.106:0/2044121170 >> v2:172.21.15.156:3300/0 conn(0x7f43d40cbf20 0x7f43d40c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d259c640 1 -- 172.21.15.106:0/2044121170 >> v1:172.21.15.106:6789/0 conn(0x7f43d40c57e0 legacy=0x7f43d40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d159a640 1 -- 172.21.15.106:0/2044121170 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1313623251 0 0) 0x7f43d415eea0 con 0x7f43d40c57e0 2023-11-24T08:09:02.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.713+0000 7f43d259c640 1 -- 172.21.15.106:0/2044121170 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d406d600 con 0x7f43d415b440 2023-11-24T08:09:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.714+0000 7f43d259c640 1 --2- 172.21.15.106:0/2044121170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40cb9e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43d406e890 tx=0x7f43c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73416b19caf3383 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d159a640 1 -- 172.21.15.106:0/2044121170 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c402e070 con 0x7f43d415b440 2023-11-24T08:09:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d159a640 1 -- 172.21.15.106:0/2044121170 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c40315a0 con 0x7f43d415b440 2023-11-24T08:09:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d159a640 1 -- 172.21.15.106:0/2044121170 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c4031930 con 0x7f43d415b440 2023-11-24T08:09:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 -- 172.21.15.106:0/2044121170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 msgr2=0x7f43d40cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/2044121170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40cb9e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f43d406e890 tx=0x7f43c402b840 comp rx=0 tx=0).stop 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 -- 172.21.15.106:0/2044121170 shutdown_connections 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/2044121170 >> v2:172.21.15.156:3300/0 conn(0x7f43d40cbf20 0x7f43d40c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/2044121170 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40cb9e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 -- 172.21.15.106:0/2044121170 >> 172.21.15.106:0/2044121170 conn(0x7f43d40b0c50 msgr2=0x7f43d40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 -- 172.21.15.106:0/2044121170 shutdown_connections 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.715+0000 7f43d8c5c640 1 -- 172.21.15.106:0/2044121170 wait complete. 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.717+0000 7f43d8c5c640 1 Processor -- start 2023-11-24T08:09:02.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.717+0000 7f43d8c5c640 1 -- start start 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d8c5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43d40c57e0 0x7f43d4154af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d8c5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43d40cbf20 0x7f43d40c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d8c5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d8c5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40cd900 con 0x7f43d415b440 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d8c5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40cdbe0 con 0x7f43d40c57e0 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d8c5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d40cdec0 con 0x7f43d40cbf20 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34724/0 (socket says 172.21.15.106:34724) 2023-11-24T08:09:02.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.718+0000 7f43d2d9d640 1 -- 172.21.15.106:0/4287928545 learned_addr learned my addr 172.21.15.106:0/4287928545 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:02.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d2d9d640 1 -- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43d40cbf20 msgr2=0x7f43d40c6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d259c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43d40c57e0 0x7f43d4154af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d1d9b640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43d40cbf20 0x7f43d40c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d2d9d640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43d40cbf20 0x7f43d40c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d2d9d640 1 -- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43d40c57e0 msgr2=0x7f43d4154af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d2d9d640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43d40c57e0 0x7f43d4154af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d2d9d640 1 -- 172.21.15.106:0/4287928545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c4009400 con 0x7f43d415b440 2023-11-24T08:09:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43d2d9d640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40c6e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00c450 tx=0x7f43cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc0074e0 con 0x7f43d415b440 2023-11-24T08:09:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.719+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43cc007660 con 0x7f43d415b440 2023-11-24T08:09:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.720+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43cc004a20 con 0x7f43d415b440 2023-11-24T08:09:02.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.720+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d40ce1a0 con 0x7f43d415b440 2023-11-24T08:09:02.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.720+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d4155030 con 0x7f43d415b440 2023-11-24T08:09:02.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.721+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d40ce6c0 con 0x7f43d415b440 2023-11-24T08:09:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.722+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f43cc004b80 con 0x7f43d415b440 2023-11-24T08:09:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.723+0000 7f43caffd640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43b406ae40 0x7f43b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.723+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43cc088890 con 0x7f43d415b440 2023-11-24T08:09:02.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.723+0000 7f43d259c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43b406ae40 0x7f43b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:02.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.724+0000 7f43d259c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43b406ae40 0x7f43b406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43c4036000 tx=0x7f43c403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:02.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.727+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43d40ce6c0 con 0x7f43d415b440 2023-11-24T08:09:02.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.916+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f43d40ce6c0 con 0x7f43d415b440 2023-11-24T08:09:02.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.917+0000 7f43caffd640 1 -- 172.21.15.106:0/4287928545 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f43d40ce6c0 con 0x7f43d415b440 2023-11-24T08:09:02.919 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:02.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43b406ae40 msgr2=0x7f43b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43b406ae40 0x7f43b406d300 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f43c4036000 tx=0x7f43c403d040 comp rx=0 tx=0).stop 2023-11-24T08:09:02.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 msgr2=0x7f43d40c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:02.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40c6e90 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f43cc00c450 tx=0x7f43cc00c910 comp rx=0 tx=0).stop 2023-11-24T08:09:02.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 shutdown_connections 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43d415b440 0x7f43d40c6e90 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43b406ae40 0x7f43b406d300 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43d40cbf20 0x7f43d40c6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 --2- 172.21.15.106:0/4287928545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43d40c57e0 0x7f43d4154af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.919+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 >> 172.21.15.106:0/4287928545 conn(0x7f43d40b0c50 msgr2=0x7f43d4155b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.920+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 shutdown_connections 2023-11-24T08:09:02.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:02.920+0000 7f43d8c5c640 1 -- 172.21.15.106:0/4287928545 wait complete. 2023-11-24T08:09:02.940 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:03.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.147+0000 7f11d604b640 1 Processor -- start 2023-11-24T08:09:03.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.148+0000 7f11d604b640 1 -- start start 2023-11-24T08:09:03.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.149+0000 7f11d604b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d013d240 0x7f11d00b30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.149+0000 7f11d604b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f11d00b35f0 0x7f11d015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.149+0000 7f11d604b640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11d00b3a50 con 0x7f11d015e420 2023-11-24T08:09:03.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.149+0000 7f11d604b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11d015b230 con 0x7f11d00b35f0 2023-11-24T08:09:03.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.149+0000 7f11d604b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d006f3c0 con 0x7f11d013d240 2023-11-24T08:09:03.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.150+0000 7f11d5049640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d013d240 0x7f11d00b30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.150+0000 7f11d5049640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d013d240 0x7f11d00b30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:48680/0 (socket says 172.21.15.106:48680) 2023-11-24T08:09:03.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.150+0000 7f11d5049640 1 -- 172.21.15.106:0/1129432212 learned_addr learned my addr 172.21.15.106:0/1129432212 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:03.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.150+0000 7f11d4848640 1 --2- 172.21.15.106:0/1129432212 >> v2:172.21.15.156:3300/0 conn(0x7f11d00b35f0 0x7f11d015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.151+0000 7f11d4848640 1 -- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d013d240 msgr2=0x7f11d00b30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.151+0000 7f11d4848640 1 --2- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d013d240 0x7f11d00b30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.151+0000 7f11d4848640 1 -- 172.21.15.106:0/1129432212 >> v1:172.21.15.156:6789/0 conn(0x7f11d015e420 legacy=0x7f11d0071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.151+0000 7f11d4848640 1 -- 172.21.15.106:0/1129432212 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d015a680 con 0x7f11d00b35f0 2023-11-24T08:09:03.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.153+0000 7f11d4848640 1 --2- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d015dee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009790 tx=0x7f11c002b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f573abdb284640 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.153+0000 7f11c7fff640 1 -- 172.21.15.106:0/1129432212 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031800 con 0x7f11d00b35f0 2023-11-24T08:09:03.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.153+0000 7f11c7fff640 1 -- 172.21.15.106:0/1129432212 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c0031960 con 0x7f11d00b35f0 2023-11-24T08:09:03.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.153+0000 7f11c7fff640 1 -- 172.21.15.106:0/1129432212 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0031cf0 con 0x7f11d00b35f0 2023-11-24T08:09:03.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 -- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 msgr2=0x7f11d015dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 --2- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d015dee0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f11c0009790 tx=0x7f11c002b8a0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 -- 172.21.15.106:0/1129432212 shutdown_connections 2023-11-24T08:09:03.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 --2- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d015dee0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 --2- 172.21.15.106:0/1129432212 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d013d240 0x7f11d00b30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 -- 172.21.15.106:0/1129432212 >> 172.21.15.106:0/1129432212 conn(0x7f11d00ccb00 msgr2=0x7f11d00ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 -- 172.21.15.106:0/1129432212 shutdown_connections 2023-11-24T08:09:03.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.154+0000 7f11d604b640 1 -- 172.21.15.106:0/1129432212 wait complete. 2023-11-24T08:09:03.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.156+0000 7f11d604b640 1 Processor -- start 2023-11-24T08:09:03.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.157+0000 7f11d604b640 1 -- start start 2023-11-24T08:09:03.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d604b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d01f65f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d604b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 0x7f11d00cf650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d604b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d015e420 0x7f11d013c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d604b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d013b370 con 0x7f11d013d240 2023-11-24T08:09:03.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d604b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d013b650 con 0x7f11d00b35f0 2023-11-24T08:09:03.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d604b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d013b930 con 0x7f11d015e420 2023-11-24T08:09:03.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d4848640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 0x7f11d00cf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d4848640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 0x7f11d00cf650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34740/0 (socket says 172.21.15.106:34740) 2023-11-24T08:09:03.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.158+0000 7f11d4848640 1 -- 172.21.15.106:0/1433837176 learned_addr learned my addr 172.21.15.106:0/1433837176 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:03.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d584a640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d015e420 0x7f11d013c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d5049640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d01f65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d4848640 1 -- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d015e420 msgr2=0x7f11d013c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d4848640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d015e420 0x7f11d013c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d4848640 1 -- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 msgr2=0x7f11d01f65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d4848640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d01f65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d4848640 1 -- 172.21.15.106:0/1433837176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c00093a0 con 0x7f11d013d240 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11d4848640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 0x7f11d00cf650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11c002beb0 tx=0x7f11c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.159+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c003bc30 con 0x7f11d013d240 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.160+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11c003bdb0 con 0x7f11d013d240 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.160+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c0045630 con 0x7f11d013d240 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.160+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d013bc10 con 0x7f11d013d240 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.160+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d01f7bf0 con 0x7f11d013d240 2023-11-24T08:09:03.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.161+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d0168090 con 0x7f11d013d240 2023-11-24T08:09:03.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.162+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f11c004e450 con 0x7f11d013d240 2023-11-24T08:09:03.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.162+0000 7f11c5ffb640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f11b406ae40 0x7f11b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.162+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11c002e070 con 0x7f11d013d240 2023-11-24T08:09:03.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.163+0000 7f11d5049640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f11b406ae40 0x7f11b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.163+0000 7f11d5049640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f11b406ae40 0x7f11b406d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009b60 tx=0x7f11cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.166+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f11c0076970 con 0x7f11d013d240 2023-11-24T08:09:03.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.396+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11d013c130 con 0x7f11d013d240 2023-11-24T08:09:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.397+0000 7f11c5ffb640 1 -- 172.21.15.106:0/1433837176 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f11d013c130 con 0x7f11d013d240 2023-11-24T08:09:03.400 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:03.400 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:03.400 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.399+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f11b406ae40 msgr2=0x7f11b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.399+0000 7f11d604b640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f11b406ae40 0x7f11b406d300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009b60 tx=0x7f11cc008040 comp rx=0 tx=0).stop 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.399+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 msgr2=0x7f11d00cf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.399+0000 7f11d604b640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 0x7f11d00cf650 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f11c002beb0 tx=0x7f11c00046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 shutdown_connections 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f11b406ae40 0x7f11b406d300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f11d015e420 0x7f11d013c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f11d013d240 0x7f11d00cf650 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 --2- 172.21.15.106:0/1433837176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d00b35f0 0x7f11d01f65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 >> 172.21.15.106:0/1433837176 conn(0x7f11d00ccb00 msgr2=0x7f11d0159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 shutdown_connections 2023-11-24T08:09:03.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.400+0000 7f11d604b640 1 -- 172.21.15.106:0/1433837176 wait complete. 2023-11-24T08:09:03.419 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.610+0000 7fcc6d8da640 1 Processor -- start 2023-11-24T08:09:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.610+0000 7fcc6d8da640 1 -- start start 2023-11-24T08:09:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.612+0000 7fcc6d8da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 0x7fcc6815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.612+0000 7fcc6d8da640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc6815bf80 con 0x7fcc680b1de0 2023-11-24T08:09:03.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.612+0000 7fcc6d8da640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc680b1560 con 0x7fcc68144b30 2023-11-24T08:09:03.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.612+0000 7fcc6d8da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b1280 con 0x7fcc6815c410 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.612+0000 7fcc6d0d9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcc680b1de0 0x7fcc680b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:32906/0 (socket says 172.21.15.106:32906) 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc6d0d9640 1 -- 172.21.15.106:0/1137730855 learned_addr learned my addr 172.21.15.106:0/1137730855 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc67fff640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 0x7fcc6815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974513471 0 0) 0x7fcc6815bf80 con 0x7fcc680b1de0 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc58003520 con 0x7fcc680b1de0 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988551983 0 0) 0x7fcc680b1560 con 0x7fcc68144b30 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc6815bf80 con 0x7fcc68144b30 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 60879872 0 0) 0x7fcc58003520 con 0x7fcc680b1de0 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 msgr2=0x7fcc6815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 0x7fcc6815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 >> v1:172.21.15.156:6789/0 conn(0x7fcc68144b30 legacy=0x7fcc680b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc6815b930 con 0x7fcc680b1de0 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fcc600019f0 con 0x7fcc680b1de0 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.613+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 >> v1:172.21.15.106:6789/0 conn(0x7fcc680b1de0 legacy=0x7fcc680b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc677fe640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc58003b30 0x7fcc58003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc677fe640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 0x7fcc580047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc677fe640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc58004d00 0x7fcc58005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680b1280 con 0x7fcc58003b30 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5800ba30 con 0x7fcc58004d00 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5800bbf0 con 0x7fcc6815c410 2023-11-24T08:09:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc58003b30 0x7fcc58003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc67fff640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 0x7fcc580047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6d0d9640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc58004d00 0x7fcc58005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6c8d8640 1 -- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 msgr2=0x7fcc580047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc6815c410 0x7fcc580047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6c8d8640 1 -- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc58004d00 msgr2=0x7fcc58005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc58004d00 0x7fcc58005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.614+0000 7fcc6c8d8640 1 -- 172.21.15.106:0/1137730855 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58003520 con 0x7fcc58003b30 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc58003b30 0x7fcc58003f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0018d0 tx=0x7fcc580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27b3c17cf5f99128 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0052d0 con 0x7fcc58003b30 2023-11-24T08:09:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c005430 con 0x7fcc58003b30 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc677fe640 1 -- 172.21.15.106:0/1137730855 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c003810 con 0x7fcc58003b30 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc6d8da640 1 -- 172.21.15.106:0/1137730855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc58003b30 msgr2=0x7fcc58003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/1137730855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc58003b30 0x7fcc58003f30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c0018d0 tx=0x7fcc580057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc6d0d9640 1 -- 172.21.15.106:0/1137730855 reap_dead start 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc6d8da640 1 -- 172.21.15.106:0/1137730855 shutdown_connections 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.616+0000 7fcc6d8da640 1 -- 172.21.15.106:0/1137730855 >> 172.21.15.106:0/1137730855 conn(0x7fcc680cb830 msgr2=0x7fcc680cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.617+0000 7fcc6d8da640 1 -- 172.21.15.106:0/1137730855 shutdown_connections 2023-11-24T08:09:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.617+0000 7fcc6d8da640 1 -- 172.21.15.106:0/1137730855 wait complete. 2023-11-24T08:09:03.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.618+0000 7fcc6d8da640 1 Processor -- start 2023-11-24T08:09:03.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.618+0000 7fcc6d8da640 1 -- start start 2023-11-24T08:09:03.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.619+0000 7fcc6d8da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc58003b30 0x7fcc6812d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.619+0000 7fcc6d8da640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 0x7fcc6815dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.619+0000 7fcc6d8da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc6815e1f0 0x7fcc680c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.619+0000 7fcc6d8da640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc6815e670 con 0x7fcc680c8180 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.619+0000 7fcc6d8da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680ca480 con 0x7fcc6815e1f0 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.619+0000 7fcc6d8da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc680c4e10 con 0x7fcc58003b30 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 0x7fcc6815dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 0x7fcc6815dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34752/0 (socket says 172.21.15.106:34752) 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 -- 172.21.15.106:0/4191559713 learned_addr learned my addr 172.21.15.106:0/4191559713 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc58003b30 0x7fcc6812d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc6d0d9640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc6815e1f0 0x7fcc680c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 -- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc58003b30 msgr2=0x7fcc6812d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc58003b30 0x7fcc6812d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 -- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc6815e1f0 msgr2=0x7fcc680c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc6815e1f0 0x7fcc680c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 -- 172.21.15.106:0/4191559713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc680c9bd0 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc67fff640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 0x7fcc6815dcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc54011600 tx=0x7fcc54011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.620+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400f920 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.621+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5400fa80 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.621+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc680c9e70 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.621+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5400fdf0 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.621+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc680ca110 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.622+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc68067990 con 0x7fcc680c8180 2023-11-24T08:09:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.623+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcc54015460 con 0x7fcc680c8180 2023-11-24T08:09:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.624+0000 7fcc657fa640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc4c06ae40 0x7fcc4c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.624+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc54089f20 con 0x7fcc680c8180 2023-11-24T08:09:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.624+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc4c06ae40 0x7fcc4c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.625+0000 7fcc6c8d8640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc4c06ae40 0x7fcc4c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c034000 tx=0x7fcc5c031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:03.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.628+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc54054f40 con 0x7fcc680c8180 2023-11-24T08:09:03.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.817+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc6804e840 con 0x7fcc680c8180 2023-11-24T08:09:03.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.820+0000 7fcc657fa640 1 -- 172.21.15.106:0/4191559713 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7fcc54058560 con 0x7fcc680c8180 2023-11-24T08:09:03.823 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:03.823 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:03.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.822+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc4c06ae40 msgr2=0x7fcc4c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.822+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc4c06ae40 0x7fcc4c06d300 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c034000 tx=0x7fcc5c031000 comp rx=0 tx=0).stop 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.822+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 msgr2=0x7fcc6815dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.822+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 0x7fcc6815dcb0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fcc54011600 tx=0x7fcc54011ac0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 shutdown_connections 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc6815e1f0 0x7fcc680c3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc680c8180 0x7fcc6815dcb0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc4c06ae40 0x7fcc4c06d300 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 --2- 172.21.15.106:0/4191559713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc58003b30 0x7fcc6812d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 >> 172.21.15.106:0/4191559713 conn(0x7fcc680cb830 msgr2=0x7fcc6815c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 shutdown_connections 2023-11-24T08:09:03.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:03.823+0000 7fcc6d8da640 1 -- 172.21.15.106:0/4191559713 wait complete. 2023-11-24T08:09:03.842 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:04.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.041+0000 7fe8363a9640 1 Processor -- start 2023-11-24T08:09:04.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.042+0000 7fe8363a9640 1 -- start start 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe8363a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe8300b2250 0x7fe83015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe8363a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2aa0 0x7fe83015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe8363a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83015ab80 con 0x7fe8300b2630 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe8363a9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83015a990 con 0x7fe8300b2250 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe8363a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83015e680 con 0x7fe8300b2aa0 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe82ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe8300b2250 0x7fe83015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe827fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe8300b2630 0x7fe8300ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56730/0 (socket says 172.21.15.106:56730) 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe82ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe8300b2250 0x7fe83015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51478/0 (socket says 172.21.15.106:51478) 2023-11-24T08:09:04.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.043+0000 7fe827fff640 1 -- 172.21.15.106:0/3423468875 learned_addr learned my addr 172.21.15.106:0/3423468875 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:04.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe834946640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2aa0 0x7fe83015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe82ffff640 1 -- 172.21.15.106:0/3423468875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2aa0 msgr2=0x7fe83015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe82ffff640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2aa0 0x7fe83015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe82ffff640 1 -- 172.21.15.106:0/3423468875 >> v1:172.21.15.106:6789/0 conn(0x7fe8300b2630 legacy=0x7fe8300ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe82ffff640 1 -- 172.21.15.106:0/3423468875 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83015e960 con 0x7fe8300b2250 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe82f7fe640 1 -- 172.21.15.106:0/3423468875 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156077879 0 0) 0x7fe83015ab80 con 0x7fe8300b2630 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.044+0000 7fe834946640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2aa0 0x7fe83015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.045+0000 7fe82ffff640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2250 0x7fe83015df80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097a0 tx=0x7fe82002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f84cb415090de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.045+0000 7fe82f7fe640 1 -- 172.21.15.106:0/3423468875 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82002d040 con 0x7fe8300b2250 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.045+0000 7fe82f7fe640 1 -- 172.21.15.106:0/3423468875 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe820031810 con 0x7fe8300b2250 2023-11-24T08:09:04.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.045+0000 7fe82f7fe640 1 -- 172.21.15.106:0/3423468875 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe820031ba0 con 0x7fe8300b2250 2023-11-24T08:09:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 -- 172.21.15.106:0/3423468875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2250 msgr2=0x7fe83015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2250 0x7fe83015df80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe8200097a0 tx=0x7fe82002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 -- 172.21.15.106:0/3423468875 shutdown_connections 2023-11-24T08:09:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2aa0 0x7fe83015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 --2- 172.21.15.106:0/3423468875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2250 0x7fe83015df80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 -- 172.21.15.106:0/3423468875 >> 172.21.15.106:0/3423468875 conn(0x7fe83006a920 msgr2=0x7fe83006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 -- 172.21.15.106:0/3423468875 shutdown_connections 2023-11-24T08:09:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.046+0000 7fe8363a9640 1 -- 172.21.15.106:0/3423468875 wait complete. 2023-11-24T08:09:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.048+0000 7fe8363a9640 1 Processor -- start 2023-11-24T08:09:04.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.048+0000 7fe8363a9640 1 -- start start 2023-11-24T08:09:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe8363a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2250 0x7fe830157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe8363a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8300b2630 0x7fe8300c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe8363a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 0x7fe8300c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe8363a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8300cafb0 con 0x7fe8300b2aa0 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe8363a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe830158000 con 0x7fe8300b2630 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe8363a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe830071a30 con 0x7fe8300b2250 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 0x7fe8300c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 0x7fe8300c90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51494/0 (socket says 172.21.15.106:51494) 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 -- 172.21.15.106:0/4262675809 learned_addr learned my addr 172.21.15.106:0/4262675809 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe82ffff640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2250 0x7fe830157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe827fff640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8300b2630 0x7fe8300c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 -- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2250 msgr2=0x7fe830157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2250 0x7fe830157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 -- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8300b2630 msgr2=0x7fe8300c8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8300b2630 0x7fe8300c8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.049+0000 7fe834946640 1 -- 172.21.15.106:0/4262675809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe820009400 con 0x7fe8300b2aa0 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.050+0000 7fe834946640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 0x7fe8300c90b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe82800cca0 tx=0x7fe82800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.050+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800f680 con 0x7fe8300b2aa0 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.050+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe82800f800 con 0x7fe8300b2aa0 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.050+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe82800fbb0 con 0x7fe8300b2aa0 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.050+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8300cd7c0 con 0x7fe8300b2aa0 2023-11-24T08:09:04.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.050+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8300ae3b0 con 0x7fe8300b2aa0 2023-11-24T08:09:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.051+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe830067470 con 0x7fe8300b2aa0 2023-11-24T08:09:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.052+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe82800b040 con 0x7fe8300b2aa0 2023-11-24T08:09:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.053+0000 7fe82d7fa640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe81006ae70 0x7fe81006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.053+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe828089fb0 con 0x7fe8300b2aa0 2023-11-24T08:09:04.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.053+0000 7fe82ffff640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe81006ae70 0x7fe81006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.054+0000 7fe82ffff640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe81006ae70 0x7fe81006d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe82003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.057+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe828054fc0 con 0x7fe8300b2aa0 2023-11-24T08:09:04.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.273+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe830071b70 con 0x7fe8300b2aa0 2023-11-24T08:09:04.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.274+0000 7fe82d7fa640 1 -- 172.21.15.106:0/4262675809 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fe82801d090 con 0x7fe8300b2aa0 2023-11-24T08:09:04.277 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:04.277 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:04.277 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe81006ae70 msgr2=0x7fe81006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe81006ae70 0x7fe81006d330 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fe820036000 tx=0x7fe82003c040 comp rx=0 tx=0).stop 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 msgr2=0x7fe8300c90b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 0x7fe8300c90b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe82800cca0 tx=0x7fe82800a650 comp rx=0 tx=0).stop 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 shutdown_connections 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8300b2aa0 0x7fe8300c90b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8300b2630 0x7fe8300c8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe81006ae70 0x7fe81006d330 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 --2- 172.21.15.106:0/4262675809 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8300b2250 0x7fe830157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 >> 172.21.15.106:0/4262675809 conn(0x7fe83006a920 msgr2=0x7fe8300ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 shutdown_connections 2023-11-24T08:09:04.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.277+0000 7fe8363a9640 1 -- 172.21.15.106:0/4262675809 wait complete. 2023-11-24T08:09:04.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:08:47.586538+0000', 'modified': '2023-11-24T08:08:50.630789+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4276}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4276': {'gid': 4276, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.156:6835/4220970412', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 4220970412}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 4220970412}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:09:04.700 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:09:04.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-24T08:09:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.913+0000 7f94c8c3d640 1 Processor -- start 2023-11-24T08:09:04.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.913+0000 7f94c8c3d640 1 -- start start 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c8c3d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 0x7f94c4162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c8c3d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f94c4162540 0x7f94c40c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c8c3d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c4166510 con 0x7f94c40ccbd0 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c8c3d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94c41655f0 con 0x7f94c4162540 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c8c3d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40cc380 con 0x7f94c41660b0 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f94c40ccbd0 0x7f94c40ccfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56736/0 (socket says 172.21.15.106:56736) 2023-11-24T08:09:04.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.915+0000 7f94c259c640 1 -- 172.21.15.106:0/2125449025 learned_addr learned my addr 172.21.15.106:0/2125449025 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94bbfff640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 0x7f94c4162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/2125449025 >> v2:172.21.15.156:3300/0 conn(0x7f94c4162540 0x7f94c40c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379046751 0 0) 0x7f94b4001110 con 0x7f94c40ccbd0 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94b0003520 con 0x7f94c40ccbd0 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3521487971 0 0) 0x7f94b0003520 con 0x7f94c40ccbd0 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 msgr2=0x7f94c4162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 0x7f94c4162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 >> v2:172.21.15.156:3300/0 conn(0x7f94c4162540 msgr2=0x7f94c40c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 --2- 172.21.15.106:0/2125449025 >> v2:172.21.15.156:3300/0 conn(0x7f94c4162540 0x7f94c40c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4162b30 con 0x7f94c40ccbd0 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f94b4001800 con 0x7f94c40ccbd0 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 >> v1:172.21.15.106:6789/0 conn(0x7f94c40ccbd0 legacy=0x7f94c40ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.916+0000 7f94c1d9b640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b0003970 0x7f94b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c1d9b640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94b00047c0 0x7f94b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c1d9b640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 0x7f94b0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41655f0 con 0x7f94b0003970 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40cc380 con 0x7f94b00047c0 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94b000b980 con 0x7f94c41660b0 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c259c640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b0003970 0x7f94b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c259c640 1 -- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 msgr2=0x7f94b0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 0x7f94b0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94bbfff640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94b00047c0 0x7f94b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c259c640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94c41660b0 0x7f94b0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c259c640 1 -- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94b00047c0 msgr2=0x7f94b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c259c640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94b00047c0 0x7f94b0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.917+0000 7f94c259c640 1 -- 172.21.15.106:0/2125449025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0003520 con 0x7f94b0003970 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.918+0000 7f94c259c640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b0003970 0x7f94b0003d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c240 tx=0x7f94b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3522e0a63547a42 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.918+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4005ad0 con 0x7f94b0003970 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.918+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b4005c50 con 0x7f94b0003970 2023-11-24T08:09:04.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.918+0000 7f94c1d9b640 1 -- 172.21.15.106:0/2125449025 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b402c8b0 con 0x7f94b0003970 2023-11-24T08:09:04.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c8c3d640 1 -- 172.21.15.106:0/2125449025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b0003970 msgr2=0x7f94b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/2125449025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94b0003970 0x7f94b0003d90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c240 tx=0x7f94b00057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c2d9d640 1 -- 172.21.15.106:0/2125449025 reap_dead start 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c8c3d640 1 -- 172.21.15.106:0/2125449025 shutdown_connections 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c8c3d640 1 -- 172.21.15.106:0/2125449025 >> 172.21.15.106:0/2125449025 conn(0x7f94c40b0c50 msgr2=0x7f94c40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c8c3d640 1 -- 172.21.15.106:0/2125449025 shutdown_connections 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.919+0000 7f94c8c3d640 1 -- 172.21.15.106:0/2125449025 wait complete. 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.921+0000 7f94c8c3d640 1 Processor -- start 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.921+0000 7f94c8c3d640 1 -- start start 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94c8c3d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94b0003970 0x7f94c4158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94c8c3d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 0x7f94c41609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94c8c3d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c4160f10 0x7f94c40abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94c8c3d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4161390 con 0x7f94c40ad7a0 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94c8c3d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41586d0 con 0x7f94c4160f10 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94c8c3d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4070660 con 0x7f94b0003970 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 0x7f94c41609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 0x7f94c41609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51518/0 (socket says 172.21.15.106:51518) 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.922+0000 7f94bbfff640 1 -- 172.21.15.106:0/3791961613 learned_addr learned my addr 172.21.15.106:0/3791961613 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94c259c640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94b0003970 0x7f94c4158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94bbfff640 1 -- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94b0003970 msgr2=0x7f94c4158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94bbfff640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94b0003970 0x7f94c4158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c4160f10 0x7f94c40abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94bbfff640 1 -- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c4160f10 msgr2=0x7f94c40abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94bbfff640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c4160f10 0x7f94c40abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94bbfff640 1 -- 172.21.15.106:0/3791961613 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c4070940 con 0x7f94c40ad7a0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94c259c640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94b0003970 0x7f94c4158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94bbfff640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 0x7f94c41609d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94ac010860 tx=0x7f94ac010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94baffd640 1 -- 172.21.15.106:0/3791961613 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac009610 con 0x7f94c40ad7a0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94baffd640 1 -- 172.21.15.106:0/3791961613 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac00eb60 con 0x7f94c40ad7a0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94baffd640 1 -- 172.21.15.106:0/3791961613 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac01c400 con 0x7f94c40ad7a0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.923+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c4070c20 con 0x7f94c40ad7a0 2023-11-24T08:09:04.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.924+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c40711e0 con 0x7f94c40ad7a0 2023-11-24T08:09:04.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.926+0000 7f94baffd640 1 -- 172.21.15.106:0/3791961613 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f94ac01c560 con 0x7f94c40ad7a0 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.926+0000 7f94baffd640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f94a406ae70 0x7f94a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.927+0000 7f94baffd640 1 -- 172.21.15.106:0/3791961613 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94ac089be0 con 0x7f94c40ad7a0 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.927+0000 7f94c259c640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f94a406ae70 0x7f94a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.927+0000 7f94cb7d4740 1 Processor -- start 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.927+0000 7f94cb7d4740 1 -- start start 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.927+0000 7f94c259c640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f94a406ae70 0x7f94a406d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94b4034000 tx=0x7f94b4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cbea0 0x55773a6cc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6cb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cb5e0 0x55773a70f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a62a880 con 0x55773a6c6ba0 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a639ae0 con 0x55773a6cb5e0 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55773a6371d0 con 0x55773a6cbea0 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6cb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6cb0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51532/0 (socket says 172.21.15.106:51532) 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.928+0000 7f94bbfff640 1 -- 172.21.15.106:0/1381301805 learned_addr learned my addr 172.21.15.106:0/1381301805 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94bbfff640 1 -- 172.21.15.106:0/1381301805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cbea0 msgr2=0x55773a6cc280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94bbfff640 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cbea0 0x55773a6cc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94bbfff640 1 -- 172.21.15.106:0/1381301805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cb5e0 msgr2=0x55773a70f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94bbfff640 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cb5e0 0x55773a70f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94bbfff640 1 -- 172.21.15.106:0/1381301805 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773a5fa860 con 0x55773a6c6ba0 2023-11-24T08:09:04.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94bbfff640 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6cb0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f94ac087fb0 tx=0x7f94ac058960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbfae9905c1746d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94a3fff640 1 -- 172.21.15.106:0/1381301805 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac060bb0 con 0x55773a6c6ba0 2023-11-24T08:09:04.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94a3fff640 1 -- 172.21.15.106:0/1381301805 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac070130 con 0x55773a6c6ba0 2023-11-24T08:09:04.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.929+0000 7f94a3fff640 1 -- 172.21.15.106:0/1381301805 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac055070 con 0x55773a6c6ba0 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1381301805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 msgr2=0x55773a6cb0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6cb0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f94ac087fb0 tx=0x7f94ac058960 comp rx=0 tx=0).stop 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1381301805 shutdown_connections 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cb5e0 0x55773a70f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6cb0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1381301805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cbea0 0x55773a6cc280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1381301805 >> 172.21.15.106:0/1381301805 conn(0x55773a68dc70 msgr2=0x55773a711990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1381301805 shutdown_connections 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.930+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1381301805 wait complete. 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.931+0000 7f94cb7d4740 1 -- start start 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.931+0000 7f94cb7d4740 1 Processor -- start 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6c6ba0 0x55773a6f2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a70d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 0x55773a716e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a632ac0 con 0x55773a6cbea0 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a5cf2f0 con 0x55773a6c6ba0 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55773a5ecc70 con 0x55773a6cb5e0 2023-11-24T08:09:04.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94cb7d4740 10 client.0 _ll_register_callbacks cb 0x55773a5f13b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94c2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 0x55773a716e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94c2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 0x55773a716e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51548/0 (socket says 172.21.15.106:51548) 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.932+0000 7f94c2d9d640 1 -- 172.21.15.106:0/2370709031 learned_addr learned my addr 172.21.15.106:0/2370709031 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94bbfff640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a70d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c259c640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6c6ba0 0x55773a6f2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c2d9d640 1 -- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 msgr2=0x55773a70d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a70d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c2d9d640 1 -- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6c6ba0 msgr2=0x55773a6f2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6c6ba0 0x55773a6f2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c2d9d640 1 -- 172.21.15.106:0/2370709031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c40711e0 con 0x55773a6cbea0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c259c640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6c6ba0 0x55773a6f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 0x55773a716e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94c406e700 tx=0x7f94bc004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b778d1c9dc605ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94a17fa640 1 -- 172.21.15.106:0/2370709031 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94bc007b30 con 0x55773a6cbea0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94a17fa640 1 -- 172.21.15.106:0/2370709031 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94bc004830 con 0x55773a6cbea0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55773a62baa0 con 0x55773a6cbea0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94a17fa640 1 -- 172.21.15.106:0/2370709031 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94bc007c90 con 0x55773a6cbea0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94cb7d4740 10 client.4641 fetch_fsmap learned FSMap version 7 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.933+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55773a631140 con 0x55773a6cbea0 2023-11-24T08:09:04.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94a17fa640 1 -- 172.21.15.106:0/2370709031 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55773a62baa0 con 0x55773a6cbea0 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94a17fa640 1 -- 172.21.15.106:0/2370709031 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94bc016400 con 0x55773a6cbea0 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94cb7d4740 10 client.4641 fetch_fsmap finished waiting for FSMap version 7 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94cb7d4740 10 client.4641 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94cb7d4740 20 client.4641 populate_metadata read hostname 'smithi106' 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a719bb0 0x55773a70a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94cb7d4740 4 client.4641 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.934+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 --> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55773a62b7c0 con 0x55773a719bb0 2023-11-24T08:09:04.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.935+0000 7f94c259c640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a719bb0 0x55773a70a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.935+0000 7f94c259c640 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a719bb0 0x55773a70a7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ce6962b598110f7 server_cookie=b3d3a60e492a3157 in_seq=0 out_seq=0 2023-11-24T08:09:04.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.935+0000 7f94a17fa640 10 client.4641 ms_handle_connect on v2:172.21.15.156:6834/4220970412 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.937+0000 7f94a17fa640 1 -- 172.21.15.106:0/2370709031 <== mds.0 v2:172.21.15.156:6834/4220970412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55773a62b7c0 con 0x55773a719bb0 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.937+0000 7f94a17fa640 10 client.4641 handle_command_reply: tid=0 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.937+0000 7f94cb7d4740 1 client.4641 shutdown 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 msgr2=0x55773a716e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 0x55773a716e00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f94c406e700 tx=0x7f94bc004700 comp rx=0 tx=0).stop 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 shutdown_connections 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a719bb0 0x55773a70a7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cbea0 0x55773a716e00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a70d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/2370709031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6c6ba0 0x55773a6f2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94a17fa640 0 client.4641 ms_handle_reset on v2:172.21.15.156:6834/4220970412 2023-11-24T08:09:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 >> 172.21.15.106:0/2370709031 conn(0x55773a68dc70 msgr2=0x55773a6c5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 shutdown_connections 2023-11-24T08:09:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 1 -- 172.21.15.106:0/2370709031 wait complete. 2023-11-24T08:09:04.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.938+0000 7f94cb7d4740 20 client.4641 trim_cache size 0 max 16384 2023-11-24T08:09:04.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.954+0000 7f94cb7d4740 1 Processor -- start 2023-11-24T08:09:04.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.954+0000 7f94cb7d4740 1 -- start start 2023-11-24T08:09:04.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.955+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6c6ba0 0x55773a6fa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.955+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 0x55773a6fa810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.955+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a718990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.955+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a395f20 con 0x55773a6cb5e0 2023-11-24T08:09:04.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.955+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a717090 con 0x55773a6cbea0 2023-11-24T08:09:04.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.955+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55773a710960 con 0x55773a6c6ba0 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 0x55773a6fa810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 0x55773a6fa810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51556/0 (socket says 172.21.15.106:51556) 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 -- 172.21.15.106:0/1906474730 learned_addr learned my addr 172.21.15.106:0/1906474730 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94c259c640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6c6ba0 0x55773a6fa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a718990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 -- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6c6ba0 msgr2=0x55773a6fa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6c6ba0 0x55773a6fa2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 -- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 msgr2=0x55773a718990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a718990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 -- 172.21.15.106:0/1906474730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55773a5ceb80 con 0x55773a6cb5e0 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94c259c640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6c6ba0 0x55773a6fa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94c2d9d640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a718990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.956+0000 7f94bbfff640 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 0x55773a6fa810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0589a0 tx=0x7f94ac05b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d7dff31003d73e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94b8ff9640 1 -- 172.21.15.106:0/1906474730 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac060bb0 con 0x55773a6cb5e0 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94b8ff9640 1 -- 172.21.15.106:0/1906474730 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac056d00 con 0x55773a6cb5e0 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94b8ff9640 1 -- 172.21.15.106:0/1906474730 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ac059be0 con 0x55773a6cb5e0 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1906474730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 msgr2=0x55773a6fa810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 0x55773a6fa810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f94ac0589a0 tx=0x7f94ac05b000 comp rx=0 tx=0).stop 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1906474730 shutdown_connections 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a718990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6cb5e0 0x55773a6fa810 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/1906474730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6c6ba0 0x55773a6fa2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1906474730 >> 172.21.15.106:0/1906474730 conn(0x55773a68dc70 msgr2=0x55773a6c2310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1906474730 shutdown_connections 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.957+0000 7f94cb7d4740 1 -- 172.21.15.106:0/1906474730 wait complete. 2023-11-24T08:09:04.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.958+0000 7f94cb7d4740 1 -- start start 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.958+0000 7f94cb7d4740 1 Processor -- start 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a6c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a6c60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a6c6a30 con 0x55773a6c6ba0 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55773a77cc20 con 0x55773a6cbea0 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55773a6f0e80 con 0x55773a6cb5e0 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94cb7d4740 10 client.0 _ll_register_callbacks cb 0x7f94bc009b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.966 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6c6f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51566/0 (socket says 172.21.15.106:51566) 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 -- 172.21.15.106:0/723361472 learned_addr learned my addr 172.21.15.106:0/723361472 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 -- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 msgr2=0x55773a6c5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a6c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 -- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 msgr2=0x55773a6c60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a6c60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.959+0000 7f94c259c640 1 -- 172.21.15.106:0/723361472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55773a5fa860 con 0x55773a6c6ba0 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94c259c640 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6c6f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c360 tx=0x7f94b402e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3707cfe8e7aa23f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94a17fa640 1 -- 172.21.15.106:0/723361472 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b402de90 con 0x55773a6c6ba0 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94a17fa640 1 -- 172.21.15.106:0/723361472 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f94b404e3c0 con 0x55773a6c6ba0 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94ac070130 con 0x55773a6c6ba0 2023-11-24T08:09:04.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94a17fa640 1 -- 172.21.15.106:0/723361472 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94b404e580 con 0x55773a6c6ba0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94cb7d4740 10 client.4647 fetch_fsmap learned FSMap version 7 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.960+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x55773a5f4170 con 0x55773a6c6ba0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94a17fa640 1 -- 172.21.15.106:0/723361472 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ac070130 con 0x55773a6c6ba0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94a17fa640 1 -- 172.21.15.106:0/723361472 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 7) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f94b404d040 con 0x55773a6c6ba0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94cb7d4740 10 client.4647 fetch_fsmap finished waiting for FSMap version 7 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94cb7d4740 10 client.4647 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94cb7d4740 20 client.4647 populate_metadata read hostname 'smithi106' 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a72b190 0x55773a72b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94cb7d4740 4 client.4647 mds_command: new command op to 4276 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 --> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55773a3336c0 con 0x55773a72b190 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.961+0000 7f94bbfff640 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a72b190 0x55773a72b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:04.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.962+0000 7f94a17fa640 10 client.4647 ms_handle_connect on v2:172.21.15.156:6834/4220970412 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.962+0000 7f94bbfff640 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a72b190 0x55773a72b570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdccc6012687d768 server_cookie=51167ba771c7ceb1 in_seq=0 out_seq=0 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.962+0000 7f94a17fa640 1 -- 172.21.15.106:0/723361472 <== mds.0 v2:172.21.15.156:6834/4220970412 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55773a3336c0 con 0x55773a72b190 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.962+0000 7f94a17fa640 10 client.4647 handle_command_reply: tid=0 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.962+0000 7f94cb7d4740 1 client.4647 shutdown 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 msgr2=0x55773a6c6f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6c6f80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f94b400c360 tx=0x7f94b402e070 comp rx=0 tx=0).stop 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 shutdown_connections 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:6834/4220970412,v1:172.21.15.156:6835/4220970412] conn(0x55773a72b190 0x55773a72b570 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55773a6cbea0 0x55773a6c60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55773a6cb5e0 0x55773a6c5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94a17fa640 0 client.4647 ms_handle_reset on v2:172.21.15.156:6834/4220970412 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 --2- 172.21.15.106:0/723361472 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55773a6c6ba0 0x55773a6c6f80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 >> 172.21.15.106:0/723361472 conn(0x55773a68dc70 msgr2=0x55773a6c22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 shutdown_connections 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 1 -- 172.21.15.106:0/723361472 wait complete. 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94cb7d4740 20 client.4647 trim_cache size 0 max 16384 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d1a897c2-15ce-4272-ad66-4b3723970cb1", 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 4276, 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2023-11-24T08:09:04.970 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 16.846479884000001, 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 7, 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 22, 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 22, 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 24.420434988 2023-11-24T08:09:04.971 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f94a406ae70 msgr2=0x7f94a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f94a406ae70 0x7f94a406d330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f94b4034000 tx=0x7f94b4009030 comp rx=0 tx=0).stop 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 msgr2=0x7f94c41609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.963+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 0x7f94c41609d0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f94ac010860 tx=0x7f94ac010d20 comp rx=0 tx=0).stop 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 shutdown_connections 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94c4160f10 0x7f94c40abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94c40ad7a0 0x7f94c41609d0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f94a406ae70 0x7f94a406d330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 --2- 172.21.15.106:0/3791961613 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94b0003970 0x7f94c4158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:04.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 >> 172.21.15.106:0/3791961613 conn(0x7f94c40b0c50 msgr2=0x7f94c406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:04.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 shutdown_connections 2023-11-24T08:09:04.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:04.964+0000 7f94c8c3d640 1 -- 172.21.15.106:0/3791961613 wait complete. 2023-11-24T08:09:04.983 INFO:teuthology.run_tasks:Running task kclient... 2023-11-24T08:09:05.005 INFO:tasks.kclient:Mounting kernel clients... 2023-11-24T08:09:05.005 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-24T08:09:05.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.005 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:09:05.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.061 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2023-11-24T08:09:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2023-11-24T08:09:05.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:05.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.168 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:09:05.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.222 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2023-11-24T08:09:05.311 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2023-11-24T08:09:05.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:05.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.314 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:09:05.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.376 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-24T08:09:05.476 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-24T08:09:05.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:05.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.480 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:09:05.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:05.537 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-24T08:09:05.634 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-24T08:09:05.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:09:05.637 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-11-24T08:09:05.637 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:09:05.637 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:09:05.638 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:09:05.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.855+0000 7f46bd5d5640 1 Processor -- start 2023-11-24T08:09:05.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.855+0000 7f46bd5d5640 1 -- start start 2023-11-24T08:09:05.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.856+0000 7f46bd5d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46b80af670 0x7f46b806d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.856+0000 7f46bd5d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b8072bb0 con 0x7f46b80721e0 2023-11-24T08:09:05.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.856+0000 7f46bd5d5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b806ef10 con 0x7f46b806a500 2023-11-24T08:09:05.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.856+0000 7f46bd5d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b8071dc0 con 0x7f46b80af670 2023-11-24T08:09:05.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46b80af670 0x7f46b806d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b67fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f46b80721e0 0x7f46b80acf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56748/0 (socket says 172.21.15.106:56748) 2023-11-24T08:09:05.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b67fc640 1 -- 172.21.15.106:0/3818913666 learned_addr learned my addr 172.21.15.106:0/3818913666 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b77fe640 1 -- 172.21.15.106:0/3818913666 >> v1:172.21.15.156:6789/0 conn(0x7f46b806a500 legacy=0x7f46b806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b77fe640 1 -- 172.21.15.106:0/3818913666 >> v1:172.21.15.106:6789/0 conn(0x7f46b80721e0 legacy=0x7f46b80acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b77fe640 1 -- 172.21.15.106:0/3818913666 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b806f1b0 con 0x7f46b80af670 2023-11-24T08:09:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.857+0000 7f46b5ffb640 1 -- 172.21.15.106:0/3818913666 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991560976 0 0) 0x7f46b8072bb0 con 0x7f46b80721e0 2023-11-24T08:09:05.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.858+0000 7f46b77fe640 1 --2- 172.21.15.106:0/3818913666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b80af670 0x7f46b806d6c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009d90 tx=0x7f46b002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970accd22ff152d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46b5ffb640 1 -- 172.21.15.106:0/3818913666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b002cc60 con 0x7f46b80af670 2023-11-24T08:09:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46b5ffb640 1 -- 172.21.15.106:0/3818913666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b002cdc0 con 0x7f46b80af670 2023-11-24T08:09:05.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46b5ffb640 1 -- 172.21.15.106:0/3818913666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b0034660 con 0x7f46b80af670 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 -- 172.21.15.106:0/3818913666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b80af670 msgr2=0x7f46b806d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/3818913666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b80af670 0x7f46b806d6c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f46b0009d90 tx=0x7f46b002beb0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 -- 172.21.15.106:0/3818913666 shutdown_connections 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/3818913666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b80af670 0x7f46b806d6c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 -- 172.21.15.106:0/3818913666 >> 172.21.15.106:0/3818913666 conn(0x7f46b80d01b0 msgr2=0x7f46b80d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 -- 172.21.15.106:0/3818913666 shutdown_connections 2023-11-24T08:09:05.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.859+0000 7f46bd5d5640 1 -- 172.21.15.106:0/3818913666 wait complete. 2023-11-24T08:09:05.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.861+0000 7f46bd5d5640 1 Processor -- start 2023-11-24T08:09:05.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.861+0000 7f46bd5d5640 1 -- start start 2023-11-24T08:09:05.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46bd5d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 0x7f46b80c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46bd5d5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b80721e0 0x7f46b80b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46bd5d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b80af670 0x7f46b80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46bd5d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80b3e30 con 0x7f46b806a500 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46bd5d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80b0f30 con 0x7f46b80af670 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46bd5d5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b80b4060 con 0x7f46b80721e0 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46b6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 0x7f46b80c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46b6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 0x7f46b80c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51582/0 (socket says 172.21.15.106:51582) 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.862+0000 7f46b6ffd640 1 -- 172.21.15.106:0/1120417091 learned_addr learned my addr 172.21.15.106:0/1120417091 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46b6ffd640 1 -- 172.21.15.106:0/1120417091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b80721e0 msgr2=0x7f46b80b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46b6ffd640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b80721e0 0x7f46b80b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46b6ffd640 1 -- 172.21.15.106:0/1120417091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b80af670 msgr2=0x7f46b80b4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:05.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46b6ffd640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b80af670 0x7f46b80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46b6ffd640 1 -- 172.21.15.106:0/1120417091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b0009a70 con 0x7f46b806a500 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46b6ffd640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 0x7f46b80c3ce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c400 tx=0x7f46a800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a80052d0 con 0x7f46b806a500 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46a8005430 con 0x7f46b806a500 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46a8003830 con 0x7f46b806a500 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.863+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b80b4340 con 0x7f46b806a500 2023-11-24T08:09:05.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.864+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b80b46d0 con 0x7f46b806a500 2023-11-24T08:09:05.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.865+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b80b2620 con 0x7f46b806a500 2023-11-24T08:09:05.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.866+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f46a8003990 con 0x7f46b806a500 2023-11-24T08:09:05.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.866+0000 7f469f7fe640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46a406ae40 0x7f46a406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:05.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.867+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46a8088b90 con 0x7f46b806a500 2023-11-24T08:09:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.867+0000 7f46b67fc640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46a406ae40 0x7f46a406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:05.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.867+0000 7f46b67fc640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46a406ae40 0x7f46a406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46ac007fd0 tx=0x7f46ac007900 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:05.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:05.871+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46a8053c40 con 0x7f46b806a500 2023-11-24T08:09:06.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.101+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f46b8067b80 con 0x7f46b806a500 2023-11-24T08:09:06.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.102+0000 7f469f7fe640 1 -- 172.21.15.106:0/1120417091 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f46a8017070 con 0x7f46b806a500 2023-11-24T08:09:06.104 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46a406ae40 msgr2=0x7f46a406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46a406ae40 0x7f46a406d300 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f46ac007fd0 tx=0x7f46ac007900 comp rx=0 tx=0).stop 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 msgr2=0x7f46b80c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 0x7f46b80c3ce0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c400 tx=0x7f46a800c8c0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 shutdown_connections 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b80af670 0x7f46b80b4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46a406ae40 0x7f46a406d300 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b80721e0 0x7f46b80b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 --2- 172.21.15.106:0/1120417091 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b806a500 0x7f46b80c3ce0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.104+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 >> 172.21.15.106:0/1120417091 conn(0x7f46b80d01b0 msgr2=0x7f46b80acf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.105+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 shutdown_connections 2023-11-24T08:09:06.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:06.105+0000 7f46bd5d5640 1 -- 172.21.15.106:0/1120417091 wait complete. 2023-11-24T08:09:06.126 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:09:06.126 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:09:06.126 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:09:06.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:09:06.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:09:06.126 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:09:06.127 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:09:06.127 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:09:06.127 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:09:06.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:06.127 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:09:06.179 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42370sec preferred_lft 42370sec 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:09:06.180 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:06.181 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T08:09:06.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:06.181 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:06.181 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:06.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add name ceph-brx type bridge 2023-11-24T08:09:06.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr flush dev ceph-brx 2023-11-24T08:09:06.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set ceph-brx up 2023-11-24T08:09:06.182 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T08:09:06.182 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:06.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:06.612 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:09:06.686 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:09:06.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:06.690 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:09:06.743 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:09:06.744 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:09:06.744 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:09:06.744 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:09:06.744 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:09:06.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:06.744 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:06.745 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:06.745 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:09:06.745 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:09:06.745 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:09:06.745 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:07.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:07.492 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:09:07.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:07.549 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2023-11-24T08:09:07.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:07.601 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:07.601 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:07.601 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:09:07.601 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-24T08:09:07.602 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:07.740 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-24T08:09:07.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-24T08:09:07.741 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:08.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:08.020 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:08.020 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:08.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2023-11-24T08:09:08.021 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T08:09:08.021 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:08.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:08.191 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:09:08.239 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:09:08.239 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:08.239 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:09:08.283 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:09:08.284 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:08.284 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:08.482 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:08.483 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:08.943 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:08.943 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:08.943 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:09:08.943 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:09:08.943 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:09:08.948 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-24T08:09:08.948 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:09:08.948 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:09:08.948 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:09:08.948 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:09:08.954 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:09:08.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:08.954 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:09:09.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:09.046 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:09.122 INFO:teuthology.orchestra.run.smithi106.stdout:client4659 2023-11-24T08:09:09.122 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:09.123 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:09.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:09.181 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:09.262 INFO:teuthology.orchestra.run.smithi106.stdout:client4659 2023-11-24T08:09:09.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:09.262 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:09.346 INFO:teuthology.orchestra.run.smithi106.stdout:client4659 2023-11-24T08:09:09.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:09.347 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1a897c2-15ce-4272-ad66-4b3723970cb1.client4659/status') 2023-11-24T08:09:09.459 INFO:teuthology.orchestra.run.smithi106.stdout:instance: client.4659 (3)192.168.0.1:0/3149688706 2023-11-24T08:09:09.460 INFO:teuthology.orchestra.run.smithi106.stdout:blocklisted: false 2023-11-24T08:09:09.460 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2023-11-24T08:09:09.460 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2023-11-24T08:09:09.460 INFO:teuthology.orchestra.run.smithi106.stderr:69 bytes copied, 6.9446e-05 s, 994 kB/s 2023-11-24T08:09:09.464 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-11-24T08:09:09.464 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:09:09.464 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T08:09:09.465 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:09:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.672+0000 7fa4ea74c640 1 Processor -- start 2023-11-24T08:09:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.672+0000 7fa4ea74c640 1 -- start start 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4ea74c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa4e4159ce0 0x7fa4e406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4ea74c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e40b21d0 0x7fa4e40cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4ea74c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa4e40cbf80 0x7fa4e415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4ea74c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40cc410 con 0x7fa4e40cbf80 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4ea74c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40b3a40 con 0x7fa4e4159ce0 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4ea74c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40cdfa0 con 0x7fa4e40b21d0 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.673+0000 7fa4e8ce9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa4e40cbf80 0x7fa4e415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa4e40cbf80 0x7fa4e415d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51586/0 (socket says 172.21.15.106:51586) 2023-11-24T08:09:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 -- 172.21.15.106:0/3395589538 learned_addr learned my addr 172.21.15.106:0/3395589538 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 -- 172.21.15.106:0/3395589538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e40b21d0 msgr2=0x7fa4e40cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 --2- 172.21.15.106:0/3395589538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e40b21d0 0x7fa4e40cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 -- 172.21.15.106:0/3395589538 >> v2:172.21.15.156:3300/0 conn(0x7fa4e4159ce0 msgr2=0x7fa4e406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 --2- 172.21.15.106:0/3395589538 >> v2:172.21.15.156:3300/0 conn(0x7fa4e4159ce0 0x7fa4e406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.674+0000 7fa4e8ce9640 1 -- 172.21.15.106:0/3395589538 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e40cd9e0 con 0x7fa4e40cbf80 2023-11-24T08:09:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.675+0000 7fa4e8ce9640 1 --2- 172.21.15.106:0/3395589538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40cbf80 0x7fa4e415d760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa4e406e770 tx=0x7fa4dc02cfe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febc74f09254f819 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.675+0000 7fa4e37fe640 1 -- 172.21.15.106:0/3395589538 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc02db80 con 0x7fa4e40cbf80 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.675+0000 7fa4e37fe640 1 -- 172.21.15.106:0/3395589538 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4dc02dce0 con 0x7fa4e40cbf80 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.675+0000 7fa4e37fe640 1 -- 172.21.15.106:0/3395589538 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4dc035610 con 0x7fa4e40cbf80 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3395589538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40cbf80 msgr2=0x7fa4e415d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3395589538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40cbf80 0x7fa4e415d760 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa4e406e770 tx=0x7fa4dc02cfe0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3395589538 shutdown_connections 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3395589538 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40cbf80 0x7fa4e415d760 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3395589538 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e40b21d0 0x7fa4e40cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3395589538 >> v2:172.21.15.156:3300/0 conn(0x7fa4e4159ce0 0x7fa4e406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3395589538 >> 172.21.15.106:0/3395589538 conn(0x7fa4e406a920 msgr2=0x7fa4e406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3395589538 shutdown_connections 2023-11-24T08:09:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.676+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3395589538 wait complete. 2023-11-24T08:09:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.678+0000 7fa4ea74c640 1 Processor -- start 2023-11-24T08:09:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.678+0000 7fa4ea74c640 1 -- start start 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4ea74c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 0x7fa4e40987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4ea74c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e40cbf80 0x7fa4e41e45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4ea74c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e4159ce0 0x7fa4e40ad0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4ea74c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41e4f20 con 0x7fa4e40b21d0 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4ea74c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41e5200 con 0x7fa4e40cbf80 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4ea74c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e41e54e0 con 0x7fa4e4159ce0 2023-11-24T08:09:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 0x7fa4e40987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4e3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 0x7fa4e40987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51602/0 (socket says 172.21.15.106:51602) 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.679+0000 7fa4e3fff640 1 -- 172.21.15.106:0/3901018597 learned_addr learned my addr 172.21.15.106:0/3901018597 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e8ce9640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e4159ce0 0x7fa4e40ad0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4dbfff640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e40cbf80 0x7fa4e41e45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e3fff640 1 -- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e4159ce0 msgr2=0x7fa4e40ad0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e3fff640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e4159ce0 0x7fa4e40ad0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e3fff640 1 -- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e40cbf80 msgr2=0x7fa4e41e45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e3fff640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e40cbf80 0x7fa4e41e45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e3fff640 1 -- 172.21.15.106:0/3901018597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4dc009b30 con 0x7fa4e40b21d0 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e8ce9640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e4159ce0 0x7fa4e40ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e3fff640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 0x7fa4e40987f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400b4a0 tx=0x7fa4d400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:09.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4010610 con 0x7fa4e40b21d0 2023-11-24T08:09:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4010770 con 0x7fa4e40b21d0 2023-11-24T08:09:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4010b20 con 0x7fa4e40b21d0 2023-11-24T08:09:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.680+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e41e57c0 con 0x7fa4e40b21d0 2023-11-24T08:09:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.681+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e4098d30 con 0x7fa4e40b21d0 2023-11-24T08:09:09.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.681+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e41e5ce0 con 0x7fa4e40b21d0 2023-11-24T08:09:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.682+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa4d4010c80 con 0x7fa4e40b21d0 2023-11-24T08:09:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.683+0000 7fa4e17fa640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa4c406ae40 0x7fa4c406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.683+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa4d4089190 con 0x7fa4e40b21d0 2023-11-24T08:09:09.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.684+0000 7fa4dbfff640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa4c406ae40 0x7fa4c406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:09.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.684+0000 7fa4dbfff640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa4c406ae40 0x7fa4c406d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40ae090 tx=0x7fa4cc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:09.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.688+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa4e41e5ce0 con 0x7fa4e40b21d0 2023-11-24T08:09:09.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.915+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa4e41e5ce0 con 0x7fa4e40b21d0 2023-11-24T08:09:09.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.916+0000 7fa4e17fa640 1 -- 172.21.15.106:0/3901018597 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa4e41e5ce0 con 0x7fa4e40b21d0 2023-11-24T08:09:09.918 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:09:09.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa4c406ae40 msgr2=0x7fa4c406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa4c406ae40 0x7fa4c406d300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa4e40ae090 tx=0x7fa4cc008040 comp rx=0 tx=0).stop 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 msgr2=0x7fa4e40987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 0x7fa4e40987f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400b4a0 tx=0x7fa4d400b960 comp rx=0 tx=0).stop 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 shutdown_connections 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa4c406ae40 0x7fa4c406d300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa4e4159ce0 0x7fa4e40ad0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa4e40cbf80 0x7fa4e41e45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 --2- 172.21.15.106:0/3901018597 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa4e40b21d0 0x7fa4e40987f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 >> 172.21.15.106:0/3901018597 conn(0x7fa4e406a920 msgr2=0x7fa4e40b0530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.919+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 shutdown_connections 2023-11-24T08:09:09.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:09.920+0000 7fa4ea74c640 1 -- 172.21.15.106:0/3901018597 wait complete. 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:09:09.939 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-24T08:09:09.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:09.939 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:09.988 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:09.989 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:09.989 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:09:09.989 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:09.989 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:09.989 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:09:09.989 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42366sec preferred_lft 42366sec 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:09:09.990 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a407:f7ff:fe84:63d0/64 scope link 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9079:d8ff:febb:c08c/64 scope link 2023-11-24T08:09:09.991 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:09.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:09.992 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:09:10.080 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:09:10.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.084 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:09:10.139 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:09:10.140 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:09:10.140 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:09:10.140 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:09:10.140 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:09:10.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.141 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:10.141 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:10.141 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:09:10.142 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:09:10.142 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:09:10.142 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:10.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.328 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:09:10.383 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:09:10.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.384 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list-id 2023-11-24T08:09:10.436 INFO:teuthology.orchestra.run.smithi106.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-24T08:09:10.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.437 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:10.437 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:10.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:09:10.437 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-24T08:09:10.437 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:10.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.574 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T08:09:10.669 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 42:bc:66:0b:2c:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::40bc:66ff:fe0b:2c44/64 scope link 2023-11-24T08:09:10.670 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.674 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-24T08:09:10.772 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 42:bc:66:0b:2c:44 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.773 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::40bc:66ff:fe0b:2c44/64 scope link 2023-11-24T08:09:10.774 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:10.777 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-24T08:09:10.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-24T08:09:10.778 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:11.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.053 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:09:11.053 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:09:11.054 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.1 up 2023-11-24T08:09:11.054 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T08:09:11.054 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:09:11.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:11.223 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-24T08:09:11.274 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-24T08:09:11.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:11.275 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-24T08:09:11.323 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-11-24T08:09:11.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.323 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:11.511 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:11.511 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:11.547 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:11.547 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:11.547 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:09:11.547 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 2@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:09:11.547 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:09:11.550 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2023-11-24T08:09:11.550 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:09:11.550 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:09:11.550 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:09:11.551 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:09:11.554 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:09:11.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.554 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-24T08:09:11.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.644 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:11.704 INFO:teuthology.orchestra.run.smithi106.stdout:client4668 2023-11-24T08:09:11.705 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:11.705 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:11.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.758 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:11.815 INFO:teuthology.orchestra.run.smithi106.stdout:client4668 2023-11-24T08:09:11.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.815 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:11.873 INFO:teuthology.orchestra.run.smithi106.stdout:client4668 2023-11-24T08:09:11.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:11.874 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1a897c2-15ce-4272-ad66-4b3723970cb1.client4668/status') 2023-11-24T08:09:11.965 INFO:teuthology.orchestra.run.smithi106.stdout:instance: client.4668 (3)192.168.0.2:0/1540119522 2023-11-24T08:09:11.965 INFO:teuthology.orchestra.run.smithi106.stdout:blocklisted: false 2023-11-24T08:09:11.966 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records in 2023-11-24T08:09:11.966 INFO:teuthology.orchestra.run.smithi106.stderr:0+1 records out 2023-11-24T08:09:11.966 INFO:teuthology.orchestra.run.smithi106.stderr:69 bytes copied, 6.4821e-05 s, 1.1 MB/s 2023-11-24T08:09:11.969 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-11-24T08:09:11.969 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:09:11.969 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:09:11.969 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:09:12.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.180+0000 7fc6b8fba640 1 Processor -- start 2023-11-24T08:09:12.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.180+0000 7fc6b8fba640 1 -- start start 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b8fba640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc6b4161080 0x7fc6b40b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b8fba640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 0x7fc6b40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b8fba640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b4161510 con 0x7fc6b4143870 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b8fba640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b40b1460 con 0x7fc6b4161080 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b8fba640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4070460 con 0x7fc6b40b0fb0 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc6b4143870 0x7fc6b4146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:56764/0 (socket says 172.21.15.106:56764) 2023-11-24T08:09:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.182+0000 7fc6b37fe640 1 -- 172.21.15.106:0/309561221 learned_addr learned my addr 172.21.15.106:0/309561221 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b3fff640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 0x7fc6b40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6abfff640 1 --2- 172.21.15.106:0/309561221 >> v2:172.21.15.156:3300/0 conn(0x7fc6b4161080 0x7fc6b40b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1512012366 0 0) 0x7fc6b4161510 con 0x7fc6b4143870 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6a0003520 con 0x7fc6b4143870 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1974025040 0 0) 0x7fc6a0003520 con 0x7fc6b4143870 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 msgr2=0x7fc6b40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 0x7fc6b40cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 >> v2:172.21.15.156:3300/0 conn(0x7fc6b4161080 msgr2=0x7fc6b40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 --2- 172.21.15.106:0/309561221 >> v2:172.21.15.156:3300/0 conn(0x7fc6b4161080 0x7fc6b40b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b41617f0 con 0x7fc6b4143870 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc6a4001670 con 0x7fc6b4143870 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 >> v1:172.21.15.106:6789/0 conn(0x7fc6b4143870 legacy=0x7fc6b4146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:12.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.183+0000 7fc6b2ffd640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b2ffd640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6a00047c0 0x7fc6a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b2ffd640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 0x7fc6a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40b1460 con 0x7fc6a0003970 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4070460 con 0x7fc6a00047c0 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a000b980 con 0x7fc6b40b0fb0 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6abfff640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6a00047c0 0x7fc6a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b3fff640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 0x7fc6a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b37fe640 1 -- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 msgr2=0x7fc6a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b0fb0 0x7fc6a0005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b37fe640 1 -- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6a00047c0 msgr2=0x7fc6a0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:12.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6a00047c0 0x7fc6a0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6b37fe640 1 -- 172.21.15.106:0/309561221 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a0003520 con 0x7fc6a0003970 2023-11-24T08:09:12.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.184+0000 7fc6abfff640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6a00047c0 0x7fc6a0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.185+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6a0003d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c240 tx=0x7fc6a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45313f75b3b6c609 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a402cb00 con 0x7fc6a0003970 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a402cc60 con 0x7fc6a0003970 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b2ffd640 1 -- 172.21.15.106:0/309561221 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4005af0 con 0x7fc6a0003970 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b8fba640 1 -- 172.21.15.106:0/309561221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 msgr2=0x7fc6a0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/309561221 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6a0003d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c240 tx=0x7fc6a00057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b3fff640 1 -- 172.21.15.106:0/309561221 reap_dead start 2023-11-24T08:09:12.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b8fba640 1 -- 172.21.15.106:0/309561221 shutdown_connections 2023-11-24T08:09:12.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b8fba640 1 -- 172.21.15.106:0/309561221 >> 172.21.15.106:0/309561221 conn(0x7fc6b40cb830 msgr2=0x7fc6b40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:12.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b8fba640 1 -- 172.21.15.106:0/309561221 shutdown_connections 2023-11-24T08:09:12.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.186+0000 7fc6b8fba640 1 -- 172.21.15.106:0/309561221 wait complete. 2023-11-24T08:09:12.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.188+0000 7fc6b8fba640 1 Processor -- start 2023-11-24T08:09:12.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.188+0000 7fc6b8fba640 1 -- start start 2023-11-24T08:09:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.189+0000 7fc6b8fba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6b415db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.189+0000 7fc6b8fba640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b4830 0x7fc6b4145180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.189+0000 7fc6b8fba640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6b413f770 0x7fc6b413fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.189+0000 7fc6b8fba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b415f500 con 0x7fc6a0003970 2023-11-24T08:09:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.189+0000 7fc6b8fba640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b4140160 con 0x7fc6b413f770 2023-11-24T08:09:12.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.189+0000 7fc6b8fba640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b415e1a0 con 0x7fc6b40b4830 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6b415db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6b415db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51622/0 (socket says 172.21.15.106:51622) 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 -- 172.21.15.106:0/2861189080 learned_addr learned my addr 172.21.15.106:0/2861189080 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6abfff640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b4830 0x7fc6b4145180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b3fff640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6b413f770 0x7fc6b413fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 -- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b4830 msgr2=0x7fc6b4145180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b4830 0x7fc6b4145180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 -- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6b413f770 msgr2=0x7fc6b413fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6b413f770 0x7fc6b413fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b37fe640 1 -- 172.21.15.106:0/2861189080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b415e480 con 0x7fc6a0003970 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6b3fff640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6b413f770 0x7fc6b413fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:12.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.190+0000 7fc6abfff640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b4830 0x7fc6b4145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.191+0000 7fc6b37fe640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6b415db10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c240 tx=0x7fc6a4001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.191+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4039030 con 0x7fc6a0003970 2023-11-24T08:09:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.191+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4004bf0 con 0x7fc6a0003970 2023-11-24T08:09:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.191+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a403c040 con 0x7fc6a0003970 2023-11-24T08:09:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.191+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b415e760 con 0x7fc6a0003970 2023-11-24T08:09:12.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.191+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b40b4cb0 con 0x7fc6a0003970 2023-11-24T08:09:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.192+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4067990 con 0x7fc6a0003970 2023-11-24T08:09:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.193+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc6a4032070 con 0x7fc6a0003970 2023-11-24T08:09:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.194+0000 7fc6b0ff9640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc69406ae70 0x7fc69406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.194+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6a40aaa40 con 0x7fc6a0003970 2023-11-24T08:09:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.194+0000 7fc6abfff640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc69406ae70 0x7fc69406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:12.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.195+0000 7fc6abfff640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc69406ae70 0x7fc69406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40b1b80 tx=0x7fc69c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:12.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.198+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc6a40759c0 con 0x7fc6a0003970 2023-11-24T08:09:12.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.433+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc6b404e840 con 0x7fc6a0003970 2023-11-24T08:09:12.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.433+0000 7fc6b0ff9640 1 -- 172.21.15.106:0/2861189080 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc6a4078fe0 con 0x7fc6a0003970 2023-11-24T08:09:12.435 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc69406ae70 msgr2=0x7fc69406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc69406ae70 0x7fc69406d330 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc6b40b1b80 tx=0x7fc69c008040 comp rx=0 tx=0).stop 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 msgr2=0x7fc6b415db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6b415db10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c240 tx=0x7fc6a4001a00 comp rx=0 tx=0).stop 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 shutdown_connections 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc6b413f770 0x7fc6b413fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc69406ae70 0x7fc69406d330 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc6b40b4830 0x7fc6b4145180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 --2- 172.21.15.106:0/2861189080 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc6a0003970 0x7fc6b415db10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:12.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 >> 172.21.15.106:0/2861189080 conn(0x7fc6b40cb830 msgr2=0x7fc6b40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:12.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 shutdown_connections 2023-11-24T08:09:12.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:12.436+0000 7fc6b8fba640 1 -- 172.21.15.106:0/2861189080 wait complete. 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:09:12.455 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:09:12.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:12.456 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:09:12.517 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:09:12.518 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42379sec preferred_lft 42379sec 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:09:12.519 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:12.520 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-24T08:09:12.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:12.520 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:12.520 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:12.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add name ceph-brx type bridge 2023-11-24T08:09:12.520 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr flush dev ceph-brx 2023-11-24T08:09:12.521 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set ceph-brx up 2023-11-24T08:09:12.521 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-24T08:09:12.521 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:12.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:12.970 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:09:13.051 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:09:13.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:13.054 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:09:13.109 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:09:13.110 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:09:13.110 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:09:13.110 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:09:13.110 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:09:13.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:13.111 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:13.111 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:13.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:09:13.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:09:13.111 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:09:13.112 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:13.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:13.885 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:09:13.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:13.944 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-11-24T08:09:14.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:14.002 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:14.002 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:14.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:09:14.002 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-24T08:09:14.002 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:14.148 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-24T08:09:14.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-24T08:09:14.149 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:14.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:14.434 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:14.434 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:14.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-24T08:09:14.434 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.0 master ceph-brx 2023-11-24T08:09:14.434 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:14.586 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:14.586 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:09:14.632 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:09:14.632 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:14.632 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:09:14.729 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:09:14.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:14.730 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:14.880 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:14.880 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:15.230 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:15.239 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:15.360 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:15.360 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:15.360 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:09:15.360 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 1@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:09:15.360 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:09:15.362 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-24T08:09:15.362 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:09:15.362 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:09:15.362 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:09:15.362 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:09:15.366 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:09:15.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:15.367 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:09:15.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:15.450 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:15.550 INFO:teuthology.orchestra.run.smithi156.stdout:client4531 2023-11-24T08:09:15.551 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:15.551 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:15.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:15.605 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:15.661 INFO:teuthology.orchestra.run.smithi156.stdout:client4531 2023-11-24T08:09:15.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:15.661 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:15.719 INFO:teuthology.orchestra.run.smithi156.stdout:client4531 2023-11-24T08:09:15.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:15.720 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1a897c2-15ce-4272-ad66-4b3723970cb1.client4531/status') 2023-11-24T08:09:15.809 INFO:teuthology.orchestra.run.smithi156.stdout:instance: client.4531 (3)192.168.0.1:0/2431898340 2023-11-24T08:09:15.809 INFO:teuthology.orchestra.run.smithi156.stdout:blocklisted: false 2023-11-24T08:09:15.810 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records in 2023-11-24T08:09:15.810 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records out 2023-11-24T08:09:15.810 INFO:teuthology.orchestra.run.smithi156.stderr:69 bytes copied, 0.000105396 s, 655 kB/s 2023-11-24T08:09:15.813 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-11-24T08:09:15.814 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-24T08:09:15.814 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T08:09:15.814 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:09:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.039+0000 7f864c83d640 1 Processor -- start 2023-11-24T08:09:16.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.039+0000 7f864c83d640 1 -- start start 2023-11-24T08:09:16.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864c83d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f864815df80 0x7f86480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864c83d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 0x7f864815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864c83d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f864815e3f0 con 0x7f86480b2250 2023-11-24T08:09:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864c83d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f864815a990 con 0x7f864815df80 2023-11-24T08:09:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864c83d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f864815ab80 con 0x7f86480cd470 2023-11-24T08:09:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864659c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f86480b2250 0x7f86480b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54030/0 (socket says 172.21.15.106:54030) 2023-11-24T08:09:16.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.041+0000 7f864659c640 1 -- 172.21.15.106:0/1467734642 learned_addr learned my addr 172.21.15.106:0/1467734642 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f8646d9d640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 0x7f864815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f8645d9b640 1 --2- 172.21.15.106:0/1467734642 >> v2:172.21.15.156:3300/0 conn(0x7f864815df80 0x7f86480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2407827781 0 0) 0x7f864815e3f0 con 0x7f86480b2250 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8634003520 con 0x7f86480b2250 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2404736934 0 0) 0x7f8634003520 con 0x7f86480b2250 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 msgr2=0x7f864815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 0x7f864815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 >> v2:172.21.15.156:3300/0 conn(0x7f864815df80 msgr2=0x7f86480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 --2- 172.21.15.106:0/1467734642 >> v2:172.21.15.156:3300/0 conn(0x7f864815df80 0x7f86480ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86480b2d40 con 0x7f86480b2250 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f86380019f0 con 0x7f86480b2250 2023-11-24T08:09:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 >> v1:172.21.15.106:6789/0 conn(0x7f86480b2250 legacy=0x7f86480b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.042+0000 7f864559a640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8634003970 0x7f8634003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864559a640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 0x7f86340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864559a640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8634004d00 0x7f8634005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864815ab80 con 0x7f8634003970 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f863400db90 con 0x7f8634004d00 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f863400dcd0 con 0x7f86480cd470 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864659c640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8634003970 0x7f8634003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864659c640 1 -- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 msgr2=0x7f86340047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864659c640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f86480cd470 0x7f86340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864659c640 1 -- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8634004d00 msgr2=0x7f8634005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864659c640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8634004d00 0x7f8634005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.043+0000 7f864659c640 1 -- 172.21.15.106:0/1467734642 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634003520 con 0x7f8634003970 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.044+0000 7f864659c640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8634003970 0x7f8634003d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f863800a810 tx=0x7f86340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6313657d908c3ca6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.044+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638034630 con 0x7f8634003970 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.044+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863802f070 con 0x7f8634003970 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.044+0000 7f864559a640 1 -- 172.21.15.106:0/1467734642 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380349f0 con 0x7f8634003970 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f864c83d640 1 -- 172.21.15.106:0/1467734642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8634003970 msgr2=0x7f8634003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f864c83d640 1 --2- 172.21.15.106:0/1467734642 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8634003970 0x7f8634003d90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f863800a810 tx=0x7f86340057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f8646d9d640 1 -- 172.21.15.106:0/1467734642 reap_dead start 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f864c83d640 1 -- 172.21.15.106:0/1467734642 shutdown_connections 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f864c83d640 1 -- 172.21.15.106:0/1467734642 >> 172.21.15.106:0/1467734642 conn(0x7f864806a920 msgr2=0x7f864806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:16.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f864c83d640 1 -- 172.21.15.106:0/1467734642 shutdown_connections 2023-11-24T08:09:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.045+0000 7f864c83d640 1 -- 172.21.15.106:0/1467734642 wait complete. 2023-11-24T08:09:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.047+0000 7f864c83d640 1 Processor -- start 2023-11-24T08:09:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.047+0000 7f864c83d640 1 -- start start 2023-11-24T08:09:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f864c83d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8634003970 0x7f86481494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f864c83d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f864815c550 0x7f8648155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f864c83d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 0x7f86480c91b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f864c83d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86480c96f0 con 0x7f86480c8d20 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f864c83d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864815daf0 con 0x7f864815c550 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f864c83d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8648149a20 con 0x7f8634003970 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f8646d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 0x7f86480c91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.048+0000 7f8646d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 0x7f86480c91b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34802/0 (socket says 172.21.15.106:34802) 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 -- 172.21.15.106:0/1407307223 learned_addr learned my addr 172.21.15.106:0/1407307223 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8645d9b640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f864815c550 0x7f8648155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f864659c640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8634003970 0x7f86481494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 -- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8634003970 msgr2=0x7f86481494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:16.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8634003970 0x7f86481494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 -- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f864815c550 msgr2=0x7f8648155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f864815c550 0x7f8648155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 -- 172.21.15.106:0/1407307223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864815ca50 con 0x7f86480c8d20 2023-11-24T08:09:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f864659c640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8634003970 0x7f86481494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:16.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.049+0000 7f8646d9d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 0x7f86480c91b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8648158560 tx=0x7f864000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.050+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640013680 con 0x7f86480c8d20 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.050+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8640013800 con 0x7f86480c8d20 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.050+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864815cd30 con 0x7f86480c8d20 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.050+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8640013bb0 con 0x7f86480c8d20 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.050+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864815d2f0 con 0x7f86480c8d20 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.051+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86480cb3f0 con 0x7f86480c8d20 2023-11-24T08:09:16.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.052+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8640013d10 con 0x7f86480c8d20 2023-11-24T08:09:16.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.053+0000 7f863effd640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f862806ae40 0x7f862806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:16.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.053+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f86400891d0 con 0x7f86480c8d20 2023-11-24T08:09:16.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.053+0000 7f864659c640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f862806ae40 0x7f862806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:16.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.053+0000 7f864659c640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f862806ae40 0x7f862806d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8638034000 tx=0x7f8638009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:16.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.058+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8640054180 con 0x7f86480c8d20 2023-11-24T08:09:16.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.286+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8648067470 con 0x7f86480c8d20 2023-11-24T08:09:16.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.286+0000 7f863effd640 1 -- 172.21.15.106:0/1407307223 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+73 (secure 0 0 0) 0x7f86400577a0 con 0x7f86480c8d20 2023-11-24T08:09:16.288 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:09:16.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f862806ae40 msgr2=0x7f862806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:16.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f862806ae40 0x7f862806d300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8638034000 tx=0x7f8638009030 comp rx=0 tx=0).stop 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 msgr2=0x7f86480c91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 0x7f86480c91b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f8648158560 tx=0x7f864000e960 comp rx=0 tx=0).stop 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 shutdown_connections 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f86480c8d20 0x7f86480c91b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f864815c550 0x7f8648155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f862806ae40 0x7f862806d300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 --2- 172.21.15.106:0/1407307223 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8634003970 0x7f86481494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 >> 172.21.15.106:0/1407307223 conn(0x7f864806a920 msgr2=0x7f86480ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 shutdown_connections 2023-11-24T08:09:16.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:16.289+0000 7f864c83d640 1 -- 172.21.15.106:0/1407307223 wait complete. 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:09:16.308 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-24T08:09:16.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.309 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:09:16.371 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42376sec preferred_lft 42376sec 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:16.372 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::1831:a6ff:fe65:552d/64 scope link 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::683b:9fff:fe67:7152/64 scope link 2023-11-24T08:09:16.373 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:16.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.374 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:09:16.463 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:09:16.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.468 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:09:16.522 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:09:16.522 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:09:16.522 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:09:16.522 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:09:16.522 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:09:16.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.523 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:16.523 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:16.523 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:09:16.523 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:09:16.523 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:09:16.523 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:16.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.704 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:09:16.753 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:09:16.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.755 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list-id 2023-11-24T08:09:16.804 INFO:teuthology.orchestra.run.smithi156.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-24T08:09:16.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.805 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:16.805 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:16.805 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:09:16.805 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-24T08:09:16.805 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:16.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:16.947 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T08:09:17.039 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether fa:34:09:f5:6f:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::f834:9ff:fef5:6fab/64 scope link 2023-11-24T08:09:17.040 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:17.043 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-24T08:09:17.130 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:09:17.131 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:09:17.131 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:09:17.131 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.131 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:09:17.131 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.131 INFO:teuthology.orchestra.run.smithi156.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:09:17.132 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether fa:34:09:f5:6f:ab brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-24T08:09:17.132 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-24T08:09:17.132 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.132 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::f834:9ff:fef5:6fab/64 scope link 2023-11-24T08:09:17.132 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:09:17.134 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-24T08:09:17.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-24T08:09:17.135 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-24T08:09:17.136 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:17.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:17.402 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:09:17.402 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:09:17.402 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.1 up 2023-11-24T08:09:17.402 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set dev brx.1 master ceph-brx 2023-11-24T08:09:17.402 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:09:17.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:17.570 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-24T08:09:17.618 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-24T08:09:17.619 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:09:17.619 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-24T08:09:17.669 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-11-24T08:09:17.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:17.669 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:17.801 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:17.802 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:17.834 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:09:17.834 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:09:17.834 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:09:17.834 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 3@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:09:17.834 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:09:17.836 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2023-11-24T08:09:17.836 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:09:17.836 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:09:17.836 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:09:17.836 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:09:17.840 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:09:17.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:17.841 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-24T08:09:17.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:17.934 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:17.992 INFO:teuthology.orchestra.run.smithi156.stdout:client4397 2023-11-24T08:09:17.992 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:17.992 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:18.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:18.043 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:18.149 INFO:teuthology.orchestra.run.smithi156.stdout:client4397 2023-11-24T08:09:18.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:18.149 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:09:18.207 INFO:teuthology.orchestra.run.smithi156.stdout:client4397 2023-11-24T08:09:18.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:18.208 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/d1a897c2-15ce-4272-ad66-4b3723970cb1.client4397/status') 2023-11-24T08:09:18.300 INFO:teuthology.orchestra.run.smithi156.stdout:instance: client.4397 (3)192.168.0.2:0/2331703112 2023-11-24T08:09:18.301 INFO:teuthology.orchestra.run.smithi156.stdout:blocklisted: false 2023-11-24T08:09:18.301 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records in 2023-11-24T08:09:18.301 INFO:teuthology.orchestra.run.smithi156.stderr:0+1 records out 2023-11-24T08:09:18.301 INFO:teuthology.orchestra.run.smithi156.stderr:69 bytes copied, 8.1511e-05 s, 847 kB/s 2023-11-24T08:09:18.305 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-24T08:09:18.321 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:18.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.542+0000 7f641b59e640 1 Processor -- start 2023-11-24T08:09:18.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.543+0000 7f641b59e640 1 -- start start 2023-11-24T08:09:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.544+0000 7f641b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 0x7f641c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.544+0000 7f641b59e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641c15bf80 con 0x7f641c0b1de0 2023-11-24T08:09:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.544+0000 7f641b59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f641c0b1560 con 0x7f641c15c410 2023-11-24T08:09:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.544+0000 7f641b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0b1280 con 0x7f641c144b30 2023-11-24T08:09:18.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f641ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f641c0b1de0 0x7f641c0b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54042/0 (socket says 172.21.15.106:54042) 2023-11-24T08:09:18.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f641ad9d640 1 -- 172.21.15.106:0/3251721578 learned_addr learned my addr 172.21.15.106:0/3251721578 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:18.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f641a59c640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 0x7f641c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2266727785 0 0) 0x7f641c15bf80 con 0x7f641c0b1de0 2023-11-24T08:09:18.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6408003520 con 0x7f641c0b1de0 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3735585291 0 0) 0x7f6408003520 con 0x7f641c0b1de0 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 msgr2=0x7f641c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f6419d9b640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 0x7f641c15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.545+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 >> v1:172.21.15.156:6789/0 conn(0x7f641c15c410 legacy=0x7f641c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c15b930 con 0x7f641c0b1de0 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f6414001670 con 0x7f641c0b1de0 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 >> v1:172.21.15.106:6789/0 conn(0x7f641c0b1de0 legacy=0x7f641c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 0x7f64080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408003f20 0x7f64080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408004d00 0x7f6408005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0b1280 con 0x7f6408003f20 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640800ba60 con 0x7f6408004d00 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f640800bc20 con 0x7f641c144b30 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6413fff640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408003f20 0x7f64080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f641ad9d640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408004d00 0x7f6408005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f641a59c640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 0x7f64080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6413fff640 1 -- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 msgr2=0x7f64080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6413fff640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c144b30 0x7f64080039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6413fff640 1 -- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408004d00 msgr2=0x7f6408005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6413fff640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408004d00 0x7f6408005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.546+0000 7f6413fff640 1 -- 172.21.15.106:0/3251721578 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6408003520 con 0x7f6408003f20 2023-11-24T08:09:18.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f6413fff640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408003f20 0x7f64080047c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6404001750 tx=0x7f64080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c75772595ecdf5b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64040052d0 con 0x7f6408003f20 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f640402e050 con 0x7f6408003f20 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f6419d9b640 1 -- 172.21.15.106:0/3251721578 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6404003830 con 0x7f6408003f20 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f641b59e640 1 -- 172.21.15.106:0/3251721578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408003f20 msgr2=0x7f64080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f641b59e640 1 --2- 172.21.15.106:0/3251721578 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6408003f20 0x7f64080047c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6404001750 tx=0x7f64080057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f641ad9d640 1 -- 172.21.15.106:0/3251721578 reap_dead start 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f641b59e640 1 -- 172.21.15.106:0/3251721578 shutdown_connections 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.548+0000 7f641b59e640 1 -- 172.21.15.106:0/3251721578 >> 172.21.15.106:0/3251721578 conn(0x7f641c0cb830 msgr2=0x7f641c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.549+0000 7f641b59e640 1 -- 172.21.15.106:0/3251721578 shutdown_connections 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.549+0000 7f641b59e640 1 -- 172.21.15.106:0/3251721578 wait complete. 2023-11-24T08:09:18.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.550+0000 7f641b59e640 1 Processor -- start 2023-11-24T08:09:18.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.550+0000 7f641b59e640 1 -- start start 2023-11-24T08:09:18.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.551+0000 7f641b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408003f20 0x7f641c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.551+0000 7f641b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c1f4cf0 0x7f641c0c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.551+0000 7f641b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 0x7f641c12f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.551+0000 7f641b59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c130230 con 0x7f641c0c7fa0 2023-11-24T08:09:18.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.551+0000 7f641b59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0b1000 con 0x7f6408003f20 2023-11-24T08:09:18.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.551+0000 7f641b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c0c8420 con 0x7f641c1f4cf0 2023-11-24T08:09:18.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 0x7f641c12f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 0x7f641c12f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34816/0 (socket says 172.21.15.106:34816) 2023-11-24T08:09:18.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 -- 172.21.15.106:0/2361372887 learned_addr learned my addr 172.21.15.106:0/2361372887 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:18.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 -- 172.21.15.106:0/2361372887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c1f4cf0 msgr2=0x7f641c0c7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c1f4cf0 0x7f641c0c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 -- 172.21.15.106:0/2361372887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408003f20 msgr2=0x7f641c15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408003f20 0x7f641c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 -- 172.21.15.106:0/2361372887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c1f4860 con 0x7f641c0c7fa0 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f641ad9d640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 0x7f641c12f290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f641400e700 tx=0x7f641400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.552+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641400c5e0 con 0x7f641c0c7fa0 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.553+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f641400c740 con 0x7f641c0c7fa0 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.553+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f641400caf0 con 0x7f641c0c7fa0 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.553+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641c12ff50 con 0x7f641c0c7fa0 2023-11-24T08:09:18.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.553+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641c1f44f0 con 0x7f641c0c7fa0 2023-11-24T08:09:18.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.554+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f641c1f4070 con 0x7f641c0c7fa0 2023-11-24T08:09:18.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.555+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f641400cc50 con 0x7f641c0c7fa0 2023-11-24T08:09:18.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.556+0000 7f6412ffd640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f63fc06ae70 0x7f63fc06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.556+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6414088700 con 0x7f641c0c7fa0 2023-11-24T08:09:18.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.556+0000 7f641a59c640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f63fc06ae70 0x7f63fc06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.557+0000 7f641a59c640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f63fc06ae70 0x7f63fc06d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f640c009fd0 tx=0x7f640c009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:18.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.560+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f641c1f4070 con 0x7f641c0c7fa0 2023-11-24T08:09:18.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.749+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f641c1f4070 con 0x7f641c0c7fa0 2023-11-24T08:09:18.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.751+0000 7f6412ffd640 1 -- 172.21.15.106:0/2361372887 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f641c1f4070 con 0x7f641c0c7fa0 2023-11-24T08:09:18.753 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:18.754 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:18.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f63fc06ae70 msgr2=0x7f63fc06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f63fc06ae70 0x7f63fc06d330 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f640c009fd0 tx=0x7f640c009750 comp rx=0 tx=0).stop 2023-11-24T08:09:18.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 msgr2=0x7f641c12f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 0x7f641c12f290 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f641400e700 tx=0x7f641400ebc0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 shutdown_connections 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f641c0c7fa0 0x7f641c12f290 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f63fc06ae70 0x7f63fc06d330 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f641c1f4cf0 0x7f641c0c7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 --2- 172.21.15.106:0/2361372887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6408003f20 0x7f641c15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.754+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 >> 172.21.15.106:0/2361372887 conn(0x7f641c0cb830 msgr2=0x7f641c0b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.755+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 shutdown_connections 2023-11-24T08:09:18.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.755+0000 7f641b59e640 1 -- 172.21.15.106:0/2361372887 wait complete. 2023-11-24T08:09:18.773 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:18.774 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.987+0000 7f20f2ae1640 1 Processor -- start 2023-11-24T08:09:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.987+0000 7f20f2ae1640 1 -- start start 2023-11-24T08:09:18.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.988+0000 7f20f2ae1640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20ec15b260 0x7f20ec06e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.988+0000 7f20f2ae1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20ec0b1250 0x7f20ec06c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.988+0000 7f20f2ae1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20ec15ed20 con 0x7f20ec06e580 2023-11-24T08:09:18.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.988+0000 7f20f2ae1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec15b640 con 0x7f20ec0b1250 2023-11-24T08:09:18.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.988+0000 7f20f2ae1640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20ec15f8a0 con 0x7f20ec15b260 2023-11-24T08:09:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f12de640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f20ec06e580 0x7f20ec0b0b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54054/0 (socket says 172.21.15.106:54054) 2023-11-24T08:09:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f12de640 1 -- 172.21.15.106:0/3323587456 learned_addr learned my addr 172.21.15.106:0/3323587456 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f22e0640 1 --2- 172.21.15.106:0/3323587456 >> v2:172.21.15.106:3300/0 conn(0x7f20ec0b1250 0x7f20ec06c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f1adf640 1 --2- 172.21.15.106:0/3323587456 >> v2:172.21.15.156:3300/0 conn(0x7f20ec15b260 0x7f20ec06e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f0add640 1 -- 172.21.15.106:0/3323587456 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2815457693 0 0) 0x7f20ec15ed20 con 0x7f20ec06e580 2023-11-24T08:09:18.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f0add640 1 -- 172.21.15.106:0/3323587456 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20dc003520 con 0x7f20ec06e580 2023-11-24T08:09:18.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f22e0640 1 -- 172.21.15.106:0/3323587456 >> v2:172.21.15.156:3300/0 conn(0x7f20ec15b260 msgr2=0x7f20ec06e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:18.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f22e0640 1 --2- 172.21.15.106:0/3323587456 >> v2:172.21.15.156:3300/0 conn(0x7f20ec15b260 0x7f20ec06e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f22e0640 1 -- 172.21.15.106:0/3323587456 >> v1:172.21.15.106:6789/0 conn(0x7f20ec06e580 legacy=0x7f20ec0b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.989+0000 7f20f22e0640 1 -- 172.21.15.106:0/3323587456 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20ec0abe50 con 0x7f20ec0b1250 2023-11-24T08:09:18.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.991+0000 7f20f22e0640 1 --2- 172.21.15.106:0/3323587456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec0b1250 0x7f20ec06c5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0d0160 tx=0x7f20e402b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba21f82aafd2898c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:18.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.991+0000 7f20f0add640 1 -- 172.21.15.106:0/3323587456 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4031710 con 0x7f20ec0b1250 2023-11-24T08:09:18.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.991+0000 7f20f0add640 1 -- 172.21.15.106:0/3323587456 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e402f050 con 0x7f20ec0b1250 2023-11-24T08:09:18.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.991+0000 7f20f0add640 1 -- 172.21.15.106:0/3323587456 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e4031a90 con 0x7f20ec0b1250 2023-11-24T08:09:18.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.991+0000 7f20f2ae1640 1 -- 172.21.15.106:0/3323587456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec0b1250 msgr2=0x7f20ec06c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.991+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/3323587456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec0b1250 0x7f20ec06c5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0d0160 tx=0x7f20e402b7b0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.992+0000 7f20f2ae1640 1 -- 172.21.15.106:0/3323587456 shutdown_connections 2023-11-24T08:09:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.992+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/3323587456 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec0b1250 0x7f20ec06c5b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.992+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/3323587456 >> v2:172.21.15.156:3300/0 conn(0x7f20ec15b260 0x7f20ec06e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.992+0000 7f20f2ae1640 1 -- 172.21.15.106:0/3323587456 >> 172.21.15.106:0/3323587456 conn(0x7f20ec0cbb40 msgr2=0x7f20ec0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.992+0000 7f20f2ae1640 1 -- 172.21.15.106:0/3323587456 shutdown_connections 2023-11-24T08:09:18.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.992+0000 7f20f2ae1640 1 -- 172.21.15.106:0/3323587456 wait complete. 2023-11-24T08:09:18.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.993+0000 7f20f2ae1640 1 Processor -- start 2023-11-24T08:09:18.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.994+0000 7f20f2ae1640 1 -- start start 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f2ae1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 0x7f20ec15c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f2ae1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20ec0b1250 0x7f20ec15da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f2ae1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ec15b260 0x7f20ec14c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f2ae1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec14a310 con 0x7f20ec06e580 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f2ae1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec150980 con 0x7f20ec15b260 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f2ae1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20ec15c960 con 0x7f20ec0b1250 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 0x7f20ec15c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 0x7f20ec15c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34844/0 (socket says 172.21.15.106:34844) 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 -- 172.21.15.106:0/2100663347 learned_addr learned my addr 172.21.15.106:0/2100663347 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:18.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f22e0640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ec15b260 0x7f20ec14c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:18.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 -- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20ec0b1250 msgr2=0x7f20ec15da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20ec0b1250 0x7f20ec15da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 -- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ec15b260 msgr2=0x7f20ec14c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:18.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ec15b260 0x7f20ec14c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:18.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.995+0000 7f20f1adf640 1 -- 172.21.15.106:0/2100663347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4009400 con 0x7f20ec06e580 2023-11-24T08:09:18.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.996+0000 7f20f1adf640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 0x7f20ec15c380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20e0007e40 tx=0x7f20e000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.996+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e0014620 con 0x7f20ec06e580 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.996+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e000ee90 con 0x7f20ec06e580 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.996+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e00149c0 con 0x7f20ec06e580 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.996+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20ec15cc40 con 0x7f20ec06e580 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.996+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20ec15d200 con 0x7f20ec06e580 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.997+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20ec04e840 con 0x7f20ec06e580 2023-11-24T08:09:18.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.998+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f20e0014b20 con 0x7f20ec06e580 2023-11-24T08:09:19.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.999+0000 7f20ea7fc640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20d006ae40 0x7f20d006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.999+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20e0089f50 con 0x7f20ec06e580 2023-11-24T08:09:19.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:18.999+0000 7f20f12de640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20d006ae40 0x7f20d006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.000+0000 7f20f12de640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20d006ae40 0x7f20d006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0ac660 tx=0x7f20d8007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.003+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20e0055000 con 0x7f20ec06e580 2023-11-24T08:09:19.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.191+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20ec0ce640 con 0x7f20ec06e580 2023-11-24T08:09:19.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.192+0000 7f20ea7fc640 1 -- 172.21.15.106:0/2100663347 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20e0058620 con 0x7f20ec06e580 2023-11-24T08:09:19.194 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:19.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20d006ae40 msgr2=0x7f20d006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20d006ae40 0x7f20d006d300 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0ac660 tx=0x7f20d8007930 comp rx=0 tx=0).stop 2023-11-24T08:09:19.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 msgr2=0x7f20ec15c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 0x7f20ec15c380 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f20e0007e40 tx=0x7f20e000e4b0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 shutdown_connections 2023-11-24T08:09:19.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20ec15b260 0x7f20ec14c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20d006ae40 0x7f20d006d300 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20ec0b1250 0x7f20ec15da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 --2- 172.21.15.106:0/2100663347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20ec06e580 0x7f20ec15c380 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 >> 172.21.15.106:0/2100663347 conn(0x7f20ec0cbb40 msgr2=0x7f20ec0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:19.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 shutdown_connections 2023-11-24T08:09:19.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.195+0000 7f20f2ae1640 1 -- 172.21.15.106:0/2100663347 wait complete. 2023-11-24T08:09:19.215 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.417+0000 7f2fe0be5640 1 Processor -- start 2023-11-24T08:09:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.417+0000 7f2fe0be5640 1 -- start start 2023-11-24T08:09:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.419+0000 7f2fe0be5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2fdc140720 0x7f2fdc0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.419+0000 7f2fe0be5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fdc161080 0x7f2fdc0c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.419+0000 7f2fe0be5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fdc161600 con 0x7f2fdc140340 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.419+0000 7f2fe0be5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc1618e0 con 0x7f2fdc161080 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.419+0000 7f2fe0be5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fdc161bc0 con 0x7f2fdc140720 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.419+0000 7f2fdad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fdc161080 0x7f2fdc0c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fdad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fdc161080 0x7f2fdc0c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34860/0 (socket says 172.21.15.106:34860) 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fdad9d640 1 -- 172.21.15.106:0/4065834807 learned_addr learned my addr 172.21.15.106:0/4065834807 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fd3fff640 1 --2- 172.21.15.106:0/4065834807 >> v2:172.21.15.156:3300/0 conn(0x7f2fdc140720 0x7f2fdc0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fdad9d640 1 -- 172.21.15.106:0/4065834807 >> v2:172.21.15.156:3300/0 conn(0x7f2fdc140720 msgr2=0x7f2fdc0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fdad9d640 1 --2- 172.21.15.106:0/4065834807 >> v2:172.21.15.156:3300/0 conn(0x7f2fdc140720 0x7f2fdc0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fdad9d640 1 -- 172.21.15.106:0/4065834807 >> v1:172.21.15.156:6789/0 conn(0x7f2fdc140340 legacy=0x7f2fdc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.420+0000 7f2fdad9d640 1 -- 172.21.15.106:0/4065834807 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fdc161ea0 con 0x7f2fdc161080 2023-11-24T08:09:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.421+0000 7f2fdad9d640 1 --2- 172.21.15.106:0/4065834807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc161080 0x7f2fdc0c60b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0d22d0 tx=0x7f2fd402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d0335c5679a6f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fd9d9b640 1 -- 172.21.15.106:0/4065834807 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd402c9f0 con 0x7f2fdc161080 2023-11-24T08:09:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fd9d9b640 1 -- 172.21.15.106:0/4065834807 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd402cb50 con 0x7f2fdc161080 2023-11-24T08:09:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fd9d9b640 1 -- 172.21.15.106:0/4065834807 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40343f0 con 0x7f2fdc161080 2023-11-24T08:09:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fe0be5640 1 -- 172.21.15.106:0/4065834807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc161080 msgr2=0x7f2fdc0c60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/4065834807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc161080 0x7f2fdc0c60b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc0d22d0 tx=0x7f2fd402be50 comp rx=0 tx=0).stop 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fe0be5640 1 -- 172.21.15.106:0/4065834807 shutdown_connections 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/4065834807 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc161080 0x7f2fdc0c60b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/4065834807 >> v2:172.21.15.156:3300/0 conn(0x7f2fdc140720 0x7f2fdc0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.422+0000 7f2fe0be5640 1 -- 172.21.15.106:0/4065834807 >> 172.21.15.106:0/4065834807 conn(0x7f2fdc0b2e40 msgr2=0x7f2fdc0b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.423+0000 7f2fe0be5640 1 -- 172.21.15.106:0/4065834807 shutdown_connections 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.423+0000 7f2fe0be5640 1 -- 172.21.15.106:0/4065834807 wait complete. 2023-11-24T08:09:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.424+0000 7f2fe0be5640 1 Processor -- start 2023-11-24T08:09:19.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.424+0000 7f2fe0be5640 1 -- start start 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.425+0000 7f2fe0be5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fdc140340 0x7f2fdc13fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.425+0000 7f2fe0be5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 0x7f2fdc15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.425+0000 7f2fe0be5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc161080 0x7f2fdc06bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.425+0000 7f2fe0be5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1f63b0 con 0x7f2fdc140720 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.425+0000 7f2fe0be5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1f6650 con 0x7f2fdc161080 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.425+0000 7f2fe0be5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1f6930 con 0x7f2fdc140340 2023-11-24T08:09:19.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 0x7f2fdc15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 0x7f2fdc15fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34876/0 (socket says 172.21.15.106:34876) 2023-11-24T08:09:19.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 -- 172.21.15.106:0/2149493490 learned_addr learned my addr 172.21.15.106:0/2149493490 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 -- 172.21.15.106:0/2149493490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fdc140340 msgr2=0x7f2fdc13fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fdc140340 0x7f2fdc13fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 -- 172.21.15.106:0/2149493490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc161080 msgr2=0x7f2fdc06bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc161080 0x7f2fdc06bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 -- 172.21.15.106:0/2149493490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fd4009b30 con 0x7f2fdc140720 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.426+0000 7f2fd3fff640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 0x7f2fdc15fa90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400d800 tx=0x7f2fc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.427+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400b6c0 con 0x7f2fdc140720 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.427+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fc400b820 con 0x7f2fdc140720 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.427+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc400bbb0 con 0x7f2fdc140720 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.427+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc1f6c10 con 0x7f2fdc140720 2023-11-24T08:09:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.427+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc1400d0 con 0x7f2fdc140720 2023-11-24T08:09:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.428+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fdc067990 con 0x7f2fdc140720 2023-11-24T08:09:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.429+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2fc400bd10 con 0x7f2fdc140720 2023-11-24T08:09:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.430+0000 7f2fd2ffd640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fbc06ae90 0x7f2fbc06d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.430+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fc4089b40 con 0x7f2fdc140720 2023-11-24T08:09:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.430+0000 7f2fda59c640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fbc06ae90 0x7f2fbc06d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.431+0000 7f2fda59c640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fbc06ae90 0x7f2fbc06d350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc001560 tx=0x7f2fcc007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.434+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fc4054c60 con 0x7f2fdc140720 2023-11-24T08:09:19.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.628+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2fdc04e8b0 con 0x7f2fdc140720 2023-11-24T08:09:19.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.629+0000 7f2fd2ffd640 1 -- 172.21.15.106:0/2149493490 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2fc4058280 con 0x7f2fdc140720 2023-11-24T08:09:19.631 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:19.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fbc06ae90 msgr2=0x7f2fbc06d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fbc06ae90 0x7f2fbc06d350 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc001560 tx=0x7f2fcc007990 comp rx=0 tx=0).stop 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 msgr2=0x7f2fdc15fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 0x7f2fdc15fa90 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2fc400d800 tx=0x7f2fc400dcc0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 shutdown_connections 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc161080 0x7f2fdc06bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fdc140720 0x7f2fdc15fa90 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fbc06ae90 0x7f2fbc06d350 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 --2- 172.21.15.106:0/2149493490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fdc140340 0x7f2fdc13fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 >> 172.21.15.106:0/2149493490 conn(0x7f2fdc0b2e40 msgr2=0x7f2fdc0d18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 shutdown_connections 2023-11-24T08:09:19.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.632+0000 7f2fe0be5640 1 -- 172.21.15.106:0/2149493490 wait complete. 2023-11-24T08:09:19.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:19.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.858+0000 7fd1d3289640 1 Processor -- start 2023-11-24T08:09:19.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.858+0000 7fd1d3289640 1 -- start start 2023-11-24T08:09:19.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d3289640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 0x7fd1cc15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d3289640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc15ec90 con 0x7fd1cc072160 2023-11-24T08:09:19.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d3289640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0cd530 con 0x7fd1cc0c49f0 2023-11-24T08:09:19.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d3289640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c4ed0 con 0x7fd1cc06de10 2023-11-24T08:09:19.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d1025640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd1cc072160 0x7fd1cc06d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54062/0 (socket says 172.21.15.106:54062) 2023-11-24T08:09:19.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d1025640 1 -- 172.21.15.106:0/782622615 learned_addr learned my addr 172.21.15.106:0/782622615 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179635492 0 0) 0x7fd1c8001110 con 0x7fd1cc072160 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b8003520 con 0x7fd1cc072160 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.860+0000 7fd1d0824640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 0x7fd1cc15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1736291470 0 0) 0x7fd1b8003520 con 0x7fd1cc072160 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 msgr2=0x7fd1cc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 0x7fd1cc15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 >> v1:172.21.15.156:6789/0 conn(0x7fd1cc0c49f0 legacy=0x7fd1cc0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0c51b0 con 0x7fd1cc072160 2023-11-24T08:09:19.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fd1c8001800 con 0x7fd1cc072160 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 >> v1:172.21.15.106:6789/0 conn(0x7fd1cc072160 legacy=0x7fd1cc06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1b80039a0 0x7fd1b8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 0x7fd1b80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0c4ed0 con 0x7fd1b80039a0 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800bab0 con 0x7fd1b80047c0 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800bc70 con 0x7fd1cc06de10 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1d1025640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1b80039a0 0x7fd1b8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.861+0000 7fd1d0824640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.862+0000 7fd1d1025640 1 -- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 msgr2=0x7fd1b80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.862+0000 7fd1d1025640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1cc06de10 0x7fd1b80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.862+0000 7fd1d1025640 1 -- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1b80047c0 msgr2=0x7fd1b8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:19.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.862+0000 7fd1d1025640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.862+0000 7fd1d1025640 1 -- 172.21.15.106:0/782622615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8003520 con 0x7fd1b80039a0 2023-11-24T08:09:19.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.862+0000 7fd1d0824640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1b80047c0 0x7fd1b8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:19.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1d1025640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1b80039a0 0x7fd1b8003dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c190 tx=0x7fd1b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8918d27b550be04 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c8005ad0 con 0x7fd1b80039a0 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c8005c50 con 0x7fd1b80039a0 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1c3fff640 1 -- 172.21.15.106:0/782622615 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c802c7e0 con 0x7fd1b80039a0 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1d3289640 1 -- 172.21.15.106:0/782622615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1b80039a0 msgr2=0x7fd1b8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1d3289640 1 --2- 172.21.15.106:0/782622615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1b80039a0 0x7fd1b8003dc0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd1c800c190 tx=0x7fd1b80057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1d1826640 1 -- 172.21.15.106:0/782622615 reap_dead start 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.863+0000 7fd1d3289640 1 -- 172.21.15.106:0/782622615 shutdown_connections 2023-11-24T08:09:19.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.864+0000 7fd1d3289640 1 -- 172.21.15.106:0/782622615 >> 172.21.15.106:0/782622615 conn(0x7fd1cc0b0c50 msgr2=0x7fd1cc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:19.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.864+0000 7fd1d3289640 1 -- 172.21.15.106:0/782622615 shutdown_connections 2023-11-24T08:09:19.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.864+0000 7fd1d3289640 1 -- 172.21.15.106:0/782622615 wait complete. 2023-11-24T08:09:19.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.865+0000 7fd1d3289640 1 Processor -- start 2023-11-24T08:09:19.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.866+0000 7fd1d3289640 1 -- start start 2023-11-24T08:09:19.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d3289640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1b80039a0 0x7fd1cc154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d3289640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc15cf00 0x7fd1cc0af0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d3289640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 0x7fd1cc0b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d3289640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0afac0 con 0x7fd1cc0af610 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d3289640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15be60 con 0x7fd1cc15cf00 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d3289640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc15c140 con 0x7fd1b80039a0 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 0x7fd1cc0b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 0x7fd1cc0b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34896/0 (socket says 172.21.15.106:34896) 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 -- 172.21.15.106:0/3024922111 learned_addr learned my addr 172.21.15.106:0/3024922111 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d0824640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc15cf00 0x7fd1cc0af0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1025640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1b80039a0 0x7fd1cc154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 -- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1b80039a0 msgr2=0x7fd1cc154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1b80039a0 0x7fd1cc154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 -- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc15cf00 msgr2=0x7fd1cc0af0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc15cf00 0x7fd1cc0af0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d1826640 1 -- 172.21.15.106:0/3024922111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc15c420 con 0x7fd1cc0af610 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.867+0000 7fd1d0824640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc15cf00 0x7fd1cc0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.868+0000 7fd1d1826640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 0x7fd1cc0b1c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0cd1b0 tx=0x7fd1c400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.868+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c4005d40 con 0x7fd1cc0af610 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.868+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c400ced0 con 0x7fd1cc0af610 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.868+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc15c700 con 0x7fd1cc0af610 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.868+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400a850 con 0x7fd1cc0af610 2023-11-24T08:09:19.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.868+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc155340 con 0x7fd1cc0af610 2023-11-24T08:09:19.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.869+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc15cc20 con 0x7fd1cc0af610 2023-11-24T08:09:19.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.870+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd1c400a9b0 con 0x7fd1cc0af610 2023-11-24T08:09:19.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.871+0000 7fd1c1ffb640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1b006ae70 0x7fd1b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:19.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.871+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1c4088aa0 con 0x7fd1cc0af610 2023-11-24T08:09:19.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.871+0000 7fd1d1025640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1b006ae70 0x7fd1b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:19.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.872+0000 7fd1d1025640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1b006ae70 0x7fd1b006d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8034000 tx=0x7fd1c8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:19.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:19.876+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd1cc15cc20 con 0x7fd1cc0af610 2023-11-24T08:09:20.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.052+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd1cc15cc20 con 0x7fd1cc0af610 2023-11-24T08:09:20.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.053+0000 7fd1c1ffb640 1 -- 172.21.15.106:0/3024922111 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd1cc15cc20 con 0x7fd1cc0af610 2023-11-24T08:09:20.055 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:20.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.055+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1b006ae70 msgr2=0x7fd1b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1b006ae70 0x7fd1b006d330 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8034000 tx=0x7fd1c8009030 comp rx=0 tx=0).stop 2023-11-24T08:09:20.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 msgr2=0x7fd1cc0b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 0x7fd1cc0b1c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0cd1b0 tx=0x7fd1c400cb90 comp rx=0 tx=0).stop 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 shutdown_connections 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1cc0af610 0x7fd1cc0b1c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1cc15cf00 0x7fd1cc0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd1b006ae70 0x7fd1b006d330 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 --2- 172.21.15.106:0/3024922111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd1b80039a0 0x7fd1cc154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 >> 172.21.15.106:0/3024922111 conn(0x7fd1cc0b0c50 msgr2=0x7fd1cc1591b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 shutdown_connections 2023-11-24T08:09:20.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.056+0000 7fd1d3289640 1 -- 172.21.15.106:0/3024922111 wait complete. 2023-11-24T08:09:20.074 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:20.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.280+0000 7f4c9ed78640 1 Processor -- start 2023-11-24T08:09:20.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.280+0000 7f4c9ed78640 1 -- start start 2023-11-24T08:09:20.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9ed78640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4c980b2250 0x7f4c9815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9ed78640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4c980b2aa0 0x7f4c9815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9ed78640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c9815ab80 con 0x7f4c980b2630 2023-11-24T08:09:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9ed78640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9815a990 con 0x7f4c980b2250 2023-11-24T08:09:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9ed78640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c9815e680 con 0x7f4c980b2aa0 2023-11-24T08:09:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9cb14640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4c980b2250 0x7f4c9815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c8ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4c980b2630 0x7f4c980ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54078/0 (socket says 172.21.15.106:54078) 2023-11-24T08:09:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c8ffff640 1 -- 172.21.15.106:0/111905339 learned_addr learned my addr 172.21.15.106:0/111905339 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:20.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.282+0000 7f4c9d315640 1 --2- 172.21.15.106:0/111905339 >> v2:172.21.15.156:3300/0 conn(0x7f4c980b2aa0 0x7f4c9815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.283+0000 7f4c9cb14640 1 -- 172.21.15.106:0/111905339 >> v2:172.21.15.156:3300/0 conn(0x7f4c980b2aa0 msgr2=0x7f4c9815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.283+0000 7f4c9cb14640 1 --2- 172.21.15.106:0/111905339 >> v2:172.21.15.156:3300/0 conn(0x7f4c980b2aa0 0x7f4c9815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.283+0000 7f4c9cb14640 1 -- 172.21.15.106:0/111905339 >> v1:172.21.15.106:6789/0 conn(0x7f4c980b2630 legacy=0x7f4c980ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.283+0000 7f4c9cb14640 1 -- 172.21.15.106:0/111905339 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9815e960 con 0x7f4c980b2250 2023-11-24T08:09:20.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.283+0000 7f4c9d315640 1 --2- 172.21.15.106:0/111905339 >> v2:172.21.15.156:3300/0 conn(0x7f4c980b2aa0 0x7f4c9815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.284+0000 7f4c9cb14640 1 --2- 172.21.15.106:0/111905339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2250 0x7f4c9815df80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c980b2fa0 tx=0x7f4c9402bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=321a0eec512c0c8e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.284+0000 7f4c8f7fe640 1 -- 172.21.15.106:0/111905339 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9402cc70 con 0x7f4c980b2250 2023-11-24T08:09:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.284+0000 7f4c8f7fe640 1 -- 172.21.15.106:0/111905339 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9402cdd0 con 0x7f4c980b2250 2023-11-24T08:09:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.284+0000 7f4c8f7fe640 1 -- 172.21.15.106:0/111905339 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c94034660 con 0x7f4c980b2250 2023-11-24T08:09:20.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.284+0000 7f4c9ed78640 1 -- 172.21.15.106:0/111905339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2250 msgr2=0x7f4c9815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.284+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/111905339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2250 0x7f4c9815df80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4c980b2fa0 tx=0x7f4c9402bec0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.285+0000 7f4c9ed78640 1 -- 172.21.15.106:0/111905339 shutdown_connections 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.285+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/111905339 >> v2:172.21.15.156:3300/0 conn(0x7f4c980b2aa0 0x7f4c9815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.285+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/111905339 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2250 0x7f4c9815df80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.285+0000 7f4c9ed78640 1 -- 172.21.15.106:0/111905339 >> 172.21.15.106:0/111905339 conn(0x7f4c9806a920 msgr2=0x7f4c9806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.285+0000 7f4c9ed78640 1 -- 172.21.15.106:0/111905339 shutdown_connections 2023-11-24T08:09:20.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.285+0000 7f4c9ed78640 1 -- 172.21.15.106:0/111905339 wait complete. 2023-11-24T08:09:20.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.286+0000 7f4c9ed78640 1 Processor -- start 2023-11-24T08:09:20.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.287+0000 7f4c9ed78640 1 -- start start 2023-11-24T08:09:20.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.287+0000 7f4c9ed78640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c980b2250 0x7f4c9814c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.287+0000 7f4c9ed78640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 0x7f4c980ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c9ed78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c980b2aa0 0x7f4c980cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c9ed78640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980707d0 con 0x7f4c980b2630 2023-11-24T08:09:20.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c9ed78640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98070ab0 con 0x7f4c980b2aa0 2023-11-24T08:09:20.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c9ed78640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98070d90 con 0x7f4c980b2250 2023-11-24T08:09:20.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 0x7f4c980ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 0x7f4c980ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34922/0 (socket says 172.21.15.106:34922) 2023-11-24T08:09:20.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 -- 172.21.15.106:0/374491508 learned_addr learned my addr 172.21.15.106:0/374491508 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 -- 172.21.15.106:0/374491508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c980b2250 msgr2=0x7f4c9814c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c980b2250 0x7f4c9814c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 -- 172.21.15.106:0/374491508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c980b2aa0 msgr2=0x7f4c980cab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c980b2aa0 0x7f4c980cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 -- 172.21.15.106:0/374491508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009a70 con 0x7f4c980b2630 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.288+0000 7f4c8ffff640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 0x7f4c980ca5f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c4a0 tx=0x7f4c8800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.289+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88005ce0 con 0x7f4c980b2630 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.289+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c88005e40 con 0x7f4c980b2630 2023-11-24T08:09:20.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.289+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c88007270 con 0x7f4c980b2630 2023-11-24T08:09:20.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.289+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c980cd7c0 con 0x7f4c980b2630 2023-11-24T08:09:20.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.289+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c98155050 con 0x7f4c980b2630 2023-11-24T08:09:20.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.290+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c980712b0 con 0x7f4c980b2630 2023-11-24T08:09:20.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.291+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4c88007490 con 0x7f4c980b2630 2023-11-24T08:09:20.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.292+0000 7f4c8d7fa640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4c8406ae40 0x7f4c8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.292+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c88088a60 con 0x7f4c980b2630 2023-11-24T08:09:20.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.292+0000 7f4c9cb14640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4c8406ae40 0x7f4c8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.293+0000 7f4c9cb14640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4c8406ae40 0x7f4c8406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c94036000 tx=0x7f4c9402e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:20.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.296+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4c980712b0 con 0x7f4c980b2630 2023-11-24T08:09:20.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.487+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c980712b0 con 0x7f4c980b2630 2023-11-24T08:09:20.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.490+0000 7f4c8d7fa640 1 -- 172.21.15.106:0/374491508 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f4c980712b0 con 0x7f4c980b2630 2023-11-24T08:09:20.492 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:20.492 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:20.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4c8406ae40 msgr2=0x7f4c8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4c8406ae40 0x7f4c8406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f4c94036000 tx=0x7f4c9402e000 comp rx=0 tx=0).stop 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 msgr2=0x7f4c980ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 0x7f4c980ca5f0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4c8800c4a0 tx=0x7f4c8800c960 comp rx=0 tx=0).stop 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 shutdown_connections 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c980b2aa0 0x7f4c980cab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4c980b2630 0x7f4c980ca5f0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4c8406ae40 0x7f4c8406d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 --2- 172.21.15.106:0/374491508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4c980b2250 0x7f4c9814c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 >> 172.21.15.106:0/374491508 conn(0x7f4c9806a920 msgr2=0x7f4c981597e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:20.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 shutdown_connections 2023-11-24T08:09:20.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.493+0000 7f4c9ed78640 1 -- 172.21.15.106:0/374491508 wait complete. 2023-11-24T08:09:20.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:20.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:20.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.716+0000 7f941c116640 1 Processor -- start 2023-11-24T08:09:20.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.716+0000 7f941c116640 1 -- start start 2023-11-24T08:09:20.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.717+0000 7f941c116640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f94140c59b0 0x7f9414166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f941c116640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94140c5dc0 con 0x7f941415b440 2023-11-24T08:09:20.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f941c116640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f941415eea0 con 0x7f94140cc0f0 2023-11-24T08:09:20.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f941c116640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f941415b820 con 0x7f94140c59b0 2023-11-24T08:09:20.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f9419eb2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f941415b440 0x7f94140cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54084/0 (socket says 172.21.15.106:54084) 2023-11-24T08:09:20.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f9419eb2640 1 -- 172.21.15.106:0/3597268451 learned_addr learned my addr 172.21.15.106:0/3597268451 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:20.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1182817247 0 0) 0x7f94140c5dc0 con 0x7f941415b440 2023-11-24T08:09:20.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9400003520 con 0x7f941415b440 2023-11-24T08:09:20.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.718+0000 7f941a6b3640 1 --2- 172.21.15.106:0/3597268451 >> v2:172.21.15.156:3300/0 conn(0x7f94140c59b0 0x7f9414166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3134486374 0 0) 0x7f9400003520 con 0x7f941415b440 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 >> v2:172.21.15.156:3300/0 conn(0x7f94140c59b0 msgr2=0x7f9414166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 --2- 172.21.15.106:0/3597268451 >> v2:172.21.15.156:3300/0 conn(0x7f94140c59b0 0x7f9414166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 >> v1:172.21.15.156:6789/0 conn(0x7f94140cc0f0 legacy=0x7f94140c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941406d600 con 0x7f941415b440 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f9410001670 con 0x7f941415b440 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 >> v1:172.21.15.106:6789/0 conn(0x7f941415b440 legacy=0x7f94140cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94000039c0 0x7f9400003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000047c0 0x7f9400004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9400005250 0x7f940000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941415b820 con 0x7f94000039c0 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f940000c1a0 con 0x7f94000047c0 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f940000c360 con 0x7f9400005250 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f9419eb2640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94000039c0 0x7f9400003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.719+0000 7f94196b1640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000047c0 0x7f9400004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f941a6b3640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9400005250 0x7f940000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f9419eb2640 1 -- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9400005250 msgr2=0x7f940000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f9419eb2640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9400005250 0x7f940000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f9419eb2640 1 -- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000047c0 msgr2=0x7f9400004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:20.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f9419eb2640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000047c0 0x7f9400004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f9419eb2640 1 -- 172.21.15.106:0/3597268451 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9400003520 con 0x7f94000039c0 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f941a6b3640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9400005250 0x7f940000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.720+0000 7f941a6b3640 1 -- 172.21.15.106:0/3597268451 reap_dead start 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.721+0000 7f9419eb2640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94000039c0 0x7f9400003dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f941000c0e0 tx=0x7f94000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0adc826dfaa40a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.721+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9410005ad0 con 0x7f94000039c0 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.721+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9410005c30 con 0x7f94000039c0 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.721+0000 7f9418eb0640 1 -- 172.21.15.106:0/3597268451 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941002c7c0 con 0x7f94000039c0 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 -- 172.21.15.106:0/3597268451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94000039c0 msgr2=0x7f9400003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94000039c0 0x7f9400003dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f941000c0e0 tx=0x7f94000057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 -- 172.21.15.106:0/3597268451 shutdown_connections 2023-11-24T08:09:20.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 --2- 172.21.15.106:0/3597268451 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94000039c0 0x7f9400003dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 -- 172.21.15.106:0/3597268451 >> 172.21.15.106:0/3597268451 conn(0x7f94140b0c50 msgr2=0x7f94140b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:20.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 -- 172.21.15.106:0/3597268451 shutdown_connections 2023-11-24T08:09:20.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.722+0000 7f941c116640 1 -- 172.21.15.106:0/3597268451 wait complete. 2023-11-24T08:09:20.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.724+0000 7f941c116640 1 Processor -- start 2023-11-24T08:09:20.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.724+0000 7f941c116640 1 -- start start 2023-11-24T08:09:20.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f941c116640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000039c0 0x7f9414151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f941c116640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 0x7f9414145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f941c116640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f941415b960 0x7f94140c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f941c116640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941415ca70 con 0x7f94141521c0 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f941c116640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f941415bdf0 con 0x7f94000039c0 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f941c116640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f941415c0d0 con 0x7f941415b960 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f94196b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 0x7f9414145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f94196b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 0x7f9414145c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34938/0 (socket says 172.21.15.106:34938) 2023-11-24T08:09:20.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f94196b1640 1 -- 172.21.15.106:0/1445400667 learned_addr learned my addr 172.21.15.106:0/1445400667 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.725+0000 7f9419eb2640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000039c0 0x7f9414151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f941a6b3640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f941415b960 0x7f94140c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f94196b1640 1 -- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f941415b960 msgr2=0x7f94140c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f94196b1640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f941415b960 0x7f94140c8410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f94196b1640 1 -- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000039c0 msgr2=0x7f9414151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f94196b1640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000039c0 0x7f9414151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f94196b1640 1 -- 172.21.15.106:0/1445400667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941415c3b0 con 0x7f94141521c0 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f9419eb2640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000039c0 0x7f9414151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f94196b1640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 0x7f9414145c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f940400e4b0 tx=0x7f940400e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94040135f0 con 0x7f94141521c0 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9404013770 con 0x7f94141521c0 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f941415c690 con 0x7f94141521c0 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9404013b20 con 0x7f94141521c0 2023-11-24T08:09:20.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.726+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94140c8b30 con 0x7f94141521c0 2023-11-24T08:09:20.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.727+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941415cbb0 con 0x7f94141521c0 2023-11-24T08:09:20.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.728+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9404013c80 con 0x7f94141521c0 2023-11-24T08:09:20.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.729+0000 7f940a7fc640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f93f806ae70 0x7f93f806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:20.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.729+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9404089820 con 0x7f94141521c0 2023-11-24T08:09:20.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.730+0000 7f9419eb2640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f93f806ae70 0x7f93f806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:20.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.730+0000 7f9419eb2640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f93f806ae70 0x7f93f806d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9410034000 tx=0x7f9410009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:20.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.734+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f941415cbb0 con 0x7f94141521c0 2023-11-24T08:09:20.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.921+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f941415cbb0 con 0x7f94141521c0 2023-11-24T08:09:20.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.922+0000 7f940a7fc640 1 -- 172.21.15.106:0/1445400667 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f941415cbb0 con 0x7f94141521c0 2023-11-24T08:09:20.923 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.924+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f93f806ae70 msgr2=0x7f93f806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.924+0000 7f941c116640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f93f806ae70 0x7f93f806d330 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9410034000 tx=0x7f9410009030 comp rx=0 tx=0).stop 2023-11-24T08:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.924+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 msgr2=0x7f9414145c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:20.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.924+0000 7f941c116640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 0x7f9414145c10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f940400e4b0 tx=0x7f940400e970 comp rx=0 tx=0).stop 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 shutdown_connections 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f93f806ae70 0x7f93f806d330 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f941415b960 0x7f94140c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94141521c0 0x7f9414145c10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 --2- 172.21.15.106:0/1445400667 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94000039c0 0x7f9414151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 >> 172.21.15.106:0/1445400667 conn(0x7f94140b0c50 msgr2=0x7f94140c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 shutdown_connections 2023-11-24T08:09:20.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:20.925+0000 7f941c116640 1 -- 172.21.15.106:0/1445400667 wait complete. 2023-11-24T08:09:20.944 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:21.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.157+0000 7fea1ab78640 1 Processor -- start 2023-11-24T08:09:21.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.157+0000 7fea1ab78640 1 -- start start 2023-11-24T08:09:21.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.158+0000 7fea1ab78640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fea1406de10 0x7fea1415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea1ab78640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea1415ec90 con 0x7fea14072160 2023-11-24T08:09:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea1ab78640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140cd530 con 0x7fea140c49f0 2023-11-24T08:09:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea1ab78640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140c4ed0 con 0x7fea1406de10 2023-11-24T08:09:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea18914640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fea14072160 0x7fea1406d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54088/0 (socket says 172.21.15.106:54088) 2023-11-24T08:09:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea18914640 1 -- 172.21.15.106:0/3610360658 learned_addr learned my addr 172.21.15.106:0/3610360658 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea13fff640 1 --2- 172.21.15.106:0/3610360658 >> v2:172.21.15.156:3300/0 conn(0x7fea1406de10 0x7fea1415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472579402 0 0) 0x7fea08001110 con 0x7fea14072160 2023-11-24T08:09:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.159+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea04003520 con 0x7fea14072160 2023-11-24T08:09:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3786854267 0 0) 0x7fea04003520 con 0x7fea14072160 2023-11-24T08:09:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 >> v2:172.21.15.156:3300/0 conn(0x7fea1406de10 msgr2=0x7fea1415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 --2- 172.21.15.106:0/3610360658 >> v2:172.21.15.156:3300/0 conn(0x7fea1406de10 0x7fea1415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 >> v1:172.21.15.156:6789/0 conn(0x7fea140c49f0 legacy=0x7fea140705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140c51b0 con 0x7fea14072160 2023-11-24T08:09:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fea08001800 con 0x7fea14072160 2023-11-24T08:09:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 >> v1:172.21.15.106:6789/0 conn(0x7fea14072160 legacy=0x7fea1406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea04003970 0x7fea04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea04005180 0x7fea0400db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea140c4ed0 con 0x7fea040047c0 2023-11-24T08:09:21.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0400e230 con 0x7fea04005180 2023-11-24T08:09:21.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0400e3f0 con 0x7fea04003970 2023-11-24T08:09:21.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.160+0000 7fea13fff640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.161+0000 7fea13fff640 1 -- 172.21.15.106:0/3610360658 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea04003970 msgr2=0x7fea04003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.161+0000 7fea13fff640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea04003970 0x7fea04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.161+0000 7fea13fff640 1 -- 172.21.15.106:0/3610360658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea04005180 msgr2=0x7fea0400db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.161+0000 7fea13fff640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea04005180 0x7fea0400db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.161+0000 7fea13fff640 1 -- 172.21.15.106:0/3610360658 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04003520 con 0x7fea040047c0 2023-11-24T08:09:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.161+0000 7fea19115640 1 -- 172.21.15.106:0/3610360658 reap_dead start 2023-11-24T08:09:21.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.162+0000 7fea13fff640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea04004c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fea00009ac0 tx=0x7fea040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96971b10da89ebed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:21.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.162+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0002c6d0 con 0x7fea040047c0 2023-11-24T08:09:21.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.162+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0002c830 con 0x7fea040047c0 2023-11-24T08:09:21.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.162+0000 7fea137fe640 1 -- 172.21.15.106:0/3610360658 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0002cbc0 con 0x7fea040047c0 2023-11-24T08:09:21.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 -- 172.21.15.106:0/3610360658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 msgr2=0x7fea04004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea04004c40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fea00009ac0 tx=0x7fea040057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 -- 172.21.15.106:0/3610360658 shutdown_connections 2023-11-24T08:09:21.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 --2- 172.21.15.106:0/3610360658 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea04004c40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 -- 172.21.15.106:0/3610360658 >> 172.21.15.106:0/3610360658 conn(0x7fea140b0c50 msgr2=0x7fea140b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:21.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 -- 172.21.15.106:0/3610360658 shutdown_connections 2023-11-24T08:09:21.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.163+0000 7fea1ab78640 1 -- 172.21.15.106:0/3610360658 wait complete. 2023-11-24T08:09:21.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.164+0000 7fea1ab78640 1 Processor -- start 2023-11-24T08:09:21.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.165+0000 7fea1ab78640 1 -- start start 2023-11-24T08:09:21.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.165+0000 7fea1ab78640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea1415cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea1ab78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1415b970 0x7fea140b1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea1ab78640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1414ca80 0x7fea141e4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea1ab78640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea140b23a0 con 0x7fea040047c0 2023-11-24T08:09:21.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea1ab78640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1415a020 con 0x7fea1415b970 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea1ab78640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1414cf00 con 0x7fea1414ca80 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea1415cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea1415cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34956/0 (socket says 172.21.15.106:34956) 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 -- 172.21.15.106:0/1395623119 learned_addr learned my addr 172.21.15.106:0/1395623119 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea13fff640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1415b970 0x7fea140b1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea19115640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1414ca80 0x7fea141e4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 -- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1414ca80 msgr2=0x7fea141e4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1414ca80 0x7fea141e4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 -- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1415b970 msgr2=0x7fea140b1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1415b970 0x7fea140b1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea18914640 1 -- 172.21.15.106:0/1395623119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1415bf30 con 0x7fea040047c0 2023-11-24T08:09:21.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.166+0000 7fea13fff640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1415b970 0x7fea140b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:21.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.167+0000 7fea18914640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea1415cfd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c820 tx=0x7fea0800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:21.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.167+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea080074e0 con 0x7fea040047c0 2023-11-24T08:09:21.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.167+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea08007640 con 0x7fea040047c0 2023-11-24T08:09:21.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.167+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea1415c210 con 0x7fea040047c0 2023-11-24T08:09:21.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.167+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea08004a20 con 0x7fea040047c0 2023-11-24T08:09:21.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.167+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1415c7d0 con 0x7fea040047c0 2023-11-24T08:09:21.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.168+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1404e8b0 con 0x7fea040047c0 2023-11-24T08:09:21.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.169+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fea08004b80 con 0x7fea040047c0 2023-11-24T08:09:21.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.170+0000 7fea117fa640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe9f806ae40 0x7fe9f806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.170+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea080889b0 con 0x7fea040047c0 2023-11-24T08:09:21.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.170+0000 7fea13fff640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe9f806ae40 0x7fe9f806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.171+0000 7fea13fff640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe9f806ae40 0x7fe9f806d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea00036000 tx=0x7fea00030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:21.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.174+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fea08053af0 con 0x7fea040047c0 2023-11-24T08:09:21.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.365+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fea1415ec90 con 0x7fea040047c0 2023-11-24T08:09:21.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.366+0000 7fea117fa640 1 -- 172.21.15.106:0/1395623119 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fea1415ec90 con 0x7fea040047c0 2023-11-24T08:09:21.368 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:21.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe9f806ae40 msgr2=0x7fe9f806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe9f806ae40 0x7fe9f806d300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fea00036000 tx=0x7fea00030000 comp rx=0 tx=0).stop 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 msgr2=0x7fea1415cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea1415cfd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fea0800c820 tx=0x7fea0800cce0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 shutdown_connections 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe9f806ae40 0x7fe9f806d300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fea1414ca80 0x7fea141e4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fea1415b970 0x7fea140b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 --2- 172.21.15.106:0/1395623119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fea040047c0 0x7fea1415cfd0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 >> 172.21.15.106:0/1395623119 conn(0x7fea140b0c50 msgr2=0x7fea140ac110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 shutdown_connections 2023-11-24T08:09:21.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.369+0000 7fea1ab78640 1 -- 172.21.15.106:0/1395623119 wait complete. 2023-11-24T08:09:21.389 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:21.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.603+0000 7f0b5c09b640 1 Processor -- start 2023-11-24T08:09:21.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.603+0000 7f0b5c09b640 1 -- start start 2023-11-24T08:09:21.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5c09b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0b54072160 0x7f0b5415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5c09b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b540c49f0 0x7f0b5406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5c09b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5415ec90 con 0x7f0b5406dc40 2023-11-24T08:09:21.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5c09b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5406e0f0 con 0x7f0b54072160 2023-11-24T08:09:21.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5c09b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540cd530 con 0x7f0b540c49f0 2023-11-24T08:09:21.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5a638640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0b5406dc40 0x7f0b540cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54100/0 (socket says 172.21.15.106:54100) 2023-11-24T08:09:21.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b5a638640 1 -- 172.21.15.106:0/58717733 learned_addr learned my addr 172.21.15.106:0/58717733 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:21.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.605+0000 7f0b59e37640 1 --2- 172.21.15.106:0/58717733 >> v2:172.21.15.106:3300/0 conn(0x7f0b54072160 0x7f0b5415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b59636640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b540c49f0 0x7f0b5406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b58e35640 1 -- 172.21.15.106:0/58717733 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094481916 0 0) 0x7f0b4c001490 con 0x7f0b5406dc40 2023-11-24T08:09:21.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b58e35640 1 -- 172.21.15.106:0/58717733 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b40003520 con 0x7f0b5406dc40 2023-11-24T08:09:21.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b59e37640 1 -- 172.21.15.106:0/58717733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b540c49f0 msgr2=0x7f0b5406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:21.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b59e37640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b540c49f0 0x7f0b5406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b59e37640 1 -- 172.21.15.106:0/58717733 >> v1:172.21.15.106:6789/0 conn(0x7f0b5406dc40 legacy=0x7f0b540cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b59e37640 1 -- 172.21.15.106:0/58717733 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540c4ed0 con 0x7f0b54072160 2023-11-24T08:09:21.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.606+0000 7f0b59636640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b540c49f0 0x7f0b5406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:21.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.607+0000 7f0b59e37640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b54072160 0x7f0b5415e750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b500097a0 tx=0x7f0b5002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce607541040c869 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:21.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.607+0000 7f0b58e35640 1 -- 172.21.15.106:0/58717733 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5002d040 con 0x7f0b54072160 2023-11-24T08:09:21.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.607+0000 7f0b58e35640 1 -- 172.21.15.106:0/58717733 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b50031810 con 0x7f0b54072160 2023-11-24T08:09:21.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.607+0000 7f0b58e35640 1 -- 172.21.15.106:0/58717733 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50031ba0 con 0x7f0b54072160 2023-11-24T08:09:21.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 -- 172.21.15.106:0/58717733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b54072160 msgr2=0x7f0b5415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b54072160 0x7f0b5415e750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0b500097a0 tx=0x7f0b5002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 -- 172.21.15.106:0/58717733 shutdown_connections 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b540c49f0 0x7f0b5406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/58717733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b54072160 0x7f0b5415e750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 -- 172.21.15.106:0/58717733 >> 172.21.15.106:0/58717733 conn(0x7f0b540b0c50 msgr2=0x7f0b540b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 -- 172.21.15.106:0/58717733 shutdown_connections 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.608+0000 7f0b5c09b640 1 -- 172.21.15.106:0/58717733 wait complete. 2023-11-24T08:09:21.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.610+0000 7f0b5c09b640 1 Processor -- start 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.610+0000 7f0b5c09b640 1 -- start start 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b5c09b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 0x7f0b5415d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b5c09b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b54072160 0x7f0b5415aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b5c09b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b540c49f0 0x7f0b540cbd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b5c09b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b4000 con 0x7f0b5406dc40 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b5c09b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b33f0 con 0x7f0b540c49f0 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b5c09b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540b36d0 con 0x7f0b54072160 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 0x7f0b5415d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 0x7f0b5415d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34978/0 (socket says 172.21.15.106:34978) 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 -- 172.21.15.106:0/1587329187 learned_addr learned my addr 172.21.15.106:0/1587329187 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:21.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 -- 172.21.15.106:0/1587329187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b54072160 msgr2=0x7f0b5415aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b54072160 0x7f0b5415aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 -- 172.21.15.106:0/1587329187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b540c49f0 msgr2=0x7f0b540cbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.611+0000 7f0b59e37640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b540c49f0 0x7f0b540cbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b59e37640 1 -- 172.21.15.106:0/1587329187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50009400 con 0x7f0b5406dc40 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b59e37640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 0x7f0b5415d060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b500098c0 tx=0x7f0b500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50006e10 con 0x7f0b5406dc40 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5003ad60 con 0x7f0b5406dc40 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50044620 con 0x7f0b5406dc40 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b540b39b0 con 0x7f0b5406dc40 2023-11-24T08:09:21.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.612+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5415d5a0 con 0x7f0b5406dc40 2023-11-24T08:09:21.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.613+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b540c3ec0 con 0x7f0b5406dc40 2023-11-24T08:09:21.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.614+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0b5004d440 con 0x7f0b5406dc40 2023-11-24T08:09:21.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.615+0000 7f0b4a7fc640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0b3806ae40 0x7f0b3806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:21.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.615+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b500ab9c0 con 0x7f0b5406dc40 2023-11-24T08:09:21.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.615+0000 7f0b59636640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0b3806ae40 0x7f0b3806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:21.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.616+0000 7f0b59636640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0b3806ae40 0x7f0b3806d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0b44006fd0 tx=0x7f0b44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:21.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.620+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0b50076a60 con 0x7f0b5406dc40 2023-11-24T08:09:21.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.807+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0b5404e8b0 con 0x7f0b5406dc40 2023-11-24T08:09:21.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.808+0000 7f0b4a7fc640 1 -- 172.21.15.106:0/1587329187 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0b500350a0 con 0x7f0b5406dc40 2023-11-24T08:09:21.810 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0b3806ae40 msgr2=0x7f0b3806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0b3806ae40 0x7f0b3806d300 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0b44006fd0 tx=0x7f0b44008040 comp rx=0 tx=0).stop 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 msgr2=0x7f0b5415d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 0x7f0b5415d060 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0b500098c0 tx=0x7f0b500046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 shutdown_connections 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0b540c49f0 0x7f0b540cbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0b3806ae40 0x7f0b3806d300 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0b54072160 0x7f0b5415aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 --2- 172.21.15.106:0/1587329187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0b5406dc40 0x7f0b5415d060 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 >> 172.21.15.106:0/1587329187 conn(0x7f0b540b0c50 msgr2=0x7f0b540b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 shutdown_connections 2023-11-24T08:09:21.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:21.811+0000 7f0b5c09b640 1 -- 172.21.15.106:0/1587329187 wait complete. 2023-11-24T08:09:21.832 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:22.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.042+0000 7f35a5683640 1 Processor -- start 2023-11-24T08:09:22.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.042+0000 7f35a5683640 1 -- start start 2023-11-24T08:09:22.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f35a5683640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a00cf450 0x7f35a00cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f35a5683640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f35a00cdfb0 0x7f35a00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f35a5683640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35a00cece0 con 0x7f35a015c7f0 2023-11-24T08:09:22.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f35a5683640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35a0156010 con 0x7f35a00cdfb0 2023-11-24T08:09:22.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f35a5683640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a006ce30 con 0x7f35a00cf450 2023-11-24T08:09:22.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f359effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f35a015c7f0 0x7f35a006db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54110/0 (socket says 172.21.15.106:54110) 2023-11-24T08:09:22.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f359effd640 1 -- 172.21.15.106:0/2907060944 learned_addr learned my addr 172.21.15.106:0/2907060944 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:22.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f359f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f35a00cdfb0 0x7f35a00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.044+0000 7f359e7fc640 1 --2- 172.21.15.106:0/2907060944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a00cf450 0x7f35a00cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.045+0000 7f359f7fe640 1 -- 172.21.15.106:0/2907060944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a00cf450 msgr2=0x7f35a00cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:22.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.045+0000 7f359f7fe640 1 --2- 172.21.15.106:0/2907060944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a00cf450 0x7f35a00cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.045+0000 7f359f7fe640 1 -- 172.21.15.106:0/2907060944 >> v1:172.21.15.106:6789/0 conn(0x7f35a015c7f0 legacy=0x7f35a006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.045+0000 7f359f7fe640 1 -- 172.21.15.106:0/2907060944 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a006d040 con 0x7f35a00cdfb0 2023-11-24T08:09:22.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.045+0000 7f359dffb640 1 -- 172.21.15.106:0/2907060944 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3185804850 0 0) 0x7f35a00cece0 con 0x7f35a015c7f0 2023-11-24T08:09:22.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.046+0000 7f359f7fe640 1 --2- 172.21.15.106:0/2907060944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cdfb0 0x7f35a00ce440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3598009d90 tx=0x7f359802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2c0fde4ede431a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.046+0000 7f359dffb640 1 -- 172.21.15.106:0/2907060944 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359802cc50 con 0x7f35a00cdfb0 2023-11-24T08:09:22.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.046+0000 7f359dffb640 1 -- 172.21.15.106:0/2907060944 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359802cdb0 con 0x7f35a00cdfb0 2023-11-24T08:09:22.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.046+0000 7f359dffb640 1 -- 172.21.15.106:0/2907060944 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598034660 con 0x7f35a00cdfb0 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 -- 172.21.15.106:0/2907060944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cdfb0 msgr2=0x7f35a00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 --2- 172.21.15.106:0/2907060944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cdfb0 0x7f35a00ce440 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3598009d90 tx=0x7f359802bea0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 -- 172.21.15.106:0/2907060944 shutdown_connections 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 --2- 172.21.15.106:0/2907060944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cdfb0 0x7f35a00ce440 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 --2- 172.21.15.106:0/2907060944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a00cf450 0x7f35a00cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 -- 172.21.15.106:0/2907060944 >> 172.21.15.106:0/2907060944 conn(0x7f35a006a920 msgr2=0x7f35a006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 -- 172.21.15.106:0/2907060944 shutdown_connections 2023-11-24T08:09:22.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.047+0000 7f35a5683640 1 -- 172.21.15.106:0/2907060944 wait complete. 2023-11-24T08:09:22.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.048+0000 7f35a5683640 1 Processor -- start 2023-11-24T08:09:22.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.049+0000 7f35a5683640 1 -- start start 2023-11-24T08:09:22.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.049+0000 7f35a5683640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a00cdfb0 0x7f35a0150b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f35a5683640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 0x7f35a01504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f35a5683640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a015c7f0 0x7f35a00cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f35a5683640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00adf70 con 0x7f35a00cf450 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f35a5683640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00cbf30 con 0x7f35a00cdfb0 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f35a5683640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35a00ad9b0 con 0x7f35a015c7f0 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 0x7f35a01504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 0x7f35a01504e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34984/0 (socket says 172.21.15.106:34984) 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 -- 172.21.15.106:0/1402828534 learned_addr learned my addr 172.21.15.106:0/1402828534 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 -- 172.21.15.106:0/1402828534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a015c7f0 msgr2=0x7f35a00cb9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a015c7f0 0x7f35a00cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 -- 172.21.15.106:0/1402828534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a00cdfb0 msgr2=0x7f35a0150b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a00cdfb0 0x7f35a0150b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.050+0000 7f359e7fc640 1 -- 172.21.15.106:0/1402828534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009a70 con 0x7f35a00cf450 2023-11-24T08:09:22.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.051+0000 7f359e7fc640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 0x7f35a01504e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f359400d830 tx=0x7f359400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.051+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359400b6c0 con 0x7f35a00cf450 2023-11-24T08:09:22.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.051+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f359400b820 con 0x7f35a00cf450 2023-11-24T08:09:22.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.051+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a00adc90 con 0x7f35a00cf450 2023-11-24T08:09:22.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.051+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f359400bbb0 con 0x7f35a00cf450 2023-11-24T08:09:22.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.051+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a00ac3d0 con 0x7f35a00cf450 2023-11-24T08:09:22.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.052+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a0061a50 con 0x7f35a00cf450 2023-11-24T08:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.053+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f359400bd10 con 0x7f35a00cf450 2023-11-24T08:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.054+0000 7f35877fe640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f358c06ae40 0x7f358c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.054+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3594089ba0 con 0x7f35a00cf450 2023-11-24T08:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.054+0000 7f359effd640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f358c06ae40 0x7f358c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.055+0000 7f359effd640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f358c06ae40 0x7f358c06d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3590007fd0 tx=0x7f3590007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.058+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f359408e070 con 0x7f35a00cf450 2023-11-24T08:09:22.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.250+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35a0067470 con 0x7f35a00cf450 2023-11-24T08:09:22.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.253+0000 7f35877fe640 1 -- 172.21.15.106:0/1402828534 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f35940586d0 con 0x7f35a00cf450 2023-11-24T08:09:22.255 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:22.255 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:22.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.255+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f358c06ae40 msgr2=0x7f358c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.255+0000 7f35a5683640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f358c06ae40 0x7f358c06d300 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f3590007fd0 tx=0x7f3590007960 comp rx=0 tx=0).stop 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.255+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 msgr2=0x7f35a01504e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.255+0000 7f35a5683640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 0x7f35a01504e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f359400d830 tx=0x7f359400dcf0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 shutdown_connections 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f358c06ae40 0x7f358c06d300 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f35a015c7f0 0x7f35a00cb9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f35a00cf450 0x7f35a01504e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 --2- 172.21.15.106:0/1402828534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35a00cdfb0 0x7f35a0150b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 >> 172.21.15.106:0/1402828534 conn(0x7f35a006a920 msgr2=0x7f35a01554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:22.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 shutdown_connections 2023-11-24T08:09:22.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.256+0000 7f35a5683640 1 -- 172.21.15.106:0/1402828534 wait complete. 2023-11-24T08:09:22.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:22.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:22.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.534+0000 7ff0d85b0640 1 Processor -- start 2023-11-24T08:09:22.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.535+0000 7ff0d85b0640 1 -- start start 2023-11-24T08:09:22.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.536+0000 7ff0d85b0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0d0146eb0 0x7ff0d0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.536+0000 7ff0d85b0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0d00cd2e0 0x7ff0d0070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.536+0000 7ff0d85b0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0d0161a10 con 0x7ff0d006e160 2023-11-24T08:09:22.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.536+0000 7ff0d85b0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d0161cf0 con 0x7ff0d0146eb0 2023-11-24T08:09:22.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.536+0000 7ff0d85b0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0d0161fd0 con 0x7ff0d00cd2e0 2023-11-24T08:09:22.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0d0146eb0 0x7ff0d0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff0d0146eb0 0x7ff0d0161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34988/0 (socket says 172.21.15.106:34988) 2023-11-24T08:09:22.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 -- 172.21.15.106:0/2425787737 learned_addr learned my addr 172.21.15.106:0/2425787737 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:22.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 -- 172.21.15.106:0/2425787737 >> v2:172.21.15.156:3300/0 conn(0x7ff0d00cd2e0 msgr2=0x7ff0d0070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:22.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 --2- 172.21.15.106:0/2425787737 >> v2:172.21.15.156:3300/0 conn(0x7ff0d00cd2e0 0x7ff0d0070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 -- 172.21.15.106:0/2425787737 >> v1:172.21.15.156:6789/0 conn(0x7ff0d006e160 legacy=0x7ff0d00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:22.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.537+0000 7ff0d634c640 1 -- 172.21.15.106:0/2425787737 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d0070660 con 0x7ff0d0146eb0 2023-11-24T08:09:22.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.538+0000 7ff0d634c640 1 --2- 172.21.15.106:0/2425787737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d0146eb0 0x7ff0d0161150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00ae150 tx=0x7ff0cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dd91325b58d61d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d534a640 1 -- 172.21.15.106:0/2425787737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc031870 con 0x7ff0d0146eb0 2023-11-24T08:09:22.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d534a640 1 -- 172.21.15.106:0/2425787737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc0319d0 con 0x7ff0d0146eb0 2023-11-24T08:09:22.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d534a640 1 -- 172.21.15.106:0/2425787737 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc031d60 con 0x7ff0d0146eb0 2023-11-24T08:09:22.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d85b0640 1 -- 172.21.15.106:0/2425787737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d0146eb0 msgr2=0x7ff0d0161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/2425787737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d0146eb0 0x7ff0d0161150 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00ae150 tx=0x7ff0cc02b910 comp rx=0 tx=0).stop 2023-11-24T08:09:22.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d85b0640 1 -- 172.21.15.106:0/2425787737 shutdown_connections 2023-11-24T08:09:22.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/2425787737 >> v2:172.21.15.156:3300/0 conn(0x7ff0d00cd2e0 0x7ff0d0070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/2425787737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d0146eb0 0x7ff0d0161150 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.539+0000 7ff0d85b0640 1 -- 172.21.15.106:0/2425787737 >> 172.21.15.106:0/2425787737 conn(0x7ff0d00b0c50 msgr2=0x7ff0d00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:22.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.540+0000 7ff0d85b0640 1 -- 172.21.15.106:0/2425787737 shutdown_connections 2023-11-24T08:09:22.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.540+0000 7ff0d85b0640 1 -- 172.21.15.106:0/2425787737 wait complete. 2023-11-24T08:09:22.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.541+0000 7ff0d85b0640 1 Processor -- start 2023-11-24T08:09:22.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.541+0000 7ff0d85b0640 1 -- start start 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.542+0000 7ff0d85b0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 0x7ff0d0143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.542+0000 7ff0d85b0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0d00cd2e0 0x7ff0d00b26f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.542+0000 7ff0d85b0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0d0146eb0 0x7ff0d00affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.542+0000 7ff0d85b0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d015f5d0 con 0x7ff0d006e160 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.542+0000 7ff0d85b0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d015e720 con 0x7ff0d0146eb0 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.542+0000 7ff0d85b0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d015e950 con 0x7ff0d00cd2e0 2023-11-24T08:09:22.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 0x7ff0d0143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 0x7ff0d0143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35002/0 (socket says 172.21.15.106:35002) 2023-11-24T08:09:22.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 -- 172.21.15.106:0/1224547228 learned_addr learned my addr 172.21.15.106:0/1224547228 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 -- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0d00cd2e0 msgr2=0x7ff0d00b26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0d00cd2e0 0x7ff0d00b26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d6b4d640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0d0146eb0 0x7ff0d00affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 -- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0d0146eb0 msgr2=0x7ff0d00affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0d0146eb0 0x7ff0d00affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 -- 172.21.15.106:0/1224547228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0094d0 con 0x7ff0d006e160 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.543+0000 7ff0d634c640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 0x7ff0d0143da0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc000c00 tx=0x7ff0cc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.544+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc03ad10 con 0x7ff0d006e160 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.544+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0cc03ae70 con 0x7ff0d006e160 2023-11-24T08:09:22.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.544+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc044610 con 0x7ff0d006e160 2023-11-24T08:09:22.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.544+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d015ec30 con 0x7ff0d006e160 2023-11-24T08:09:22.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.544+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d015f1f0 con 0x7ff0d006e160 2023-11-24T08:09:22.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.545+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d004e8b0 con 0x7ff0d006e160 2023-11-24T08:09:22.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.546+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0cc04d450 con 0x7ff0d006e160 2023-11-24T08:09:22.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.547+0000 7ff0c6ffd640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b406ae70 0x7ff0b406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.547+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0cc02d040 con 0x7ff0d006e160 2023-11-24T08:09:22.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.547+0000 7ff0d5b4b640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b406ae70 0x7ff0b406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.548+0000 7ff0d5b4b640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b406ae70 0x7ff0b406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00cbad0 tx=0x7ff0c0007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.555+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0cc076d50 con 0x7ff0d006e160 2023-11-24T08:09:22.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.745+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0d00cc170 con 0x7ff0d006e160 2023-11-24T08:09:22.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.745+0000 7ff0c6ffd640 1 -- 172.21.15.106:0/1224547228 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0cc07a370 con 0x7ff0d006e160 2023-11-24T08:09:22.747 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b406ae70 msgr2=0x7ff0b406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b406ae70 0x7ff0b406d330 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00cbad0 tx=0x7ff0c0007990 comp rx=0 tx=0).stop 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 msgr2=0x7ff0d0143da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 0x7ff0d0143da0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc000c00 tx=0x7ff0cc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 shutdown_connections 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0d0146eb0 0x7ff0d00affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b406ae70 0x7ff0b406d330 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0d00cd2e0 0x7ff0d00b26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 --2- 172.21.15.106:0/1224547228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0d006e160 0x7ff0d0143da0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 >> 172.21.15.106:0/1224547228 conn(0x7ff0d00b0c50 msgr2=0x7ff0d0146090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:22.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 shutdown_connections 2023-11-24T08:09:22.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.748+0000 7ff0d85b0640 1 -- 172.21.15.106:0/1224547228 wait complete. 2023-11-24T08:09:22.775 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:22.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.967+0000 7f3137890640 1 Processor -- start 2023-11-24T08:09:22.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.967+0000 7f3137890640 1 -- start start 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.968+0000 7f3137890640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f31300b2400 0x7f313006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.968+0000 7f3137890640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31301612f0 con 0x7f31300cfb90 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.968+0000 7f3137890640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3130162170 con 0x7f31300cc2b0 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.968+0000 7f3137890640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31300cff70 con 0x7f31300b2400 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.969+0000 7f3134e2b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f31300b2400 0x7f313006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.969+0000 7f313562c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f31300cfb90 0x7f3130161a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54126/0 (socket says 172.21.15.106:54126) 2023-11-24T08:09:22.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.969+0000 7f313562c640 1 -- 172.21.15.106:0/1942489789 learned_addr learned my addr 172.21.15.106:0/1942489789 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:22.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.969+0000 7f3134e2b640 1 -- 172.21.15.106:0/1942489789 >> v1:172.21.15.156:6789/0 conn(0x7f31300cc2b0 legacy=0x7f31300cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:22.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.969+0000 7f3134e2b640 1 -- 172.21.15.106:0/1942489789 >> v1:172.21.15.106:6789/0 conn(0x7f31300cfb90 legacy=0x7f3130161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.969+0000 7f3134e2b640 1 -- 172.21.15.106:0/1942489789 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313006a850 con 0x7f31300b2400 2023-11-24T08:09:22.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.970+0000 7f3134e2b640 1 --2- 172.21.15.106:0/1942489789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f313006ee50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f31200097a0 tx=0x7f312002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cad4c79183bbdb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.970+0000 7f3127fff640 1 -- 172.21.15.106:0/1942489789 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120030710 con 0x7f31300b2400 2023-11-24T08:09:22.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.970+0000 7f3127fff640 1 -- 172.21.15.106:0/1942489789 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312002e050 con 0x7f31300b2400 2023-11-24T08:09:22.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.970+0000 7f3127fff640 1 -- 172.21.15.106:0/1942489789 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3120030a90 con 0x7f31300b2400 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 -- 172.21.15.106:0/1942489789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 msgr2=0x7f313006ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 --2- 172.21.15.106:0/1942489789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f313006ee50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f31200097a0 tx=0x7f312002b790 comp rx=0 tx=0).stop 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 -- 172.21.15.106:0/1942489789 shutdown_connections 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 --2- 172.21.15.106:0/1942489789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f313006ee50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 -- 172.21.15.106:0/1942489789 >> 172.21.15.106:0/1942489789 conn(0x7f31300999d0 msgr2=0x7f31300b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 -- 172.21.15.106:0/1942489789 shutdown_connections 2023-11-24T08:09:22.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.971+0000 7f3137890640 1 -- 172.21.15.106:0/1942489789 wait complete. 2023-11-24T08:09:22.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.973+0000 7f3137890640 1 Processor -- start 2023-11-24T08:09:22.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.973+0000 7f3137890640 1 -- start start 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f3137890640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f31300b5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f3137890640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31300cc2b0 0x7f31300c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f3137890640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31300cfb90 0x7f3130159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f3137890640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300afb20 con 0x7f31300b2400 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f3137890640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c3cf0 con 0x7f31300cc2b0 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f3137890640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31300c9850 con 0x7f31300cfb90 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f313562c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f31300b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f313562c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f31300b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35022/0 (socket says 172.21.15.106:35022) 2023-11-24T08:09:22.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.974+0000 7f313562c640 1 -- 172.21.15.106:0/2886808974 learned_addr learned my addr 172.21.15.106:0/2886808974 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:22.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f313562c640 1 -- 172.21.15.106:0/2886808974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31300cfb90 msgr2=0x7f3130159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f313562c640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31300cfb90 0x7f3130159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f313562c640 1 -- 172.21.15.106:0/2886808974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31300cc2b0 msgr2=0x7f31300c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f313562c640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31300cc2b0 0x7f31300c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f313562c640 1 -- 172.21.15.106:0/2886808974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3120009400 con 0x7f31300b2400 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f313562c640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f31300b5530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c5f0 tx=0x7f312c00cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c0074e0 con 0x7f31300b2400 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c007640 con 0x7f31300b2400 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c004a00 con 0x7f31300b2400 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.975+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31300c9b30 con 0x7f31300b2400 2023-11-24T08:09:22.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.976+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31300c4040 con 0x7f31300b2400 2023-11-24T08:09:22.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.977+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313004e8b0 con 0x7f31300b2400 2023-11-24T08:09:22.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.978+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f312c01b360 con 0x7f31300b2400 2023-11-24T08:09:22.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.979+0000 7f3125ffb640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f311406ae70 0x7f311406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:22.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.979+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f312c088bb0 con 0x7f31300b2400 2023-11-24T08:09:22.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.980+0000 7f3134e2b640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f311406ae70 0x7f311406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:22.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.980+0000 7f3134e2b640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f311406ae70 0x7f311406d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3120034000 tx=0x7f312002d000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:22.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:22.982+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f312c053cd0 con 0x7f31300b2400 2023-11-24T08:09:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.173+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f31300b2bc0 con 0x7f31300b2400 2023-11-24T08:09:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.173+0000 7f3125ffb640 1 -- 172.21.15.106:0/2886808974 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f312c0180a0 con 0x7f31300b2400 2023-11-24T08:09:23.175 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f311406ae70 msgr2=0x7f311406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f311406ae70 0x7f311406d330 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3120034000 tx=0x7f312002d000 comp rx=0 tx=0).stop 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 msgr2=0x7f31300b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f31300b5530 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f312c00c5f0 tx=0x7f312c00cab0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 shutdown_connections 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f311406ae70 0x7f311406d330 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f31300cfb90 0x7f3130159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f31300cc2b0 0x7f31300c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 --2- 172.21.15.106:0/2886808974 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f31300b2400 0x7f31300b5530 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 >> 172.21.15.106:0/2886808974 conn(0x7f31300999d0 msgr2=0x7f31300b1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:23.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 shutdown_connections 2023-11-24T08:09:23.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.176+0000 7f3137890640 1 -- 172.21.15.106:0/2886808974 wait complete. 2023-11-24T08:09:23.196 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:23.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.385+0000 7fe13dd4c640 1 Processor -- start 2023-11-24T08:09:23.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.385+0000 7fe13dd4c640 1 -- start start 2023-11-24T08:09:23.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe13dd4c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe13806bc20 0x7fe1381651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe13dd4c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13815a8c0 con 0x7fe13806e1f0 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe13dd4c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13815ab10 con 0x7fe138151200 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe13dd4c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe13815adf0 con 0x7fe13806bc20 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe137fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe13806bc20 0x7fe1381651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe136ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe13806e1f0 0x7fe13806b510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54138/0 (socket says 172.21.15.106:54138) 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.387+0000 7fe136ffd640 1 -- 172.21.15.106:0/931398762 learned_addr learned my addr 172.21.15.106:0/931398762 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.388+0000 7fe137fff640 1 -- 172.21.15.106:0/931398762 >> v1:172.21.15.156:6789/0 conn(0x7fe138151200 legacy=0x7fe1380cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.388+0000 7fe137fff640 1 -- 172.21.15.106:0/931398762 >> v1:172.21.15.106:6789/0 conn(0x7fe13806e1f0 legacy=0x7fe13806b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.388+0000 7fe137fff640 1 -- 172.21.15.106:0/931398762 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe13815b0d0 con 0x7fe13806bc20 2023-11-24T08:09:23.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.388+0000 7fe1367fc640 1 -- 172.21.15.106:0/931398762 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 146030327 0 0) 0x7fe13815a8c0 con 0x7fe13806e1f0 2023-11-24T08:09:23.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.389+0000 7fe137fff640 1 --2- 172.21.15.106:0/931398762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13806bc20 0x7fe1381651d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe130009d90 tx=0x7fe13002dfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb02275e28f25d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:23.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.390+0000 7fe1367fc640 1 -- 172.21.15.106:0/931398762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13002ed50 con 0x7fe13806bc20 2023-11-24T08:09:23.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.390+0000 7fe1367fc640 1 -- 172.21.15.106:0/931398762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe130036440 con 0x7fe13806bc20 2023-11-24T08:09:23.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.390+0000 7fe1367fc640 1 -- 172.21.15.106:0/931398762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1300367d0 con 0x7fe13806bc20 2023-11-24T08:09:23.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 -- 172.21.15.106:0/931398762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13806bc20 msgr2=0x7fe1381651d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/931398762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13806bc20 0x7fe1381651d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe130009d90 tx=0x7fe13002dfa0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 -- 172.21.15.106:0/931398762 shutdown_connections 2023-11-24T08:09:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/931398762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe13806bc20 0x7fe1381651d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 -- 172.21.15.106:0/931398762 >> 172.21.15.106:0/931398762 conn(0x7fe138070fa0 msgr2=0x7fe1380713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 -- 172.21.15.106:0/931398762 shutdown_connections 2023-11-24T08:09:23.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.391+0000 7fe13dd4c640 1 -- 172.21.15.106:0/931398762 wait complete. 2023-11-24T08:09:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.393+0000 7fe13dd4c640 1 Processor -- start 2023-11-24T08:09:23.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.393+0000 7fe13dd4c640 1 -- start start 2023-11-24T08:09:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe13dd4c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe13806bc20 0x7fe1381f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe13dd4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13806e1f0 0x7fe13806d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe13dd4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 0x7fe13806dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe13dd4c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1381f3500 con 0x7fe138151200 2023-11-24T08:09:23.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe13dd4c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138151a70 con 0x7fe13806e1f0 2023-11-24T08:09:23.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe13dd4c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe138151d50 con 0x7fe13806bc20 2023-11-24T08:09:23.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 0x7fe13806dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 0x7fe13806dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35048/0 (socket says 172.21.15.106:35048) 2023-11-24T08:09:23.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 -- 172.21.15.106:0/1879130785 learned_addr learned my addr 172.21.15.106:0/1879130785 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:23.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe136ffd640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13806e1f0 0x7fe13806d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 -- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe13806bc20 msgr2=0x7fe1381f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe1377fe640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe13806bc20 0x7fe1381f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe13806bc20 0x7fe1381f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 -- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13806e1f0 msgr2=0x7fe13806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13806e1f0 0x7fe13806d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe137fff640 1 -- 172.21.15.106:0/1879130785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130009a70 con 0x7fe138151200 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe1377fe640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe13806bc20 0x7fe1381f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.394+0000 7fe136ffd640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13806e1f0 0x7fe13806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.395+0000 7fe137fff640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 0x7fe13806dba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe130009eb0 tx=0x7fe130004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.395+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130040640 con 0x7fe138151200 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.395+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1300407a0 con 0x7fe138151200 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.395+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe130040b30 con 0x7fe138151200 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.395+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe138152030 con 0x7fe138151200 2023-11-24T08:09:23.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.395+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1381525f0 con 0x7fe138151200 2023-11-24T08:09:23.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.397+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe138061130 con 0x7fe138151200 2023-11-24T08:09:23.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.397+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe130040c90 con 0x7fe138151200 2023-11-24T08:09:23.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.398+0000 7fe11ffff640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe12406ae40 0x7fe12406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.398+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1300ac100 con 0x7fe138151200 2023-11-24T08:09:23.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.398+0000 7fe1377fe640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe12406ae40 0x7fe12406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.399+0000 7fe1377fe640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe12406ae40 0x7fe12406d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe138151a00 tx=0x7fe128008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:23.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.402+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1300772c0 con 0x7fe138151200 2023-11-24T08:09:23.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.586+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe13804e8b0 con 0x7fe138151200 2023-11-24T08:09:23.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.587+0000 7fe11ffff640 1 -- 172.21.15.106:0/1879130785 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe13007a8e0 con 0x7fe138151200 2023-11-24T08:09:23.588 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe12406ae40 msgr2=0x7fe12406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe12406ae40 0x7fe12406d300 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fe138151a00 tx=0x7fe128008040 comp rx=0 tx=0).stop 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 msgr2=0x7fe13806dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 0x7fe13806dba0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe130009eb0 tx=0x7fe130004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 shutdown_connections 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe138151200 0x7fe13806dba0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13806e1f0 0x7fe13806d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe12406ae40 0x7fe12406d300 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 --2- 172.21.15.106:0/1879130785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe13806bc20 0x7fe1381f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.589+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 >> 172.21.15.106:0/1879130785 conn(0x7fe138070fa0 msgr2=0x7fe1380ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:23.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.590+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 shutdown_connections 2023-11-24T08:09:23.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.590+0000 7fe13dd4c640 1 -- 172.21.15.106:0/1879130785 wait complete. 2023-11-24T08:09:23.608 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:23.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.795+0000 7f4b59b01640 1 Processor -- start 2023-11-24T08:09:23.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.795+0000 7f4b59b01640 1 -- start start 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.796+0000 7f4b59b01640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540d35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.796+0000 7f4b59b01640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b540ce010 0x7f4b540ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.796+0000 7f4b59b01640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b540d1070 con 0x7f4b541609e0 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.796+0000 7f4b59b01640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b540cedc0 con 0x7f4b540ce010 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.796+0000 7f4b59b01640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540cf0a0 con 0x7f4b540d31a0 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.796+0000 7f4b53fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b540ce010 0x7f4b540ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b53fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b540ce010 0x7f4b540ce480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35060/0 (socket says 172.21.15.106:35060) 2023-11-24T08:09:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b53fff640 1 -- 172.21.15.106:0/1757145885 learned_addr learned my addr 172.21.15.106:0/1757145885 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b4bfff640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540d35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b53fff640 1 -- 172.21.15.106:0/1757145885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 msgr2=0x7f4b540d35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b53fff640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540d35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b53fff640 1 -- 172.21.15.106:0/1757145885 >> v1:172.21.15.156:6789/0 conn(0x7f4b541609e0 legacy=0x7f4b5406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b53fff640 1 -- 172.21.15.106:0/1757145885 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b540d1310 con 0x7f4b540ce010 2023-11-24T08:09:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.797+0000 7f4b4bfff640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:23.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.798+0000 7f4b53fff640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b540ce010 0x7f4b540ce480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b540d2b10 tx=0x7f4b4c02be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe6ba84ea242eaf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.798+0000 7f4b52ffd640 1 -- 172.21.15.106:0/1757145885 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c02ca30 con 0x7f4b540ce010 2023-11-24T08:09:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b52ffd640 1 -- 172.21.15.106:0/1757145885 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c02cb90 con 0x7f4b540ce010 2023-11-24T08:09:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b52ffd640 1 -- 172.21.15.106:0/1757145885 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c0343f0 con 0x7f4b540ce010 2023-11-24T08:09:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 -- 172.21.15.106:0/1757145885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b540ce010 msgr2=0x7f4b540ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b540ce010 0x7f4b540ce480 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4b540d2b10 tx=0x7f4b4c02be90 comp rx=0 tx=0).stop 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 -- 172.21.15.106:0/1757145885 shutdown_connections 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b540ce010 0x7f4b540ce480 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 --2- 172.21.15.106:0/1757145885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540d35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 -- 172.21.15.106:0/1757145885 >> 172.21.15.106:0/1757145885 conn(0x7f4b5408af60 msgr2=0x7f4b5409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 -- 172.21.15.106:0/1757145885 shutdown_connections 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.799+0000 7f4b59b01640 1 -- 172.21.15.106:0/1757145885 wait complete. 2023-11-24T08:09:23.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.801+0000 7f4b59b01640 1 Processor -- start 2023-11-24T08:09:23.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.801+0000 7f4b59b01640 1 -- start start 2023-11-24T08:09:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.802+0000 7f4b59b01640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b540ce010 0x7f4b540b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.802+0000 7f4b59b01640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540b8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.802+0000 7f4b59b01640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 0x7f4b5414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.802+0000 7f4b59b01640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b5415ae60 con 0x7f4b541609e0 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.802+0000 7f4b59b01640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540b3360 con 0x7f4b540ce010 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.802+0000 7f4b59b01640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b540b8eb0 con 0x7f4b540d31a0 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 0x7f4b5414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 0x7f4b5414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35076/0 (socket says 172.21.15.106:35076) 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 -- 172.21.15.106:0/3337058209 learned_addr learned my addr 172.21.15.106:0/3337058209 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b4bfff640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540b8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 -- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 msgr2=0x7f4b540b8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b537fe640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b540ce010 0x7f4b540b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540b8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 -- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b540ce010 msgr2=0x7f4b540b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b540ce010 0x7f4b540b2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 -- 172.21.15.106:0/3337058209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4c009b30 con 0x7f4b541609e0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.803+0000 7f4b53fff640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 0x7f4b5414d8b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c036040 tx=0x7f4b4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.804+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c03e450 con 0x7f4b541609e0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.804+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4c03e5d0 con 0x7f4b541609e0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.804+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b540b78d0 con 0x7f4b541609e0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.804+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c03e980 con 0x7f4b541609e0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.804+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b540b7e90 con 0x7f4b541609e0 2023-11-24T08:09:23.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.805+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b540b2890 con 0x7f4b541609e0 2023-11-24T08:09:23.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.806+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b4c03eae0 con 0x7f4b541609e0 2023-11-24T08:09:23.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.807+0000 7f4b50ff9640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae40 0x7f4b3406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:23.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.807+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b4c0ac620 con 0x7f4b541609e0 2023-11-24T08:09:23.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.807+0000 7f4b537fe640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae40 0x7f4b3406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:23.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.807+0000 7f4b537fe640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae40 0x7f4b3406d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400a750 tx=0x7f4b44008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:23.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.811+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b4c077650 con 0x7f4b541609e0 2023-11-24T08:09:23.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.990+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b54067470 con 0x7f4b541609e0 2023-11-24T08:09:23.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.993+0000 7f4b50ff9640 1 -- 172.21.15.106:0/3337058209 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f4b4c07ac70 con 0x7f4b541609e0 2023-11-24T08:09:23.995 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:23.995 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:23.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae40 msgr2=0x7f4b3406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae40 0x7f4b3406d300 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f4b4400a750 tx=0x7f4b44008040 comp rx=0 tx=0).stop 2023-11-24T08:09:23.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 msgr2=0x7f4b5414d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:23.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 0x7f4b5414d8b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c036040 tx=0x7f4b4c004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:23.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 shutdown_connections 2023-11-24T08:09:23.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b541609e0 0x7f4b5414d8b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae40 0x7f4b3406d300 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b540d31a0 0x7f4b540b8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 --2- 172.21.15.106:0/3337058209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b540ce010 0x7f4b540b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:23.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 >> 172.21.15.106:0/3337058209 conn(0x7f4b5408af60 msgr2=0x7f4b540b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:23.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 shutdown_connections 2023-11-24T08:09:23.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:23.996+0000 7f4b59b01640 1 -- 172.21.15.106:0/3337058209 wait complete. 2023-11-24T08:09:24.014 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:24.015 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:24.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.213+0000 7f4a074bb640 1 Processor -- start 2023-11-24T08:09:24.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.213+0000 7f4a074bb640 1 -- start start 2023-11-24T08:09:24.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.214+0000 7f4a074bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 0x7f4a0015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.214+0000 7f4a074bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a0015ec90 con 0x7f4a000c49f0 2023-11-24T08:09:24.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.214+0000 7f4a074bb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000cd530 con 0x7f4a0006de10 2023-11-24T08:09:24.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.214+0000 7f4a074bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c4ed0 con 0x7f4a00072160 2023-11-24T08:09:24.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f4a04a56640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4a000c49f0 0x7f4a0006d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51666/0 (socket says 172.21.15.106:51666) 2023-11-24T08:09:24.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f4a04a56640 1 -- 172.21.15.106:0/954513494 learned_addr learned my addr 172.21.15.106:0/954513494 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506979226 0 0) 0x7f49f0001110 con 0x7f4a000c49f0 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ec005730 con 0x7f4a000c49f0 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f4a05257640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 0x7f4a0015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2293134302 0 0) 0x7f49ec005730 con 0x7f4a000c49f0 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 msgr2=0x7f4a0015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 0x7f4a0015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 >> v1:172.21.15.156:6789/0 conn(0x7f4a0006de10 legacy=0x7f4a000705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.215+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000c51b0 con 0x7f4a000c49f0 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f49f0001800 con 0x7f4a000c49f0 2023-11-24T08:09:24.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 >> v1:172.21.15.106:6789/0 conn(0x7f4a000c49f0 legacy=0x7f4a0006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 0x7f49ec005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec006130 0x7f49ec0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ec006f10 0x7f49ec007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c4ed0 con 0x7f49ec006130 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec00dc70 con 0x7f49ec006f10 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec00de30 con 0x7f4a00072160 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a04a56640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec006130 0x7f49ec0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a05257640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 0x7f49ec005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a04a56640 1 -- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 msgr2=0x7f49ec005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a04a56640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a00072160 0x7f49ec005bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a04a56640 1 -- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ec006f10 msgr2=0x7f49ec007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a05a58640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ec006f10 0x7f49ec007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a04a56640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ec006f10 0x7f49ec007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.216+0000 7f4a04a56640 1 -- 172.21.15.106:0/954513494 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec005730 con 0x7f49ec006130 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.217+0000 7f4a04a56640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec006130 0x7f49ec0069d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49f000c190 tx=0x7f49ec007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd370d63fdc76de5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0005ad0 con 0x7f49ec006130 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f0005c50 con 0x7f49ec006130 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f49f7fff640 1 -- 172.21.15.106:0/954513494 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f002c7e0 con 0x7f49ec006130 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a074bb640 1 -- 172.21.15.106:0/954513494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec006130 msgr2=0x7f49ec0069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a074bb640 1 --2- 172.21.15.106:0/954513494 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ec006130 0x7f49ec0069d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f49f000c190 tx=0x7f49ec007a00 comp rx=0 tx=0).stop 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a05a58640 1 -- 172.21.15.106:0/954513494 reap_dead start 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a074bb640 1 -- 172.21.15.106:0/954513494 shutdown_connections 2023-11-24T08:09:24.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a074bb640 1 -- 172.21.15.106:0/954513494 >> 172.21.15.106:0/954513494 conn(0x7f4a000b0c50 msgr2=0x7f4a000b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:24.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a074bb640 1 -- 172.21.15.106:0/954513494 shutdown_connections 2023-11-24T08:09:24.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.218+0000 7f4a074bb640 1 -- 172.21.15.106:0/954513494 wait complete. 2023-11-24T08:09:24.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.220+0000 7f4a074bb640 1 Processor -- start 2023-11-24T08:09:24.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.220+0000 7f4a074bb640 1 -- start start 2023-11-24T08:09:24.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a074bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ec006130 0x7f4a001f7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a074bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 0x7f4a0015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a074bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a0006d4b0 0x7f4a001588c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a074bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000b4540 con 0x7f4a0009c570 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a074bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001f7c50 con 0x7f4a0006d4b0 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a074bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001f6cc0 con 0x7f49ec006130 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a04a56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 0x7f4a0015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a04a56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 0x7f4a0015c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60302/0 (socket says 172.21.15.106:60302) 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.221+0000 7f4a04a56640 1 -- 172.21.15.106:0/3891832727 learned_addr learned my addr 172.21.15.106:0/3891832727 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a04a56640 1 -- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ec006130 msgr2=0x7f4a001f7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a05257640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ec006130 0x7f4a001f7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a04a56640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ec006130 0x7f4a001f7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a04a56640 1 -- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a0006d4b0 msgr2=0x7f4a001588c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a04a56640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a0006d4b0 0x7f4a001588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a04a56640 1 -- 172.21.15.106:0/3891832727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a0015cc90 con 0x7f4a0009c570 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a04a56640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 0x7f4a0015c750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49f002c380 tx=0x7f4a000cd6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0039070 con 0x7f4a0009c570 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f002cbf0 con 0x7f4a0009c570 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f003c040 con 0x7f4a0009c570 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0015cf60 con 0x7f4a0009c570 2023-11-24T08:09:24.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.222+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a001f6f10 con 0x7f4a0009c570 2023-11-24T08:09:24.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.223+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0004e8b0 con 0x7f4a0009c570 2023-11-24T08:09:24.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.224+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49f0032070 con 0x7f4a0009c570 2023-11-24T08:09:24.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.225+0000 7f49f5ffb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49e406ae40 0x7f49e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.225+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49f00aa700 con 0x7f4a0009c570 2023-11-24T08:09:24.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.225+0000 7f4a05257640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49e406ae40 0x7f49e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.226+0000 7f4a05257640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49e406ae40 0x7f49e406d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a000ce480 tx=0x7f49fc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:24.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.229+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49f0075840 con 0x7f4a0009c570 2023-11-24T08:09:24.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.404+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4a0015ec90 con 0x7f4a0009c570 2023-11-24T08:09:24.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.405+0000 7f49f5ffb640 1 -- 172.21.15.106:0/3891832727 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4a0015ec90 con 0x7f4a0009c570 2023-11-24T08:09:24.406 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:24.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.407+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49e406ae40 msgr2=0x7f49e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.407+0000 7f4a074bb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49e406ae40 0x7f49e406d300 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f4a000ce480 tx=0x7f49fc008040 comp rx=0 tx=0).stop 2023-11-24T08:09:24.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.407+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 msgr2=0x7f4a0015c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.407+0000 7f4a074bb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 0x7f4a0015c750 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f49f002c380 tx=0x7f4a000cd6c0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 shutdown_connections 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a0006d4b0 0x7f4a001588c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a0009c570 0x7f4a0015c750 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49e406ae40 0x7f49e406d300 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 --2- 172.21.15.106:0/3891832727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ec006130 0x7f4a001f7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 >> 172.21.15.106:0/3891832727 conn(0x7f4a000b0c50 msgr2=0x7f4a000cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 shutdown_connections 2023-11-24T08:09:24.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.408+0000 7f4a074bb640 1 -- 172.21.15.106:0/3891832727 wait complete. 2023-11-24T08:09:24.427 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:24.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.635+0000 7fcbb4400640 1 Processor -- start 2023-11-24T08:09:24.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.635+0000 7fcbb4400640 1 -- start start 2023-11-24T08:09:24.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.637+0000 7fcbb4400640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcbac15d270 0x7fcbac15d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.637+0000 7fcbb4400640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac15db90 con 0x7fcbac0cf220 2023-11-24T08:09:24.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.637+0000 7fcbb4400640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbac161ce0 con 0x7fcbac161260 2023-11-24T08:09:24.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.637+0000 7fcbb4400640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbac161fc0 con 0x7fcbac15d270 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.637+0000 7fcbb299d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcbac0cf220 0x7fcbac0d19b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51670/0 (socket says 172.21.15.106:51670) 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.637+0000 7fcbb299d640 1 -- 172.21.15.106:0/1860364166 learned_addr learned my addr 172.21.15.106:0/1860364166 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1860364166 >> v2:172.21.15.156:3300/0 conn(0x7fcbac15d270 0x7fcbac15d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532773475 0 0) 0x7fcbac15db90 con 0x7fcbac0cf220 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb98003520 con 0x7fcbac0cf220 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2052481478 0 0) 0x7fcb98003520 con 0x7fcbac0cf220 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 >> v2:172.21.15.156:3300/0 conn(0x7fcbac15d270 msgr2=0x7fcbac15d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 --2- 172.21.15.106:0/1860364166 >> v2:172.21.15.156:3300/0 conn(0x7fcbac15d270 0x7fcbac15d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 >> v1:172.21.15.156:6789/0 conn(0x7fcbac161260 legacy=0x7fcbac0ceb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0ce310 con 0x7fcbac0cf220 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fcba40019f0 con 0x7fcbac0cf220 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 >> v1:172.21.15.106:6789/0 conn(0x7fcbac0cf220 legacy=0x7fcbac0d19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980039c0 0x7fcb98003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb980047c0 0x7fcb98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb98005180 0x7fcb9800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac161fc0 con 0x7fcb980039c0 2023-11-24T08:09:24.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9800c050 con 0x7fcb98005180 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9800c210 con 0x7fcb980047c0 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.638+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980039c0 0x7fcb98003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.639+0000 7fcbb219c640 1 -- 172.21.15.106:0/1860364166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb980047c0 msgr2=0x7fcb98004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.639+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb980047c0 0x7fcb98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.639+0000 7fcbb219c640 1 -- 172.21.15.106:0/1860364166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb98005180 msgr2=0x7fcb9800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.639+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb98005180 0x7fcb9800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.639+0000 7fcbb219c640 1 -- 172.21.15.106:0/1860364166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb98003520 con 0x7fcb980039c0 2023-11-24T08:09:24.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.639+0000 7fcbb299d640 1 -- 172.21.15.106:0/1860364166 reap_dead start 2023-11-24T08:09:24.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.640+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980039c0 0x7fcb98003dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ac0 tx=0x7fcb980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a35189f1f30111 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.640+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba802c6d0 con 0x7fcb980039c0 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.640+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba802c830 con 0x7fcb980039c0 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.640+0000 7fcbb119a640 1 -- 172.21.15.106:0/1860364166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba802cbc0 con 0x7fcb980039c0 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 -- 172.21.15.106:0/1860364166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980039c0 msgr2=0x7fcb98003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980039c0 0x7fcb98003dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ac0 tx=0x7fcb980057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 -- 172.21.15.106:0/1860364166 shutdown_connections 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1860364166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb980039c0 0x7fcb98003dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 -- 172.21.15.106:0/1860364166 >> 172.21.15.106:0/1860364166 conn(0x7fcbac08af60 msgr2=0x7fcbac09c760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 -- 172.21.15.106:0/1860364166 shutdown_connections 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.641+0000 7fcbb4400640 1 -- 172.21.15.106:0/1860364166 wait complete. 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.643+0000 7fcbb4400640 1 Processor -- start 2023-11-24T08:09:24.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.643+0000 7fcbb4400640 1 -- start start 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.643+0000 7fcbb4400640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb980039c0 0x7fcbac1f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb4400640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0b0970 0x7fcbac15e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb4400640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 0x7fcbac160200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb4400640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b19e0 con 0x7fcbac15ea50 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb4400640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b0e60 con 0x7fcb980039c0 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb4400640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbac0b1140 con 0x7fcbac0b0970 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 0x7fcbac160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 0x7fcbac160200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60330/0 (socket says 172.21.15.106:60330) 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 -- 172.21.15.106:0/1634851437 learned_addr learned my addr 172.21.15.106:0/1634851437 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb980039c0 0x7fcbac1f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 -- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0b0970 msgr2=0x7fcbac15e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0b0970 0x7fcbac15e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 -- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb980039c0 msgr2=0x7fcbac1f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb980039c0 0x7fcbac1f07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb299d640 1 -- 172.21.15.106:0/1634851437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbac0b1420 con 0x7fcbac15ea50 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.644+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb980039c0 0x7fcbac1f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.645+0000 7fcbb299d640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 0x7fcbac160200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0d2e00 tx=0x7fcba400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.645+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400a610 con 0x7fcbac15ea50 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.645+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba400a770 con 0x7fcbac15ea50 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.645+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbac0b1700 con 0x7fcbac15ea50 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.645+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400ab20 con 0x7fcbac15ea50 2023-11-24T08:09:24.649 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.645+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbac160920 con 0x7fcbac15ea50 2023-11-24T08:09:24.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.646+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbac0cdfc0 con 0x7fcbac15ea50 2023-11-24T08:09:24.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.647+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fcba400ac80 con 0x7fcbac15ea50 2023-11-24T08:09:24.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.648+0000 7fcb9effd640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb9006ae70 0x7fcb9006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:24.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.648+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcba40892c0 con 0x7fcbac15ea50 2023-11-24T08:09:24.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.649+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb9006ae70 0x7fcb9006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:24.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.649+0000 7fcbb219c640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb9006ae70 0x7fcb9006d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcba8036000 tx=0x7fcba8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:24.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.652+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcba40542d0 con 0x7fcbac15ea50 2023-11-24T08:09:24.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.826+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbac160b40 con 0x7fcbac15ea50 2023-11-24T08:09:24.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.827+0000 7fcb9effd640 1 -- 172.21.15.106:0/1634851437 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcba40578f0 con 0x7fcbac15ea50 2023-11-24T08:09:24.829 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:24.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.829+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb9006ae70 msgr2=0x7fcb9006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.829+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb9006ae70 0x7fcb9006d330 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fcba8036000 tx=0x7fcba8030000 comp rx=0 tx=0).stop 2023-11-24T08:09:24.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.829+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 msgr2=0x7fcbac160200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:24.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.829+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 0x7fcbac160200 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fcbac0d2e00 tx=0x7fcba400cc20 comp rx=0 tx=0).stop 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 shutdown_connections 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcbac15ea50 0x7fcbac160200 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb9006ae70 0x7fcb9006d330 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcbac0b0970 0x7fcbac15e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 --2- 172.21.15.106:0/1634851437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb980039c0 0x7fcbac1f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 >> 172.21.15.106:0/1634851437 conn(0x7fcbac08af60 msgr2=0x7fcbac0b3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 shutdown_connections 2023-11-24T08:09:24.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:24.830+0000 7fcbb4400640 1 -- 172.21.15.106:0/1634851437 wait complete. 2023-11-24T08:09:24.848 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:25.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.055+0000 7f2b32e75640 1 Processor -- start 2023-11-24T08:09:25.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.055+0000 7f2b32e75640 1 -- start start 2023-11-24T08:09:25.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b32e75640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 0x7f2b2c0d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b32e75640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b2c0d3870 0x7f2b2c0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b32e75640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2c15db00 con 0x7f2b2c0b1400 2023-11-24T08:09:25.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b32e75640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b2c0d0ce0 con 0x7f2b2c0d3870 2023-11-24T08:09:25.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b32e75640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c06d400 con 0x7f2b2c15d720 2023-11-24T08:09:25.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b31412640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2b2c0b1400 0x7f2b2c0b1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51676/0 (socket says 172.21.15.106:51676) 2023-11-24T08:09:25.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b31412640 1 -- 172.21.15.106:0/2033717419 learned_addr learned my addr 172.21.15.106:0/2033717419 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2033717419 >> v2:172.21.15.156:3300/0 conn(0x7f2b2c0d3870 0x7f2b2c0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b30c11640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 0x7f2b2c0d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.057+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699521138 0 0) 0x7f2b2c15db00 con 0x7f2b2c0b1400 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b18003520 con 0x7f2b2c0b1400 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2041389633 0 0) 0x7f2b18003520 con 0x7f2b2c0b1400 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 msgr2=0x7f2b2c0d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 0x7f2b2c0d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 >> v2:172.21.15.156:3300/0 conn(0x7f2b2c0d3870 msgr2=0x7f2b2c0b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 --2- 172.21.15.106:0/2033717419 >> v2:172.21.15.156:3300/0 conn(0x7f2b2c0d3870 0x7f2b2c0b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0d3cf0 con 0x7f2b2c0b1400 2023-11-24T08:09:25.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2b24001670 con 0x7f2b2c0b1400 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 >> v1:172.21.15.106:6789/0 conn(0x7f2b2c0b1400 legacy=0x7f2b2c0b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 0x7f2b18003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b18003ef0 0x7f2b180047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18004d00 0x7f2b18005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0d0ce0 con 0x7f2b18003ef0 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c06d400 con 0x7f2b18004d00 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1800b9d0 con 0x7f2b2c15d720 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.058+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b18003ef0 0x7f2b180047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b30c11640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 0x7f2b18003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b31412640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18004d00 0x7f2b18005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 msgr2=0x7f2b18003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:25.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c15d720 0x7f2b18003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18004d00 msgr2=0x7f2b18005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:25.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18004d00 0x7f2b18005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2033717419 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b18003520 con 0x7f2b18003ef0 2023-11-24T08:09:25.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.059+0000 7f2b31412640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18004d00 0x7f2b18005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:25.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.060+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b18003ef0 0x7f2b180047c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e170 tx=0x7f2b180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=664db451d5bd853d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:25.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.060+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1402cc20 con 0x7f2b18003ef0 2023-11-24T08:09:25.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.060+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1400f040 con 0x7f2b18003ef0 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.060+0000 7f2b2b7fe640 1 -- 172.21.15.106:0/2033717419 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14034440 con 0x7f2b18003ef0 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b32e75640 1 -- 172.21.15.106:0/2033717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b18003ef0 msgr2=0x7f2b180047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2033717419 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b18003ef0 0x7f2b180047c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2b1400e170 tx=0x7f2b180057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b31412640 1 -- 172.21.15.106:0/2033717419 reap_dead start 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b32e75640 1 -- 172.21.15.106:0/2033717419 shutdown_connections 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b32e75640 1 -- 172.21.15.106:0/2033717419 >> 172.21.15.106:0/2033717419 conn(0x7f2b2c08af60 msgr2=0x7f2b2c09c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b32e75640 1 -- 172.21.15.106:0/2033717419 shutdown_connections 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.061+0000 7f2b32e75640 1 -- 172.21.15.106:0/2033717419 wait complete. 2023-11-24T08:09:25.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.063+0000 7f2b32e75640 1 Processor -- start 2023-11-24T08:09:25.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.063+0000 7f2b32e75640 1 -- start start 2023-11-24T08:09:25.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b32e75640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18003ef0 0x7f2b2c0c94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b32e75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 0x7f2b2c0b70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b32e75640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c14edb0 0x7f2b2c15f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b32e75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0b7ab0 con 0x7f2b2c14e950 2023-11-24T08:09:25.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b32e75640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8730 con 0x7f2b18003ef0 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b32e75640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2c0c8a10 con 0x7f2b2c14edb0 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 0x7f2b2c0b70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b2bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 0x7f2b2c0b70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60336/0 (socket says 172.21.15.106:60336) 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.064+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2369779256 learned_addr learned my addr 172.21.15.106:0/2369779256 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b31412640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c14edb0 0x7f2b2c15f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b30c11640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18003ef0 0x7f2b2c0c94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c14edb0 msgr2=0x7f2b2c15f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c14edb0 0x7f2b2c15f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18003ef0 msgr2=0x7f2b2c0c94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18003ef0 0x7f2b2c0c94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b2bfff640 1 -- 172.21.15.106:0/2369779256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b2c0c8cf0 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.065+0000 7f2b2bfff640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 0x7f2b2c0b70c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b14037040 tx=0x7f2b14009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.066+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14034e50 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.066+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b14040450 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.066+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b14040800 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.066+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2c0c8fd0 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.066+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2c0c7e70 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.067+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2c0c9160 con 0x7f2b2c14e950 2023-11-24T08:09:25.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.068+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2b14040960 con 0x7f2b2c14e950 2023-11-24T08:09:25.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.069+0000 7f2b297fa640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b0c06ae40 0x7f2b0c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.069+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b140ac070 con 0x7f2b2c14e950 2023-11-24T08:09:25.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.069+0000 7f2b30c11640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b0c06ae40 0x7f2b0c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.070+0000 7f2b30c11640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b0c06ae40 0x7f2b0c06d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009870 tx=0x7f2b1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:25.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.073+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b140dee90 con 0x7f2b2c14e950 2023-11-24T08:09:25.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.262+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b2c04e8b0 con 0x7f2b2c14e950 2023-11-24T08:09:25.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.263+0000 7f2b297fa640 1 -- 172.21.15.106:0/2369779256 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b14044020 con 0x7f2b2c14e950 2023-11-24T08:09:25.265 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b0c06ae40 msgr2=0x7f2b0c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b0c06ae40 0x7f2b0c06d300 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009870 tx=0x7f2b1c008040 comp rx=0 tx=0).stop 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 msgr2=0x7f2b2c0b70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 0x7f2b2c0b70c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2b14037040 tx=0x7f2b14009750 comp rx=0 tx=0).stop 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 shutdown_connections 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b0c06ae40 0x7f2b0c06d300 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b2c14edb0 0x7f2b2c15f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b2c14e950 0x7f2b2c0b70c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 --2- 172.21.15.106:0/2369779256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b18003ef0 0x7f2b2c0c94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 >> 172.21.15.106:0/2369779256 conn(0x7f2b2c08af60 msgr2=0x7f2b2c0cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 shutdown_connections 2023-11-24T08:09:25.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.266+0000 7f2b32e75640 1 -- 172.21.15.106:0/2369779256 wait complete. 2023-11-24T08:09:25.357 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:09:25.357 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_group_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:09:25.357 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:09:25.357 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_create_and_rm (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:09:25.357 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:09:25.357 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_idempotence (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:09:25.358 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:09:25.358 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:09:25.358 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:09:25.358 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_rm_force (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... skipped 'skipping subvolumegroup snapshot tests' 2023-11-24T08:09:25.358 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) 2023-11-24T08:09:25.358 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:25.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.566+0000 7f3bef59e640 1 Processor -- start 2023-11-24T08:09:25.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.567+0000 7f3bef59e640 1 -- start start 2023-11-24T08:09:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3bef59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3bf015d4e0 0x7f3bf00d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3bef59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3bf00d0240 0x7f3bf00b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3bef59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bf00ce990 con 0x7f3bf00d38e0 2023-11-24T08:09:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3bef59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf00d4110 con 0x7f3bf00d0240 2023-11-24T08:09:25.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3bef59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bf00cec70 con 0x7f3bf015d4e0 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3beed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3bf00d0240 0x7f3bf00b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3beed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3bf00d0240 0x7f3bf00b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60352/0 (socket says 172.21.15.106:60352) 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.568+0000 7f3beed9d640 1 -- 172.21.15.106:0/2358012436 learned_addr learned my addr 172.21.15.106:0/2358012436 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.569+0000 7f3bee59c640 1 --2- 172.21.15.106:0/2358012436 >> v2:172.21.15.156:3300/0 conn(0x7f3bf015d4e0 0x7f3bf00d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.569+0000 7f3beed9d640 1 -- 172.21.15.106:0/2358012436 >> v2:172.21.15.156:3300/0 conn(0x7f3bf015d4e0 msgr2=0x7f3bf00d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.569+0000 7f3beed9d640 1 --2- 172.21.15.106:0/2358012436 >> v2:172.21.15.156:3300/0 conn(0x7f3bf015d4e0 0x7f3bf00d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.569+0000 7f3beed9d640 1 -- 172.21.15.106:0/2358012436 >> v1:172.21.15.156:6789/0 conn(0x7f3bf00d38e0 legacy=0x7f3bf00cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:25.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.569+0000 7f3beed9d640 1 -- 172.21.15.106:0/2358012436 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bf00cef50 con 0x7f3bf00d0240 2023-11-24T08:09:25.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.570+0000 7f3beed9d640 1 --2- 172.21.15.106:0/2358012436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf00d0240 0x7f3bf00b2e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bf008b800 tx=0x7f3be802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b63dd27850981aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:25.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.570+0000 7f3bedd9b640 1 -- 172.21.15.106:0/2358012436 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be802ca80 con 0x7f3bf00d0240 2023-11-24T08:09:25.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.570+0000 7f3bedd9b640 1 -- 172.21.15.106:0/2358012436 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be802cbe0 con 0x7f3bf00d0240 2023-11-24T08:09:25.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.570+0000 7f3bedd9b640 1 -- 172.21.15.106:0/2358012436 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80343f0 con 0x7f3bf00d0240 2023-11-24T08:09:25.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 -- 172.21.15.106:0/2358012436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf00d0240 msgr2=0x7f3bf00b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 --2- 172.21.15.106:0/2358012436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf00d0240 0x7f3bf00b2e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f3bf008b800 tx=0x7f3be802bee0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 -- 172.21.15.106:0/2358012436 shutdown_connections 2023-11-24T08:09:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 --2- 172.21.15.106:0/2358012436 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf00d0240 0x7f3bf00b2e50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 --2- 172.21.15.106:0/2358012436 >> v2:172.21.15.156:3300/0 conn(0x7f3bf015d4e0 0x7f3bf00d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 -- 172.21.15.106:0/2358012436 >> 172.21.15.106:0/2358012436 conn(0x7f3bf009c510 msgr2=0x7f3bf00d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 -- 172.21.15.106:0/2358012436 shutdown_connections 2023-11-24T08:09:25.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.571+0000 7f3bef59e640 1 -- 172.21.15.106:0/2358012436 wait complete. 2023-11-24T08:09:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.573+0000 7f3bef59e640 1 Processor -- start 2023-11-24T08:09:25.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.573+0000 7f3bef59e640 1 -- start start 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3bef59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf00d0240 0x7f3bf015b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3bef59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf00d38e0 0x7f3bf00cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3bef59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 0x7f3bf00b7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3bef59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00cc310 con 0x7f3bf015d4e0 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3bef59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00cb700 con 0x7f3bf00d0240 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3bef59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bf00cb9e0 con 0x7f3bf00d38e0 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 0x7f3bf00b7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 0x7f3bf00b7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60356/0 (socket says 172.21.15.106:60356) 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 -- 172.21.15.106:0/4149935976 learned_addr learned my addr 172.21.15.106:0/4149935976 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 -- 172.21.15.106:0/4149935976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf00d38e0 msgr2=0x7f3bf00cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf00d38e0 0x7f3bf00cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 -- 172.21.15.106:0/4149935976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf00d0240 msgr2=0x7f3bf015b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf00d0240 0x7f3bf015b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.574+0000 7f3beed9d640 1 -- 172.21.15.106:0/4149935976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8009b30 con 0x7f3bf015d4e0 2023-11-24T08:09:25.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.575+0000 7f3beed9d640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 0x7f3bf00b7da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3be8036040 tx=0x7f3be8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.575+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be803e450 con 0x7f3bf015d4e0 2023-11-24T08:09:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.575+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be803e5d0 con 0x7f3bf015d4e0 2023-11-24T08:09:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.575+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bf00cbcc0 con 0x7f3bf015d4e0 2023-11-24T08:09:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.575+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be803e980 con 0x7f3bf015d4e0 2023-11-24T08:09:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.575+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bf015bb40 con 0x7f3bf015d4e0 2023-11-24T08:09:25.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.576+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bf015bd00 con 0x7f3bf015d4e0 2023-11-24T08:09:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.577+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3be802cd50 con 0x7f3bf015d4e0 2023-11-24T08:09:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.578+0000 7f3be6ffd640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd406ae40 0x7f3bd406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:25.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.578+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3be80ac1d0 con 0x7f3bf015d4e0 2023-11-24T08:09:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.578+0000 7f3bee59c640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd406ae40 0x7f3bd406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:25.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.579+0000 7f3bee59c640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd406ae40 0x7f3bd406d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3bf01f6590 tx=0x7f3be0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:25.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.582+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3be8077440 con 0x7f3bf015d4e0 2023-11-24T08:09:25.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.775+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bf00cc450 con 0x7f3bf015d4e0 2023-11-24T08:09:25.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.777+0000 7f3be6ffd640 1 -- 172.21.15.106:0/4149935976 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f3bf00cc450 con 0x7f3bf015d4e0 2023-11-24T08:09:25.778 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:25.779 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:25.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.779+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd406ae40 msgr2=0x7f3bd406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.779+0000 7f3bef59e640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd406ae40 0x7f3bd406d300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3bf01f6590 tx=0x7f3be0008040 comp rx=0 tx=0).stop 2023-11-24T08:09:25.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.779+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 msgr2=0x7f3bf00b7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:25.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.779+0000 7f3bef59e640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 0x7f3bf00b7da0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3be8036040 tx=0x7f3be8004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:25.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 shutdown_connections 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bf015d4e0 0x7f3bf00b7da0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd406ae40 0x7f3bd406d300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bf00d38e0 0x7f3bf00cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 --2- 172.21.15.106:0/4149935976 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bf00d0240 0x7f3bf015b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 >> 172.21.15.106:0/4149935976 conn(0x7f3bf009c510 msgr2=0x7f3bf015c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 shutdown_connections 2023-11-24T08:09:25.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:25.780+0000 7f3bef59e640 1 -- 172.21.15.106:0/4149935976 wait complete. 2023-11-24T08:09:25.800 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:25.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:26.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.019+0000 7f454a178640 1 Processor -- start 2023-11-24T08:09:26.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.019+0000 7f454a178640 1 -- start start 2023-11-24T08:09:26.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.020+0000 7f454a178640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f454415b440 0x7f45440cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.020+0000 7f454a178640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45440c5dc0 con 0x7f45440cbf20 2023-11-24T08:09:26.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f454a178640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454415eea0 con 0x7f45440c59b0 2023-11-24T08:09:26.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f454a178640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440cc3d0 con 0x7f454415b440 2023-11-24T08:09:26.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f4542ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f45440cbf20 0x7f45440c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51682/0 (socket says 172.21.15.106:51682) 2023-11-24T08:09:26.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f4542ffd640 1 -- 172.21.15.106:0/86949779 learned_addr learned my addr 172.21.15.106:0/86949779 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:26.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f45437fe640 1 --2- 172.21.15.106:0/86949779 >> v2:172.21.15.156:3300/0 conn(0x7f454415b440 0x7f45440cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4017549955 0 0) 0x7f45440c5dc0 con 0x7f45440cbf20 2023-11-24T08:09:26.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.021+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f452c003520 con 0x7f45440cbf20 2023-11-24T08:09:26.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907571491 0 0) 0x7f452c003520 con 0x7f45440cbf20 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 >> v2:172.21.15.156:3300/0 conn(0x7f454415b440 msgr2=0x7f45440cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 --2- 172.21.15.106:0/86949779 >> v2:172.21.15.156:3300/0 conn(0x7f454415b440 0x7f45440cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 >> v1:172.21.15.156:6789/0 conn(0x7f45440c59b0 legacy=0x7f454406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440abf30 con 0x7f45440cbf20 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f4538001670 con 0x7f45440cbf20 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 >> v1:172.21.15.106:6789/0 conn(0x7f45440cbf20 legacy=0x7f45440c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c003970 0x7f452c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c0047c0 0x7f452c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f452c005180 0x7f452c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440cc3d0 con 0x7f452c003970 2023-11-24T08:09:26.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c00c120 con 0x7f452c0047c0 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c00c2e0 con 0x7f452c005180 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f45437fe640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c003970 0x7f452c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.022+0000 7f4542ffd640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c0047c0 0x7f452c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f45437fe640 1 -- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f452c005180 msgr2=0x7f452c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f4543fff640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f452c005180 0x7f452c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f45437fe640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f452c005180 0x7f452c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f45437fe640 1 -- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c0047c0 msgr2=0x7f452c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f45437fe640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c0047c0 0x7f452c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f45437fe640 1 -- 172.21.15.106:0/86949779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c003520 con 0x7f452c003970 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f4543fff640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f452c005180 0x7f452c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:26.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f4542ffd640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c0047c0 0x7f452c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:26.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.023+0000 7f4543fff640 1 -- 172.21.15.106:0/86949779 reap_dead start 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.024+0000 7f45437fe640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c003970 0x7f452c003d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4534009ac0 tx=0x7f452c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0c4979b127bc2c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.024+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453402c6b0 con 0x7f452c003970 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.024+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453402c810 con 0x7f452c003970 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.024+0000 7f45427fc640 1 -- 172.21.15.106:0/86949779 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453402cba0 con 0x7f452c003970 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.024+0000 7f454a178640 1 -- 172.21.15.106:0/86949779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c003970 msgr2=0x7f452c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.025+0000 7f454a178640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c003970 0x7f452c003d90 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4534009ac0 tx=0x7f452c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.025+0000 7f454a178640 1 -- 172.21.15.106:0/86949779 shutdown_connections 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.025+0000 7f454a178640 1 --2- 172.21.15.106:0/86949779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f452c003970 0x7f452c003d90 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.025+0000 7f454a178640 1 -- 172.21.15.106:0/86949779 >> 172.21.15.106:0/86949779 conn(0x7f45440b0c50 msgr2=0x7f45440b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:26.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.025+0000 7f454a178640 1 -- 172.21.15.106:0/86949779 shutdown_connections 2023-11-24T08:09:26.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.025+0000 7f454a178640 1 -- 172.21.15.106:0/86949779 wait complete. 2023-11-24T08:09:26.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.026+0000 7f454a178640 1 Processor -- start 2023-11-24T08:09:26.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.027+0000 7f454a178640 1 -- start start 2023-11-24T08:09:26.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.027+0000 7f454a178640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c003970 0x7f45440b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f454a178640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4544158190 0x7f4544156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f454a178640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 0x7f454415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f454a178640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4544159180 con 0x7f454415dd90 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f454a178640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f454415e750 con 0x7f452c003970 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f454a178640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454415bbd0 con 0x7f4544158190 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 0x7f454415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 0x7f454415e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60364/0 (socket says 172.21.15.106:60364) 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 -- 172.21.15.106:0/4056382200 learned_addr learned my addr 172.21.15.106:0/4056382200 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4542ffd640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4544158190 0x7f4544156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f45437fe640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c003970 0x7f45440b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 -- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4544158190 msgr2=0x7f4544156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4544158190 0x7f4544156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 -- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c003970 msgr2=0x7f45440b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c003970 0x7f45440b4100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.028+0000 7f4543fff640 1 -- 172.21.15.106:0/4056382200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4544158610 con 0x7f454415dd90 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.029+0000 7f4543fff640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 0x7f454415e210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f454409d280 tx=0x7f453c00d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.029+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c003d80 con 0x7f454415dd90 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.029+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c005070 con 0x7f454415dd90 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.029+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c005420 con 0x7f454415dd90 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.029+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4544157170 con 0x7f454415dd90 2023-11-24T08:09:26.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.029+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4544158bc0 con 0x7f454415dd90 2023-11-24T08:09:26.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.030+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454404e8b0 con 0x7f454415dd90 2023-11-24T08:09:26.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.031+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f453c005580 con 0x7f454415dd90 2023-11-24T08:09:26.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.032+0000 7f452bfff640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453006ae40 0x7f453006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.032+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f453c088cd0 con 0x7f454415dd90 2023-11-24T08:09:26.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.032+0000 7f45437fe640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453006ae40 0x7f453006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.033+0000 7f45437fe640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453006ae40 0x7f453006d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4534036000 tx=0x7f4534030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.036+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f453c053e20 con 0x7f454415dd90 2023-11-24T08:09:26.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.225+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f454415bd10 con 0x7f454415dd90 2023-11-24T08:09:26.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.226+0000 7f452bfff640 1 -- 172.21.15.106:0/4056382200 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f453c018070 con 0x7f454415dd90 2023-11-24T08:09:26.228 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453006ae40 msgr2=0x7f453006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453006ae40 0x7f453006d300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4534036000 tx=0x7f4534030000 comp rx=0 tx=0).stop 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 msgr2=0x7f454415e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 0x7f454415e210 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f454409d280 tx=0x7f453c00d9d0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 shutdown_connections 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f454415dd90 0x7f454415e210 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.229+0000 7f454a178640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453006ae40 0x7f453006d300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.230+0000 7f454a178640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4544158190 0x7f4544156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.230+0000 7f454a178640 1 --2- 172.21.15.106:0/4056382200 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f452c003970 0x7f45440b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.230+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 >> 172.21.15.106:0/4056382200 conn(0x7f45440b0c50 msgr2=0x7f4544071610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:26.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.230+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 shutdown_connections 2023-11-24T08:09:26.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.230+0000 7f454a178640 1 -- 172.21.15.106:0/4056382200 wait complete. 2023-11-24T08:09:26.251 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:26.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.467+0000 7f02317c9640 1 Processor -- start 2023-11-24T08:09:26.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.467+0000 7f02317c9640 1 -- start start 2023-11-24T08:09:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f02317c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f022c072160 0x7f022c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f02317c9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f022c0c49f0 0x7f022c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f02317c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f02317c9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c06e0d0 con 0x7f022c072160 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f02317c9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c15ec90 con 0x7f022c0c49f0 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f02317c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0cd530 con 0x7f022c06dc40 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f022affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f022c072160 0x7f022c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f022affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f022c072160 0x7f022c15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60370/0 (socket says 172.21.15.106:60370) 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.469+0000 7f022affd640 1 -- 172.21.15.106:0/2070007014 learned_addr learned my addr 172.21.15.106:0/2070007014 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022affd640 1 -- 172.21.15.106:0/2070007014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 msgr2=0x7f022c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022b7fe640 1 --2- 172.21.15.106:0/2070007014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022a7fc640 1 --2- 172.21.15.106:0/2070007014 >> v2:172.21.15.156:3300/0 conn(0x7f022c0c49f0 0x7f022c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022affd640 1 --2- 172.21.15.106:0/2070007014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022affd640 1 -- 172.21.15.106:0/2070007014 >> v2:172.21.15.156:3300/0 conn(0x7f022c0c49f0 msgr2=0x7f022c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022affd640 1 --2- 172.21.15.106:0/2070007014 >> v2:172.21.15.156:3300/0 conn(0x7f022c0c49f0 0x7f022c06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022affd640 1 -- 172.21.15.106:0/2070007014 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0c4ed0 con 0x7f022c072160 2023-11-24T08:09:26.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.470+0000 7f022a7fc640 1 --2- 172.21.15.106:0/2070007014 >> v2:172.21.15.156:3300/0 conn(0x7f022c0c49f0 0x7f022c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.471+0000 7f022affd640 1 --2- 172.21.15.106:0/2070007014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c072160 0x7f022c15e750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f022c0cef60 tx=0x7f021c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f4bb7756afd055 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.471+0000 7f0229ffb640 1 -- 172.21.15.106:0/2070007014 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c031870 con 0x7f022c072160 2023-11-24T08:09:26.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.471+0000 7f0229ffb640 1 -- 172.21.15.106:0/2070007014 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c0319d0 con 0x7f022c072160 2023-11-24T08:09:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.471+0000 7f0229ffb640 1 -- 172.21.15.106:0/2070007014 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c031d60 con 0x7f022c072160 2023-11-24T08:09:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 -- 172.21.15.106:0/2070007014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c072160 msgr2=0x7f022c15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 --2- 172.21.15.106:0/2070007014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c072160 0x7f022c15e750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f022c0cef60 tx=0x7f021c02b910 comp rx=0 tx=0).stop 2023-11-24T08:09:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 -- 172.21.15.106:0/2070007014 shutdown_connections 2023-11-24T08:09:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 --2- 172.21.15.106:0/2070007014 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 --2- 172.21.15.106:0/2070007014 >> v2:172.21.15.156:3300/0 conn(0x7f022c0c49f0 0x7f022c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 --2- 172.21.15.106:0/2070007014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c072160 0x7f022c15e750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 -- 172.21.15.106:0/2070007014 >> 172.21.15.106:0/2070007014 conn(0x7f022c0b0c50 msgr2=0x7f022c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 -- 172.21.15.106:0/2070007014 shutdown_connections 2023-11-24T08:09:26.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.472+0000 7f02317c9640 1 -- 172.21.15.106:0/2070007014 wait complete. 2023-11-24T08:09:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.474+0000 7f02317c9640 1 Processor -- start 2023-11-24T08:09:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.474+0000 7f02317c9640 1 -- start start 2023-11-24T08:09:26.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.475+0000 7f02317c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.475+0000 7f02317c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f022c072160 0x7f022c154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.475+0000 7f02317c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 0x7f022c0cbb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.475+0000 7f02317c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0cb6b0 con 0x7f022c0c49f0 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.475+0000 7f02317c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0caaa0 con 0x7f022c072160 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.475+0000 7f02317c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0cad80 con 0x7f022c06dc40 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 0x7f022c0cbb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 0x7f022c0cbb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60382/0 (socket says 172.21.15.106:60382) 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 -- 172.21.15.106:0/961621953 learned_addr learned my addr 172.21.15.106:0/961621953 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:26.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022a7fc640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f022c072160 0x7f022c154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022affd640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 -- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 msgr2=0x7f022c0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 -- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f022c072160 msgr2=0x7f022c154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f022c072160 0x7f022c154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 -- 172.21.15.106:0/961621953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c0094d0 con 0x7f022c0c49f0 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022affd640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022a7fc640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f022c072160 0x7f022c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.476+0000 7f022b7fe640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 0x7f022c0cbb40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f022400ccf0 tx=0x7f022400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.477+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224003b90 con 0x7f022c0c49f0 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.477+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224003cf0 con 0x7f022c0c49f0 2023-11-24T08:09:26.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.477+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224004240 con 0x7f022c0c49f0 2023-11-24T08:09:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.477+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c0cb060 con 0x7f022c0c49f0 2023-11-24T08:09:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.477+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c154ef0 con 0x7f022c0c49f0 2023-11-24T08:09:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.478+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c0cb860 con 0x7f022c0c49f0 2023-11-24T08:09:26.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.479+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f02240043a0 con 0x7f022c0c49f0 2023-11-24T08:09:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.480+0000 7f02137fe640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f021806ae40 0x7f021806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.480+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0224012070 con 0x7f022c0c49f0 2023-11-24T08:09:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.480+0000 7f022affd640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f021806ae40 0x7f021806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.481+0000 7f022affd640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f021806ae40 0x7f021806d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f021c036000 tx=0x7f021c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.484+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f022c0cb860 con 0x7f022c0c49f0 2023-11-24T08:09:26.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.680+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f022c04e8b0 con 0x7f022c0c49f0 2023-11-24T08:09:26.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.681+0000 7f02137fe640 1 -- 172.21.15.106:0/961621953 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0224055020 con 0x7f022c0c49f0 2023-11-24T08:09:26.682 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f021806ae40 msgr2=0x7f021806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f021806ae40 0x7f021806d300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f021c036000 tx=0x7f021c03c040 comp rx=0 tx=0).stop 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 msgr2=0x7f022c0cbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 0x7f022c0cbb40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f022400ccf0 tx=0x7f022400a660 comp rx=0 tx=0).stop 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 shutdown_connections 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f022c0c49f0 0x7f022c0cbb40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f022c072160 0x7f022c154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f021806ae40 0x7f021806d300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 --2- 172.21.15.106:0/961621953 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f022c06dc40 0x7f022c0b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 >> 172.21.15.106:0/961621953 conn(0x7f022c0b0c50 msgr2=0x7f022c0af280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:26.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 shutdown_connections 2023-11-24T08:09:26.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.684+0000 7f02317c9640 1 -- 172.21.15.106:0/961621953 wait complete. 2023-11-24T08:09:26.704 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:26.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.913+0000 7f49cb70c640 1 Processor -- start 2023-11-24T08:09:26.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.913+0000 7f49cb70c640 1 -- start start 2023-11-24T08:09:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.914+0000 7f49cb70c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49c415d9b0 0x7f49c415e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.914+0000 7f49cb70c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 0x7f49c415e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.914+0000 7f49cb70c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c406e930 con 0x7f49c40b28d0 2023-11-24T08:09:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.914+0000 7f49cb70c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c40cf0c0 con 0x7f49c415d9b0 2023-11-24T08:09:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.914+0000 7f49cb70c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c406f5c0 con 0x7f49c406eef0 2023-11-24T08:09:26.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.915+0000 7f49c9ca9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f49c40b28d0 0x7f49c406d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51696/0 (socket says 172.21.15.106:51696) 2023-11-24T08:09:26.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.915+0000 7f49c9ca9640 1 -- 172.21.15.106:0/3065140596 learned_addr learned my addr 172.21.15.106:0/3065140596 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.915+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3065140596 >> v2:172.21.15.156:3300/0 conn(0x7f49c415d9b0 0x7f49c415e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.915+0000 7f49c8ca7640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 0x7f49c415e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.915+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 991864305 0 0) 0x7f49c406e930 con 0x7f49c40b28d0 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.915+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac003520 con 0x7f49c40b28d0 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231691502 0 0) 0x7f49ac003520 con 0x7f49c40b28d0 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 msgr2=0x7f49c415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 0x7f49c415e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 >> v2:172.21.15.156:3300/0 conn(0x7f49c415d9b0 msgr2=0x7f49c415e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 --2- 172.21.15.106:0/3065140596 >> v2:172.21.15.156:3300/0 conn(0x7f49c415d9b0 0x7f49c415e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c415dd90 con 0x7f49c40b28d0 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f49bc001a10 con 0x7f49c40b28d0 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 >> v1:172.21.15.106:6789/0 conn(0x7f49c40b28d0 legacy=0x7f49c406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49ac003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 0x7f49ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac004d00 0x7f49ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c406f5c0 con 0x7f49ac003970 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00db90 con 0x7f49ac004d00 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00dcd0 con 0x7f49c406eef0 2023-11-24T08:09:26.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49ac003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49c94a8640 1 -- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 msgr2=0x7f49ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c406eef0 0x7f49ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49c94a8640 1 -- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac004d00 msgr2=0x7f49ac005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac004d00 0x7f49ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.916+0000 7f49c94a8640 1 -- 172.21.15.106:0/3065140596 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac003520 con 0x7f49ac003970 2023-11-24T08:09:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.917+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49ac003d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ac0 tx=0x7f49ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e0fcb13457e1bd0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b402c6d0 con 0x7f49ac003970 2023-11-24T08:09:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b402c830 con 0x7f49ac003970 2023-11-24T08:09:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49bbfff640 1 -- 172.21.15.106:0/3065140596 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b402cbc0 con 0x7f49ac003970 2023-11-24T08:09:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49cb70c640 1 -- 172.21.15.106:0/3065140596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 msgr2=0x7f49ac003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3065140596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49ac003d90 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ac0 tx=0x7f49ac0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49c9ca9640 1 -- 172.21.15.106:0/3065140596 reap_dead start 2023-11-24T08:09:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49cb70c640 1 -- 172.21.15.106:0/3065140596 shutdown_connections 2023-11-24T08:09:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.918+0000 7f49cb70c640 1 -- 172.21.15.106:0/3065140596 >> 172.21.15.106:0/3065140596 conn(0x7f49c40999d0 msgr2=0x7f49c40b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.919+0000 7f49cb70c640 1 -- 172.21.15.106:0/3065140596 shutdown_connections 2023-11-24T08:09:26.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.919+0000 7f49cb70c640 1 -- 172.21.15.106:0/3065140596 wait complete. 2023-11-24T08:09:26.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.920+0000 7f49cb70c640 1 Processor -- start 2023-11-24T08:09:26.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.920+0000 7f49cb70c640 1 -- start start 2023-11-24T08:09:26.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.921+0000 7f49cb70c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49c40ad870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.921+0000 7f49cb70c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c40b0700 0x7f49c40b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.921+0000 7f49cb70c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c40b3dc0 0x7f49c40b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.921+0000 7f49cb70c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40b4780 con 0x7f49ac003970 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.921+0000 7f49cb70c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40b1d60 con 0x7f49c40b3dc0 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.921+0000 7f49cb70c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40b1150 con 0x7f49c40b0700 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49c40ad870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49c40ad870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60400/0 (socket says 172.21.15.106:60400) 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 -- 172.21.15.106:0/3320852204 learned_addr learned my addr 172.21.15.106:0/3320852204 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 -- 172.21.15.106:0/3320852204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c40b0700 msgr2=0x7f49c40b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c40b0700 0x7f49c40b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 -- 172.21.15.106:0/3320852204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c40b3dc0 msgr2=0x7f49c40b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:26.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c40b3dc0 0x7f49c40b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:26.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 -- 172.21.15.106:0/3320852204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c40b1430 con 0x7f49ac003970 2023-11-24T08:09:26.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.922+0000 7f49c94a8640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49c40ad870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49b4037040 tx=0x7f49b4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.923+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4003810 con 0x7f49ac003970 2023-11-24T08:09:26.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.923+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4003970 con 0x7f49ac003970 2023-11-24T08:09:26.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.923+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c40b1710 con 0x7f49ac003970 2023-11-24T08:09:26.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.923+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4003d20 con 0x7f49ac003970 2023-11-24T08:09:26.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.923+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c40b2fa0 con 0x7f49ac003970 2023-11-24T08:09:26.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.924+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c404e8b0 con 0x7f49ac003970 2023-11-24T08:09:26.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.925+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49b4040450 con 0x7f49ac003970 2023-11-24T08:09:26.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.926+0000 7f49b9ffb640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006ae40 0x7f49b006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:26.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.926+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49b40ab610 con 0x7f49ac003970 2023-11-24T08:09:26.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.926+0000 7f49c8ca7640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006ae40 0x7f49b006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:26.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.927+0000 7f49c8ca7640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006ae40 0x7f49b006d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006a90 tx=0x7f49c0006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:26.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:26.930+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49b4076610 con 0x7f49ac003970 2023-11-24T08:09:27.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.120+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f49c40cd760 con 0x7f49ac003970 2023-11-24T08:09:27.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.121+0000 7f49b9ffb640 1 -- 172.21.15.106:0/3320852204 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f49b40044e0 con 0x7f49ac003970 2023-11-24T08:09:27.122 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006ae40 msgr2=0x7f49b006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006ae40 0x7f49b006d300 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006a90 tx=0x7f49c0006a20 comp rx=0 tx=0).stop 2023-11-24T08:09:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 msgr2=0x7f49c40ad870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49c40ad870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f49b4037040 tx=0x7f49b4004730 comp rx=0 tx=0).stop 2023-11-24T08:09:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 shutdown_connections 2023-11-24T08:09:27.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c40b3dc0 0x7f49c40b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006ae40 0x7f49b006d300 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49c40b0700 0x7f49c40b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 --2- 172.21.15.106:0/3320852204 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac003970 0x7f49c40ad870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.123+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 >> 172.21.15.106:0/3320852204 conn(0x7f49c40999d0 msgr2=0x7f49c415a710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.124+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 shutdown_connections 2023-11-24T08:09:27.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.124+0000 7f49cb70c640 1 -- 172.21.15.106:0/3320852204 wait complete. 2023-11-24T08:09:27.145 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T08:09:27.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.357+0000 7fa793933640 1 Processor -- start 2023-11-24T08:09:27.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.358+0000 7fa793933640 1 -- start start 2023-11-24T08:09:27.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.359+0000 7fa793933640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 0x7fa78c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.359+0000 7fa793933640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c0cece0 con 0x7fa78c15c7f0 2023-11-24T08:09:27.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.359+0000 7fa793933640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78c06d040 con 0x7fa78c0cdfb0 2023-11-24T08:09:27.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.359+0000 7fa793933640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c156010 con 0x7fa78c0cf450 2023-11-24T08:09:27.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.359+0000 7fa7916cf640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa78c15c7f0 0x7fa78c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51702/0 (socket says 172.21.15.106:51702) 2023-11-24T08:09:27.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.359+0000 7fa7916cf640 1 -- 172.21.15.106:0/2573259965 learned_addr learned my addr 172.21.15.106:0/2573259965 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:27.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38812028 0 0) 0x7fa78c0cece0 con 0x7fa78c15c7f0 2023-11-24T08:09:27.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa778003520 con 0x7fa78c15c7f0 2023-11-24T08:09:27.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa790ece640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 0x7fa78c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753851839 0 0) 0x7fa778003520 con 0x7fa78c15c7f0 2023-11-24T08:09:27.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 msgr2=0x7fa78c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 0x7fa78c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 >> v1:172.21.15.156:6789/0 conn(0x7fa78c0cdfb0 legacy=0x7fa78c0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c06d320 con 0x7fa78c15c7f0 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fa788001670 con 0x7fa78c15c7f0 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 >> v1:172.21.15.106:6789/0 conn(0x7fa78c15c7f0 legacy=0x7fa78c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7780039a0 0x7fa778003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.360+0000 7fa783fff640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780047c0 0x7fa778004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa783fff640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 0x7fa7780051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c156010 con 0x7fa7780039a0 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77800bac0 con 0x7fa7780047c0 2023-11-24T08:09:27.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa77800bc80 con 0x7fa78c0cf450 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa7916cf640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7780039a0 0x7fa778003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa7916cf640 1 -- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 msgr2=0x7fa7780051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa791ed0640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 0x7fa7780051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa790ece640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780047c0 0x7fa778004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa7916cf640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c0cf450 0x7fa7780051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa7916cf640 1 -- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780047c0 msgr2=0x7fa778004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa7916cf640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780047c0 0x7fa778004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.361+0000 7fa7916cf640 1 -- 172.21.15.106:0/2573259965 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778003520 con 0x7fa7780039a0 2023-11-24T08:09:27.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.362+0000 7fa7916cf640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7780039a0 0x7fa778003da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa78800c210 tx=0x7fa7780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a1d9b5f132721f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:27.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.362+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78802cae0 con 0x7fa7780039a0 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78802cc60 con 0x7fa7780039a0 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa783fff640 1 -- 172.21.15.106:0/2573259965 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7880058d0 con 0x7fa7780039a0 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa793933640 1 -- 172.21.15.106:0/2573259965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7780039a0 msgr2=0x7fa778003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa793933640 1 --2- 172.21.15.106:0/2573259965 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7780039a0 0x7fa778003da0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa78800c210 tx=0x7fa7780057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa791ed0640 1 -- 172.21.15.106:0/2573259965 reap_dead start 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa793933640 1 -- 172.21.15.106:0/2573259965 shutdown_connections 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa793933640 1 -- 172.21.15.106:0/2573259965 >> 172.21.15.106:0/2573259965 conn(0x7fa78c06a920 msgr2=0x7fa78c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa793933640 1 -- 172.21.15.106:0/2573259965 shutdown_connections 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.363+0000 7fa793933640 1 -- 172.21.15.106:0/2573259965 wait complete. 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.365+0000 7fa793933640 1 Processor -- start 2023-11-24T08:09:27.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.365+0000 7fa793933640 1 -- start start 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa793933640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780039a0 0x7fa78c0ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa793933640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c157640 0x7fa78c15f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa793933640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 0x7fa78c1494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa793933640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c157220 con 0x7fa78c149040 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa793933640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c149a00 con 0x7fa7780039a0 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa793933640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78c1565a0 con 0x7fa78c157640 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa791ed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 0x7fa78c1494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa791ed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 0x7fa78c1494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60416/0 (socket says 172.21.15.106:60416) 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.366+0000 7fa791ed0640 1 -- 172.21.15.106:0/1312810654 learned_addr learned my addr 172.21.15.106:0/1312810654 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa7916cf640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780039a0 0x7fa78c0ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa791ed0640 1 -- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c157640 msgr2=0x7fa78c15f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa791ed0640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c157640 0x7fa78c15f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa791ed0640 1 -- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780039a0 msgr2=0x7fa78c0ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa791ed0640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780039a0 0x7fa78c0ada20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa791ed0640 1 -- 172.21.15.106:0/1312810654 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78c156880 con 0x7fa78c149040 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa791ed0640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 0x7fa78c1494c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0ccd80 tx=0x7fa78400d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:27.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784004b50 con 0x7fa78c149040 2023-11-24T08:09:27.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa784004cb0 con 0x7fa78c149040 2023-11-24T08:09:27.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c156b60 con 0x7fa78c149040 2023-11-24T08:09:27.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa784005270 con 0x7fa78c149040 2023-11-24T08:09:27.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.367+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78c153590 con 0x7fa78c149040 2023-11-24T08:09:27.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.368+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78c157360 con 0x7fa78c149040 2023-11-24T08:09:27.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.369+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa7840053d0 con 0x7fa78c149040 2023-11-24T08:09:27.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.370+0000 7fa781ffb640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa77006ae70 0x7fa77006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:27.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.370+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7840896c0 con 0x7fa78c149040 2023-11-24T08:09:27.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.371+0000 7fa7916cf640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa77006ae70 0x7fa77006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:27.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.371+0000 7fa7916cf640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa77006ae70 0x7fa77006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa788034000 tx=0x7fa788009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:27.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.375+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa78c157360 con 0x7fa78c149040 2023-11-24T08:09:27.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:27.637+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7fa78c157360 con 0x7fa78c149040 2023-11-24T08:09:28.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.415+0000 7fa781ffb640 1 -- 172.21.15.106:0/1312810654 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v53) v1 ==== 184+0+0 (secure 0 0 0) 0x7fa78c157360 con 0x7fa78c149040 2023-11-24T08:09:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa77006ae70 msgr2=0x7fa77006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa77006ae70 0x7fa77006d330 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa788034000 tx=0x7fa788009030 comp rx=0 tx=0).stop 2023-11-24T08:09:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 msgr2=0x7fa78c1494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 0x7fa78c1494c0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa78c0ccd80 tx=0x7fa78400d9d0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 shutdown_connections 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa78c149040 0x7fa78c1494c0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa77006ae70 0x7fa77006d330 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa78c157640 0x7fa78c15f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 --2- 172.21.15.106:0/1312810654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7780039a0 0x7fa78c0ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.418+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 >> 172.21.15.106:0/1312810654 conn(0x7fa78c06a920 msgr2=0x7fa78c071f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.419+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 shutdown_connections 2023-11-24T08:09:28.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.419+0000 7fa793933640 1 -- 172.21.15.106:0/1312810654 wait complete. 2023-11-24T08:09:28.440 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:09:28.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.653+0000 7f27ec3f1640 1 Processor -- start 2023-11-24T08:09:28.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.653+0000 7f27ec3f1640 1 -- start start 2023-11-24T08:09:28.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.654+0000 7f27ec3f1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f27e40cc0f0 0x7f27e40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:28.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.654+0000 7f27ec3f1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40c57e0 0x7f27e406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:28.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.654+0000 7f27ec3f1640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e415eea0 con 0x7f27e415b440 2023-11-24T08:09:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.654+0000 7f27ec3f1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e415b820 con 0x7f27e40cc0f0 2023-11-24T08:09:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.654+0000 7f27ec3f1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e40abf30 con 0x7f27e40c57e0 2023-11-24T08:09:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f27e40cc0f0 0x7f27e40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f27e40cc0f0 0x7f27e40c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60430/0 (socket says 172.21.15.106:60430) 2023-11-24T08:09:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 -- 172.21.15.106:0/1325791501 learned_addr learned my addr 172.21.15.106:0/1325791501 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:28.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27ea98e640 1 --2- 172.21.15.106:0/1325791501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40c57e0 0x7f27e406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 -- 172.21.15.106:0/1325791501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40c57e0 msgr2=0x7f27e406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 --2- 172.21.15.106:0/1325791501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40c57e0 0x7f27e406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 -- 172.21.15.106:0/1325791501 >> v1:172.21.15.156:6789/0 conn(0x7f27e415b440 legacy=0x7f27e40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:28.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.655+0000 7f27e998c640 1 -- 172.21.15.106:0/1325791501 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e415f0f0 con 0x7f27e40cc0f0 2023-11-24T08:09:28.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.657+0000 7f27e998c640 1 --2- 172.21.15.106:0/1325791501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40cc0f0 0x7f27e40c52a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009870 tx=0x7f27d402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2d38617ceef089 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.657+0000 7f27e918b640 1 -- 172.21.15.106:0/1325791501 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4031640 con 0x7f27e40cc0f0 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.657+0000 7f27e918b640 1 -- 172.21.15.106:0/1325791501 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d40317a0 con 0x7f27e40cc0f0 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.657+0000 7f27e918b640 1 -- 172.21.15.106:0/1325791501 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d4031b30 con 0x7f27e40cc0f0 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 -- 172.21.15.106:0/1325791501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40cc0f0 msgr2=0x7f27e40c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/1325791501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40cc0f0 0x7f27e40c52a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f27d4009870 tx=0x7f27d402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 -- 172.21.15.106:0/1325791501 shutdown_connections 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/1325791501 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40c57e0 0x7f27e406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/1325791501 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40cc0f0 0x7f27e40c52a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 -- 172.21.15.106:0/1325791501 >> 172.21.15.106:0/1325791501 conn(0x7f27e40b0c50 msgr2=0x7f27e40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 -- 172.21.15.106:0/1325791501 shutdown_connections 2023-11-24T08:09:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.658+0000 7f27ec3f1640 1 -- 172.21.15.106:0/1325791501 wait complete. 2023-11-24T08:09:28.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.660+0000 7f27ec3f1640 1 Processor -- start 2023-11-24T08:09:28.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.660+0000 7f27ec3f1640 1 -- start start 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.661+0000 7f27ec3f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 0x7f27e4154880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.661+0000 7f27ec3f1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40cc0f0 0x7f27e40ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.661+0000 7f27ec3f1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e415b440 0x7f27e40adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.661+0000 7f27ec3f1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41f3cf0 con 0x7f27e40c57e0 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.661+0000 7f27ec3f1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e41f33b0 con 0x7f27e415b440 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.661+0000 7f27ec3f1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e4157690 con 0x7f27e40cc0f0 2023-11-24T08:09:28.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 0x7f27e4154880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 0x7f27e4154880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60440/0 (socket says 172.21.15.106:60440) 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 -- 172.21.15.106:0/2524796028 learned_addr learned my addr 172.21.15.106:0/2524796028 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27e998c640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40cc0f0 0x7f27e40ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea98e640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e415b440 0x7f27e40adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 -- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40cc0f0 msgr2=0x7f27e40ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40cc0f0 0x7f27e40ad5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 -- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e415b440 msgr2=0x7f27e40adb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e415b440 0x7f27e40adb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.662+0000 7f27ea18d640 1 -- 172.21.15.106:0/2524796028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d40094d0 con 0x7f27e40c57e0 2023-11-24T08:09:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.663+0000 7f27ea18d640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 0x7f27e4154880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c400 tx=0x7f27e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.663+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00052d0 con 0x7f27e40c57e0 2023-11-24T08:09:28.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.663+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0005430 con 0x7f27e40c57e0 2023-11-24T08:09:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.663+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0003830 con 0x7f27e40c57e0 2023-11-24T08:09:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.663+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e40c8d70 con 0x7f27e40c57e0 2023-11-24T08:09:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.663+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e41f3910 con 0x7f27e40c57e0 2023-11-24T08:09:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.664+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e41f3e30 con 0x7f27e40c57e0 2023-11-24T08:09:28.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.666+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f27e0003990 con 0x7f27e40c57e0 2023-11-24T08:09:28.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.666+0000 7f27daffd640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f27c806ae40 0x7f27c806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:28.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.667+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27e0088ba0 con 0x7f27e40c57e0 2023-11-24T08:09:28.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.667+0000 7f27e998c640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f27c806ae40 0x7f27c806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:28.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.668+0000 7f27e998c640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f27c806ae40 0x7f27c806d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27d4036000 tx=0x7f27d403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:28.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.672+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f27e41f3e30 con 0x7f27e40c57e0 2023-11-24T08:09:28.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.863+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f27e41f3e30 con 0x7f27e40c57e0 2023-11-24T08:09:28.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.868+0000 7f27daffd640 1 -- 172.21.15.106:0/2524796028 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27e00170c0 con 0x7f27e40c57e0 2023-11-24T08:09:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f27c806ae40 msgr2=0x7f27c806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f27c806ae40 0x7f27c806d300 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f27d4036000 tx=0x7f27d403c040 comp rx=0 tx=0).stop 2023-11-24T08:09:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 msgr2=0x7f27e4154880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 0x7f27e4154880 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f27e000c400 tx=0x7f27e000c8c0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 shutdown_connections 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f27e415b440 0x7f27e40adb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f27c806ae40 0x7f27c806d300 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f27e40cc0f0 0x7f27e40ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 --2- 172.21.15.106:0/2524796028 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f27e40c57e0 0x7f27e4154880 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 >> 172.21.15.106:0/2524796028 conn(0x7f27e40b0c50 msgr2=0x7f27e4071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.871+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 shutdown_connections 2023-11-24T08:09:28.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:28.872+0000 7f27ec3f1640 1 -- 172.21.15.106:0/2524796028 wait complete. 2023-11-24T08:09:28.890 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:09:28.891 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:09:28.995 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:09:29.120 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:09:29.120 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:29.120 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:29.184 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:29.185 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:29.240 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:29.240 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:29.295 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:29.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.496+0000 7f42c4ac9640 1 Processor -- start 2023-11-24T08:09:29.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.497+0000 7f42c4ac9640 1 -- start start 2023-11-24T08:09:29.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42c4ac9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f42c015df80 0x7f42c00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42c4ac9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f42c00cd470 0x7f42c015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42c4ac9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42c015e3f0 con 0x7f42c00b2250 2023-11-24T08:09:29.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42c4ac9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c015a990 con 0x7f42c015df80 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42c4ac9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c015ab80 con 0x7f42c00cd470 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42be59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f42c00b2250 0x7f42c00b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51706/0 (socket says 172.21.15.106:51706) 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42be59c640 1 -- 172.21.15.106:0/587604524 learned_addr learned my addr 172.21.15.106:0/587604524 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42b7fff640 1 --2- 172.21.15.106:0/587604524 >> v2:172.21.15.106:3300/0 conn(0x7f42c015df80 0x7f42c00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.498+0000 7f42bdd9b640 1 -- 172.21.15.106:0/587604524 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189117834 0 0) 0x7f42c015e3f0 con 0x7f42c00b2250 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.499+0000 7f42bdd9b640 1 -- 172.21.15.106:0/587604524 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42ac003520 con 0x7f42c00b2250 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.499+0000 7f42bed9d640 1 --2- 172.21.15.106:0/587604524 >> v2:172.21.15.156:3300/0 conn(0x7f42c00cd470 0x7f42c015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.499+0000 7f42b7fff640 1 -- 172.21.15.106:0/587604524 >> v2:172.21.15.156:3300/0 conn(0x7f42c00cd470 msgr2=0x7f42c015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:29.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.499+0000 7f42b7fff640 1 --2- 172.21.15.106:0/587604524 >> v2:172.21.15.156:3300/0 conn(0x7f42c00cd470 0x7f42c015b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.499+0000 7f42b7fff640 1 -- 172.21.15.106:0/587604524 >> v1:172.21.15.106:6789/0 conn(0x7f42c00b2250 legacy=0x7f42c00b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.499+0000 7f42b7fff640 1 -- 172.21.15.106:0/587604524 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c00b2d40 con 0x7f42c015df80 2023-11-24T08:09:29.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.500+0000 7f42b7fff640 1 --2- 172.21.15.106:0/587604524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c015df80 0x7f42c00ccf30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f42c00b2fa0 tx=0x7f42a802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f73a838a6bed68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.500+0000 7f42bdd9b640 1 -- 172.21.15.106:0/587604524 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8031830 con 0x7f42c015df80 2023-11-24T08:09:29.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.500+0000 7f42bdd9b640 1 -- 172.21.15.106:0/587604524 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a8031990 con 0x7f42c015df80 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.500+0000 7f42bdd9b640 1 -- 172.21.15.106:0/587604524 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8031d20 con 0x7f42c015df80 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 -- 172.21.15.106:0/587604524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c015df80 msgr2=0x7f42c00ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/587604524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c015df80 0x7f42c00ccf30 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f42c00b2fa0 tx=0x7f42a802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 -- 172.21.15.106:0/587604524 shutdown_connections 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/587604524 >> v2:172.21.15.156:3300/0 conn(0x7f42c00cd470 0x7f42c015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/587604524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c015df80 0x7f42c00ccf30 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 -- 172.21.15.106:0/587604524 >> 172.21.15.106:0/587604524 conn(0x7f42c006a920 msgr2=0x7f42c006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 -- 172.21.15.106:0/587604524 shutdown_connections 2023-11-24T08:09:29.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.501+0000 7f42c4ac9640 1 -- 172.21.15.106:0/587604524 wait complete. 2023-11-24T08:09:29.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.503+0000 7f42c4ac9640 1 Processor -- start 2023-11-24T08:09:29.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.503+0000 7f42c4ac9640 1 -- start start 2023-11-24T08:09:29.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42c4ac9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c00b2250 0x7f42c01f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42c4ac9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 0x7f42c01f0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42c4ac9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42c015df80 0x7f42c015cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42c4ac9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c01f37a0 con 0x7f42c00cd470 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42c4ac9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c015d440 con 0x7f42c00b2250 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42c4ac9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c01f3150 con 0x7f42c015df80 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 0x7f42c01f0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 0x7f42c01f0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60464/0 (socket says 172.21.15.106:60464) 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 -- 172.21.15.106:0/479719018 learned_addr learned my addr 172.21.15.106:0/479719018 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42be59c640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c00b2250 0x7f42c01f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42bed9d640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42c015df80 0x7f42c015cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 -- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42c015df80 msgr2=0x7f42c015cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42c015df80 0x7f42c015cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 -- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c00b2250 msgr2=0x7f42c01f3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c00b2250 0x7f42c01f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.504+0000 7f42b7fff640 1 -- 172.21.15.106:0/479719018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8009400 con 0x7f42c00cd470 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.505+0000 7f42b7fff640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 0x7f42c01f0630 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42a8000c00 tx=0x7f42a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.505+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a803a8f0 con 0x7f42c00cd470 2023-11-24T08:09:29.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.505+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a803aa70 con 0x7f42c00cd470 2023-11-24T08:09:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.505+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a803ae20 con 0x7f42c00cd470 2023-11-24T08:09:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.505+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c01f3430 con 0x7f42c00cd470 2023-11-24T08:09:29.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.505+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c015d010 con 0x7f42c00cd470 2023-11-24T08:09:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.506+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c01f3950 con 0x7f42c00cd470 2023-11-24T08:09:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.507+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42a804d450 con 0x7f42c00cd470 2023-11-24T08:09:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.508+0000 7f42b6ffd640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f42a006ae40 0x7f42a006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.508+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42a80ab6a0 con 0x7f42c00cd470 2023-11-24T08:09:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.508+0000 7f42be59c640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f42a006ae40 0x7f42a006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.509+0000 7f42be59c640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f42a006ae40 0x7f42a006d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42b00011e0 tx=0x7f42b0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.512+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42c01f3950 con 0x7f42c00cd470 2023-11-24T08:09:29.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.704+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42c0067470 con 0x7f42c00cd470 2023-11-24T08:09:29.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.707+0000 7f42b6ffd640 1 -- 172.21.15.106:0/479719018 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7f42a8035090 con 0x7f42c00cd470 2023-11-24T08:09:29.710 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:29.710 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f42a006ae40 msgr2=0x7f42a006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f42a006ae40 0x7f42a006d300 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f42b00011e0 tx=0x7f42b0009040 comp rx=0 tx=0).stop 2023-11-24T08:09:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 msgr2=0x7f42c01f0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 0x7f42c01f0630 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f42a8000c00 tx=0x7f42a80046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 shutdown_connections 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f42a006ae40 0x7f42a006d300 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42c015df80 0x7f42c015cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42c00cd470 0x7f42c01f0630 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 --2- 172.21.15.106:0/479719018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42c00b2250 0x7f42c01f3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 >> 172.21.15.106:0/479719018 conn(0x7f42c006a920 msgr2=0x7f42c01585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 shutdown_connections 2023-11-24T08:09:29.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.710+0000 7f42c4ac9640 1 -- 172.21.15.106:0/479719018 wait complete. 2023-11-24T08:09:29.729 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:29.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:29.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.932+0000 7fd18fc51640 1 Processor -- start 2023-11-24T08:09:29.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.932+0000 7fd18fc51640 1 -- start start 2023-11-24T08:09:29.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.933+0000 7fd18fc51640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd188072160 0x7fd18815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.933+0000 7fd18fc51640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd18815ec90 con 0x7fd18806de10 2023-11-24T08:09:29.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.933+0000 7fd18fc51640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1880cd530 con 0x7fd1880c49f0 2023-11-24T08:09:29.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.933+0000 7fd18fc51640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1880c4ed0 con 0x7fd188072160 2023-11-24T08:09:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.933+0000 7fd18d9ed640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd188072160 0x7fd18815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.934+0000 7fd18e1ee640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd18806de10 0x7fd1880705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51708/0 (socket says 172.21.15.106:51708) 2023-11-24T08:09:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.934+0000 7fd18e1ee640 1 -- 172.21.15.106:0/2886500989 learned_addr learned my addr 172.21.15.106:0/2886500989 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.934+0000 7fd18d9ed640 1 -- 172.21.15.106:0/2886500989 >> v1:172.21.15.156:6789/0 conn(0x7fd1880c49f0 legacy=0x7fd18806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.934+0000 7fd18d9ed640 1 -- 172.21.15.106:0/2886500989 >> v1:172.21.15.106:6789/0 conn(0x7fd18806de10 legacy=0x7fd1880705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.934+0000 7fd18d9ed640 1 -- 172.21.15.106:0/2886500989 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880c51b0 con 0x7fd188072160 2023-11-24T08:09:29.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.934+0000 7fd18c9eb640 1 -- 172.21.15.106:0/2886500989 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1066447737 0 0) 0x7fd180001490 con 0x7fd18806de10 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.935+0000 7fd18d9ed640 1 --2- 172.21.15.106:0/2886500989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd188072160 0x7fd18815e750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd1840097a0 tx=0x7fd18402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2372bf9ff73aebe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.935+0000 7fd18c9eb640 1 -- 172.21.15.106:0/2886500989 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184031830 con 0x7fd188072160 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.935+0000 7fd18c9eb640 1 -- 172.21.15.106:0/2886500989 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd184031990 con 0x7fd188072160 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.935+0000 7fd18c9eb640 1 -- 172.21.15.106:0/2886500989 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd184031d20 con 0x7fd188072160 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 -- 172.21.15.106:0/2886500989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd188072160 msgr2=0x7fd18815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 --2- 172.21.15.106:0/2886500989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd188072160 0x7fd18815e750 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd1840097a0 tx=0x7fd18402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 -- 172.21.15.106:0/2886500989 shutdown_connections 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 --2- 172.21.15.106:0/2886500989 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd188072160 0x7fd18815e750 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 -- 172.21.15.106:0/2886500989 >> 172.21.15.106:0/2886500989 conn(0x7fd1880b0c50 msgr2=0x7fd1880b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:29.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 -- 172.21.15.106:0/2886500989 shutdown_connections 2023-11-24T08:09:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.936+0000 7fd18fc51640 1 -- 172.21.15.106:0/2886500989 wait complete. 2023-11-24T08:09:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.938+0000 7fd18fc51640 1 Processor -- start 2023-11-24T08:09:29.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.938+0000 7fd18fc51640 1 -- start start 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18fc51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd18806de10 0x7fd18809c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18fc51640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd188072160 0x7fd1880999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18fc51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 0x7fd188099f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18fc51640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd188155430 con 0x7fd1880c49f0 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18fc51640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd18809a540 con 0x7fd188072160 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18fc51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1881e3c50 con 0x7fd18806de10 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18e1ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 0x7fd188099f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18e1ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 0x7fd188099f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60484/0 (socket says 172.21.15.106:60484) 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.939+0000 7fd18e1ee640 1 -- 172.21.15.106:0/1117528603 learned_addr learned my addr 172.21.15.106:0/1117528603 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18d9ed640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd18806de10 0x7fd18809c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18d1ec640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd188072160 0x7fd1880999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18e1ee640 1 -- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd18806de10 msgr2=0x7fd18809c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18e1ee640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd18806de10 0x7fd18809c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18e1ee640 1 -- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd188072160 msgr2=0x7fd1880999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18e1ee640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd188072160 0x7fd1880999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18e1ee640 1 -- 172.21.15.106:0/1117528603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd184009400 con 0x7fd1880c49f0 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18d1ec640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd188072160 0x7fd1880999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18e1ee640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 0x7fd188099f10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c7f0 tx=0x7fd18000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000a640 con 0x7fd1880c49f0 2023-11-24T08:09:29.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18000a7c0 con 0x7fd1880c49f0 2023-11-24T08:09:29.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18000ab70 con 0x7fd1880c49f0 2023-11-24T08:09:29.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.940+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1881e3f30 con 0x7fd1880c49f0 2023-11-24T08:09:29.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.941+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd18809a820 con 0x7fd1880c49f0 2023-11-24T08:09:29.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.942+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1881e4450 con 0x7fd1880c49f0 2023-11-24T08:09:29.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.943+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd18000acd0 con 0x7fd1880c49f0 2023-11-24T08:09:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.943+0000 7fd17a7fc640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd16c06ae70 0x7fd16c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.943+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd180088800 con 0x7fd1880c49f0 2023-11-24T08:09:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.944+0000 7fd18d9ed640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd16c06ae70 0x7fd16c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:29.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.944+0000 7fd18d9ed640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd16c06ae70 0x7fd16c06d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd184036000 tx=0x7fd184008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:29.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:29.947+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd180053940 con 0x7fd1880c49f0 2023-11-24T08:09:30.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.136+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd18815ec90 con 0x7fd1880c49f0 2023-11-24T08:09:30.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.137+0000 7fd17a7fc640 1 -- 172.21.15.106:0/1117528603 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd18815ec90 con 0x7fd1880c49f0 2023-11-24T08:09:30.139 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:30.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.139+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd16c06ae70 msgr2=0x7fd16c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.139+0000 7fd18fc51640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd16c06ae70 0x7fd16c06d330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd184036000 tx=0x7fd184008030 comp rx=0 tx=0).stop 2023-11-24T08:09:30.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.139+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 msgr2=0x7fd188099f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.139+0000 7fd18fc51640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 0x7fd188099f10 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c7f0 tx=0x7fd18000ccb0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.139+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 shutdown_connections 2023-11-24T08:09:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1880c49f0 0x7fd188099f10 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd188072160 0x7fd1880999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd16c06ae70 0x7fd16c06d330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 --2- 172.21.15.106:0/1117528603 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd18806de10 0x7fd18809c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 >> 172.21.15.106:0/1117528603 conn(0x7fd1880b0c50 msgr2=0x7fd1880734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 shutdown_connections 2023-11-24T08:09:30.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.140+0000 7fd18fc51640 1 -- 172.21.15.106:0/1117528603 wait complete. 2023-11-24T08:09:30.160 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:30.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.373+0000 7fd152e2e640 1 Processor -- start 2023-11-24T08:09:30.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.373+0000 7fd152e2e640 1 -- start start 2023-11-24T08:09:30.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd152e2e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 0x7fd14c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd152e2e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd14c0cce40 0x7fd14c0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd152e2e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14c0b14a0 con 0x7fd14c15d310 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd152e2e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14c06cd10 con 0x7fd14c0cce40 2023-11-24T08:09:30.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd152e2e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0ae5d0 con 0x7fd14c0b0b40 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd151e2c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd14c15d310 0x7fd14c0ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51716/0 (socket says 172.21.15.106:51716) 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd151e2c640 1 -- 172.21.15.106:0/2142274599 learned_addr learned my addr 172.21.15.106:0/2142274599 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd15162b640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 0x7fd14c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.375+0000 7fd15262d640 1 --2- 172.21.15.106:0/2142274599 >> v2:172.21.15.156:3300/0 conn(0x7fd14c0cce40 0x7fd14c0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433952584 0 0) 0x7fd14c0b14a0 con 0x7fd14c15d310 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd138003520 con 0x7fd14c15d310 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2011504484 0 0) 0x7fd138003520 con 0x7fd14c15d310 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 msgr2=0x7fd14c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 0x7fd14c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 >> v2:172.21.15.156:3300/0 conn(0x7fd14c0cce40 msgr2=0x7fd14c0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 --2- 172.21.15.106:0/2142274599 >> v2:172.21.15.156:3300/0 conn(0x7fd14c0cce40 0x7fd14c0cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c0ae8b0 con 0x7fd14c15d310 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fd148001670 con 0x7fd14c15d310 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 >> v1:172.21.15.106:6789/0 conn(0x7fd14c15d310 legacy=0x7fd14c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 0x7fd138003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd138003ef0 0x7fd1380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd138004d00 0x7fd138005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c06cd10 con 0x7fd138004d00 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0ae5d0 con 0x7fd138003ef0 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.376+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13800b980 con 0x7fd14c0b0b40 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15262d640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd138004d00 0x7fd138005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15162b640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd138003ef0 0x7fd1380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd151e2c640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 0x7fd138003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15262d640 1 -- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 msgr2=0x7fd138003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15262d640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0b0b40 0x7fd138003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15262d640 1 -- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd138003ef0 msgr2=0x7fd1380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15262d640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd138003ef0 0x7fd1380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.377+0000 7fd15262d640 1 -- 172.21.15.106:0/2142274599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd138003520 con 0x7fd138004d00 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.378+0000 7fd15262d640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd138004d00 0x7fd138005180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd144009fc0 tx=0x7fd1380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71f166af77898ab server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.378+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14402cc40 con 0x7fd138004d00 2023-11-24T08:09:30.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.378+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14400f040 con 0x7fd138004d00 2023-11-24T08:09:30.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.378+0000 7fd150e2a640 1 -- 172.21.15.106:0/2142274599 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144034440 con 0x7fd138004d00 2023-11-24T08:09:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd152e2e640 1 -- 172.21.15.106:0/2142274599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd138004d00 msgr2=0x7fd138005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd152e2e640 1 --2- 172.21.15.106:0/2142274599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd138004d00 0x7fd138005180 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd144009fc0 tx=0x7fd1380057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd15262d640 1 -- 172.21.15.106:0/2142274599 reap_dead start 2023-11-24T08:09:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd152e2e640 1 -- 172.21.15.106:0/2142274599 shutdown_connections 2023-11-24T08:09:30.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd152e2e640 1 -- 172.21.15.106:0/2142274599 >> 172.21.15.106:0/2142274599 conn(0x7fd14c0cbb40 msgr2=0x7fd14c0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd152e2e640 1 -- 172.21.15.106:0/2142274599 shutdown_connections 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.379+0000 7fd152e2e640 1 -- 172.21.15.106:0/2142274599 wait complete. 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.381+0000 7fd152e2e640 1 Processor -- start 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.381+0000 7fd152e2e640 1 -- start start 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd152e2e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0c4740 0x7fd14c09bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd152e2e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd14c0c4b20 0x7fd14c1561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd152e2e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 0x7fd14c0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd152e2e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0c4210 con 0x7fd14c1566e0 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd152e2e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c156b60 con 0x7fd14c0c4b20 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd152e2e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c09a830 con 0x7fd14c0c4740 2023-11-24T08:09:30.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd15262d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 0x7fd14c0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd15262d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 0x7fd14c0c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60512/0 (socket says 172.21.15.106:60512) 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd15262d640 1 -- 172.21.15.106:0/207060293 learned_addr learned my addr 172.21.15.106:0/207060293 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.382+0000 7fd151e2c640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0c4740 0x7fd14c09bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15162b640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd14c0c4b20 0x7fd14c1561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15262d640 1 -- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0c4740 msgr2=0x7fd14c09bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15262d640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0c4740 0x7fd14c09bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15262d640 1 -- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd14c0c4b20 msgr2=0x7fd14c1561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15262d640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd14c0c4b20 0x7fd14c1561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15262d640 1 -- 172.21.15.106:0/207060293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c09ab10 con 0x7fd14c1566e0 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd15262d640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 0x7fd14c0c2b20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd14400e230 tx=0x7fd144009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144034d50 con 0x7fd14c1566e0 2023-11-24T08:09:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd144003990 con 0x7fd14c1566e0 2023-11-24T08:09:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c09adf0 con 0x7fd14c1566e0 2023-11-24T08:09:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.383+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd144003d40 con 0x7fd14c1566e0 2023-11-24T08:09:30.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.384+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c09b3b0 con 0x7fd14c1566e0 2023-11-24T08:09:30.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.385+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c0c6190 con 0x7fd14c1566e0 2023-11-24T08:09:30.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.386+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd144004030 con 0x7fd14c1566e0 2023-11-24T08:09:30.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.387+0000 7fd1427fc640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13006ae70 0x7fd13006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.387+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1440ab000 con 0x7fd14c1566e0 2023-11-24T08:09:30.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.387+0000 7fd151e2c640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13006ae70 0x7fd13006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.387+0000 7fd151e2c640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13006ae70 0x7fd13006d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd148001eb0 tx=0x7fd148009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.391+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd14c0c6190 con 0x7fd14c1566e0 2023-11-24T08:09:30.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.583+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd14c0c6190 con 0x7fd14c1566e0 2023-11-24T08:09:30.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.584+0000 7fd1427fc640 1 -- 172.21.15.106:0/207060293 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd14c0c6190 con 0x7fd14c1566e0 2023-11-24T08:09:30.586 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13006ae70 msgr2=0x7fd13006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13006ae70 0x7fd13006d330 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd148001eb0 tx=0x7fd148009040 comp rx=0 tx=0).stop 2023-11-24T08:09:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 msgr2=0x7fd14c0c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 0x7fd14c0c2b20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd14400e230 tx=0x7fd144009750 comp rx=0 tx=0).stop 2023-11-24T08:09:30.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 shutdown_connections 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd14c1566e0 0x7fd14c0c2b20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd14c0c4b20 0x7fd14c1561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13006ae70 0x7fd13006d330 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 --2- 172.21.15.106:0/207060293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14c0c4740 0x7fd14c09bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 >> 172.21.15.106:0/207060293 conn(0x7fd14c0cbb40 msgr2=0x7fd14c0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 shutdown_connections 2023-11-24T08:09:30.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.586+0000 7fd152e2e640 1 -- 172.21.15.106:0/207060293 wait complete. 2023-11-24T08:09:30.606 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:30.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.823+0000 7ff684000640 1 Processor -- start 2023-11-24T08:09:30.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.824+0000 7ff684000640 1 -- start start 2023-11-24T08:09:30.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.825+0000 7ff684000640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff67c0d3a40 0x7ff67c06ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.825+0000 7ff684000640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c15db00 con 0x7ff67c15d720 2023-11-24T08:09:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.825+0000 7ff684000640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0d0ce0 con 0x7ff67c0b1ed0 2023-11-24T08:09:30.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.825+0000 7ff684000640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c06d400 con 0x7ff67c0d3a40 2023-11-24T08:09:30.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.825+0000 7ff681d9c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff67c15d720 0x7ff67c0d3330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51718/0 (socket says 172.21.15.106:51718) 2023-11-24T08:09:30.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.825+0000 7ff681d9c640 1 -- 172.21.15.106:0/2161677988 learned_addr learned my addr 172.21.15.106:0/2161677988 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff68159b640 1 --2- 172.21.15.106:0/2161677988 >> v2:172.21.15.156:3300/0 conn(0x7ff67c0d3a40 0x7ff67c06ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263119051 0 0) 0x7ff67c15db00 con 0x7ff67c15d720 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff664003520 con 0x7ff67c15d720 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3244863178 0 0) 0x7ff664003520 con 0x7ff67c15d720 2023-11-24T08:09:30.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 >> v2:172.21.15.156:3300/0 conn(0x7ff67c0d3a40 msgr2=0x7ff67c06ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 --2- 172.21.15.106:0/2161677988 >> v2:172.21.15.156:3300/0 conn(0x7ff67c0d3a40 0x7ff67c06ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 >> v1:172.21.15.156:6789/0 conn(0x7ff67c0b1ed0 legacy=0x7ff67c06bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0b0100 con 0x7ff67c15d720 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7ff66c0019f0 con 0x7ff67c15d720 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 >> v1:172.21.15.106:6789/0 conn(0x7ff67c15d720 legacy=0x7ff67c0d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff664003970 0x7ff664003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6640047c0 0x7ff664004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664005180 0x7ff66400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c06d400 con 0x7ff6640047c0 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66400c0b0 con 0x7ff664005180 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.826+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66400c270 con 0x7ff664003970 2023-11-24T08:09:30.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68159b640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6640047c0 0x7ff664004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff681d9c640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff664003970 0x7ff664003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68259d640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664005180 0x7ff66400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68159b640 1 -- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff664003970 msgr2=0x7ff664003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68159b640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff664003970 0x7ff664003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68159b640 1 -- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664005180 msgr2=0x7ff66400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68159b640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664005180 0x7ff66400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68159b640 1 -- 172.21.15.106:0/2161677988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff664003520 con 0x7ff6640047c0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.827+0000 7ff68259d640 1 -- 172.21.15.106:0/2161677988 reap_dead start 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.828+0000 7ff68159b640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6640047c0 0x7ff664004c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff670009a90 tx=0x7ff6640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa96204841ee41a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.828+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67002c680 con 0x7ff6640047c0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.828+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67002c7e0 con 0x7ff6640047c0 2023-11-24T08:09:30.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.828+0000 7ff680d9a640 1 -- 172.21.15.106:0/2161677988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67002cb70 con 0x7ff6640047c0 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 -- 172.21.15.106:0/2161677988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6640047c0 msgr2=0x7ff664004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6640047c0 0x7ff664004c40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff670009a90 tx=0x7ff6640057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 -- 172.21.15.106:0/2161677988 shutdown_connections 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 --2- 172.21.15.106:0/2161677988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6640047c0 0x7ff664004c40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 -- 172.21.15.106:0/2161677988 >> 172.21.15.106:0/2161677988 conn(0x7ff67c08af60 msgr2=0x7ff67c09c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 -- 172.21.15.106:0/2161677988 shutdown_connections 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.829+0000 7ff684000640 1 -- 172.21.15.106:0/2161677988 wait complete. 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.831+0000 7ff684000640 1 Processor -- start 2023-11-24T08:09:30.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.831+0000 7ff684000640 1 -- start start 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff684000640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6640047c0 0x7ff67c15f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff684000640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff67c15e840 0x7ff67c154a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff684000640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 0x7ff67c15c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff684000640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15c560 con 0x7ff67c15bba0 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff684000640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15ff80 con 0x7ff6640047c0 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff684000640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c7060 con 0x7ff67c15e840 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff68259d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 0x7ff67c15c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff68259d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 0x7ff67c15c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60536/0 (socket says 172.21.15.106:60536) 2023-11-24T08:09:30.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.832+0000 7ff68259d640 1 -- 172.21.15.106:0/2728531180 learned_addr learned my addr 172.21.15.106:0/2728531180 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff681d9c640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6640047c0 0x7ff67c15f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68159b640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff67c15e840 0x7ff67c154a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68259d640 1 -- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff67c15e840 msgr2=0x7ff67c154a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68259d640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff67c15e840 0x7ff67c154a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68259d640 1 -- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6640047c0 msgr2=0x7ff67c15f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68259d640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6640047c0 0x7ff67c15f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68259d640 1 -- 172.21.15.106:0/2728531180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c0c7300 con 0x7ff67c15bba0 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff68259d640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 0x7ff67c15c020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0d3250 tx=0x7ff67400e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6740054b0 con 0x7ff67c15bba0 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740069e0 con 0x7ff67c15bba0 2023-11-24T08:09:30.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674006d70 con 0x7ff67c15bba0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c0c75e0 con 0x7ff67c15bba0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.833+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c15fc70 con 0x7ff67c15bba0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.835+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c0c7b00 con 0x7ff67c15bba0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.835+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff674007030 con 0x7ff67c15bba0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.836+0000 7ff67a7fc640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff66806ae40 0x7ff66806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.836+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff674088a90 con 0x7ff67c15bba0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.837+0000 7ff681d9c640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff66806ae40 0x7ff66806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:30.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.837+0000 7ff681d9c640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff66806ae40 0x7ff66806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff66c002030 tx=0x7ff66c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:30.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:30.841+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff67c0c7b00 con 0x7ff67c15bba0 2023-11-24T08:09:31.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.029+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff67c0c7b00 con 0x7ff67c15bba0 2023-11-24T08:09:31.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.030+0000 7ff67a7fc640 1 -- 172.21.15.106:0/2728531180 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff67c0c7b00 con 0x7ff67c15bba0 2023-11-24T08:09:31.032 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:31.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff66806ae40 msgr2=0x7ff66806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff66806ae40 0x7ff66806d300 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ff66c002030 tx=0x7ff66c009040 comp rx=0 tx=0).stop 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 msgr2=0x7ff67c15c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 0x7ff67c15c020 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0d3250 tx=0x7ff67400e820 comp rx=0 tx=0).stop 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 shutdown_connections 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff67c15bba0 0x7ff67c15c020 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff66806ae40 0x7ff66806d300 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff67c15e840 0x7ff67c154a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 --2- 172.21.15.106:0/2728531180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6640047c0 0x7ff67c15f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 >> 172.21.15.106:0/2728531180 conn(0x7ff67c08af60 msgr2=0x7ff67c0b0d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 shutdown_connections 2023-11-24T08:09:31.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.033+0000 7ff684000640 1 -- 172.21.15.106:0/2728531180 wait complete. 2023-11-24T08:09:31.054 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:09:31.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.266+0000 7f427e285640 1 Processor -- start 2023-11-24T08:09:31.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.266+0000 7f427e285640 1 -- start start 2023-11-24T08:09:31.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427e285640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4278151200 0x7f42780b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:31.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427e285640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f42780aebd0 0x7f427806b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:31.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427e285640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427815de40 con 0x7f42780af010 2023-11-24T08:09:31.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427e285640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427815eac0 con 0x7f42780aebd0 2023-11-24T08:09:31.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427e285640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f427815e120 con 0x7f4278151200 2023-11-24T08:09:31.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427ca82640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f42780aebd0 0x7f427806b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:31.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427ca82640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f42780aebd0 0x7f427806b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60544/0 (socket says 172.21.15.106:60544) 2023-11-24T08:09:31.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427ca82640 1 -- 172.21.15.106:0/2272737477 learned_addr learned my addr 172.21.15.106:0/2272737477 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:31.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.268+0000 7f427d283640 1 --2- 172.21.15.106:0/2272737477 >> v2:172.21.15.156:3300/0 conn(0x7f4278151200 0x7f42780b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:31.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.269+0000 7f427ca82640 1 -- 172.21.15.106:0/2272737477 >> v2:172.21.15.156:3300/0 conn(0x7f4278151200 msgr2=0x7f42780b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:31.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.269+0000 7f427ca82640 1 --2- 172.21.15.106:0/2272737477 >> v2:172.21.15.156:3300/0 conn(0x7f4278151200 0x7f42780b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.269+0000 7f427ca82640 1 -- 172.21.15.106:0/2272737477 >> v1:172.21.15.156:6789/0 conn(0x7f42780af010 legacy=0x7f427815a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:31.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.269+0000 7f427ca82640 1 -- 172.21.15.106:0/2272737477 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427815e400 con 0x7f42780aebd0 2023-11-24T08:09:31.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.269+0000 7f427d283640 1 --2- 172.21.15.106:0/2272737477 >> v2:172.21.15.156:3300/0 conn(0x7f4278151200 0x7f42780b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:31.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.270+0000 7f427ca82640 1 --2- 172.21.15.106:0/2272737477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f427806b610 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f42780721e0 tx=0x7f426c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a41efddb4fba02 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:31.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.270+0000 7f4277fff640 1 -- 172.21.15.106:0/2272737477 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c031870 con 0x7f42780aebd0 2023-11-24T08:09:31.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.270+0000 7f4277fff640 1 -- 172.21.15.106:0/2272737477 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426c0319d0 con 0x7f42780aebd0 2023-11-24T08:09:31.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.270+0000 7f4277fff640 1 -- 172.21.15.106:0/2272737477 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c031d60 con 0x7f42780aebd0 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 -- 172.21.15.106:0/2272737477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 msgr2=0x7f427806b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 --2- 172.21.15.106:0/2272737477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f427806b610 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f42780721e0 tx=0x7f426c02b910 comp rx=0 tx=0).stop 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 -- 172.21.15.106:0/2272737477 shutdown_connections 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 --2- 172.21.15.106:0/2272737477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f427806b610 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 --2- 172.21.15.106:0/2272737477 >> v2:172.21.15.156:3300/0 conn(0x7f4278151200 0x7f42780b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 -- 172.21.15.106:0/2272737477 >> 172.21.15.106:0/2272737477 conn(0x7f42780cf960 msgr2=0x7f42780cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 -- 172.21.15.106:0/2272737477 shutdown_connections 2023-11-24T08:09:31.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.271+0000 7f427e285640 1 -- 172.21.15.106:0/2272737477 wait complete. 2023-11-24T08:09:31.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.273+0000 7f427e285640 1 Processor -- start 2023-11-24T08:09:31.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.273+0000 7f427e285640 1 -- start start 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427e285640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f4278147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427e285640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42780af010 0x7f4278148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427e285640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4278151200 0x7f427814c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427e285640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780b1320 con 0x7f42780aebd0 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427e285640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780b0710 con 0x7f4278151200 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427e285640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780b09f0 con 0x7f42780af010 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427d283640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f4278147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427d283640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f4278147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60552/0 (socket says 172.21.15.106:60552) 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.274+0000 7f427d283640 1 -- 172.21.15.106:0/658195047 learned_addr learned my addr 172.21.15.106:0/658195047 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:31.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427ca82640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42780af010 0x7f4278148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:31.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427da84640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4278151200 0x7f427814c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:31.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427d283640 1 -- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42780af010 msgr2=0x7f4278148e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427d283640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42780af010 0x7f4278148e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427d283640 1 -- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4278151200 msgr2=0x7f427814c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427d283640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4278151200 0x7f427814c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427d283640 1 -- 172.21.15.106:0/658195047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0094d0 con 0x7f42780aebd0 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427ca82640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42780af010 0x7f4278148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f427d283640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f4278147310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f426800c7f0 tx=0x7f426800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.275+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268007820 con 0x7f42780aebd0 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.276+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4268007980 con 0x7f42780aebd0 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.276+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4268007cf0 con 0x7f42780aebd0 2023-11-24T08:09:31.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.276+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42780b0cd0 con 0x7f42780aebd0 2023-11-24T08:09:31.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.276+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4278149360 con 0x7f42780aebd0 2023-11-24T08:09:31.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.277+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427804e840 con 0x7f42780aebd0 2023-11-24T08:09:31.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.278+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f42680113f0 con 0x7f42780aebd0 2023-11-24T08:09:31.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.279+0000 7f4275ffb640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f426406ae40 0x7f426406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:31.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.279+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4268089890 con 0x7f42780aebd0 2023-11-24T08:09:31.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.279+0000 7f427ca82640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f426406ae40 0x7f426406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:31.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.280+0000 7f427ca82640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f426406ae40 0x7f426406d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f426c036000 tx=0x7f426c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:31.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.283+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42680549d0 con 0x7f42780aebd0 2023-11-24T08:09:31.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.486+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4278159950 con 0x7f42780aebd0 2023-11-24T08:09:31.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.487+0000 7f4275ffb640 1 -- 172.21.15.106:0/658195047 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f426801f080 con 0x7f42780aebd0 2023-11-24T08:09:31.489 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:09:31.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f426406ae40 msgr2=0x7f426406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f426406ae40 0x7f426406d300 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f426c036000 tx=0x7f426c03c040 comp rx=0 tx=0).stop 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 msgr2=0x7f4278147310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f4278147310 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f426800c7f0 tx=0x7f426800ccb0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 shutdown_connections 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4278151200 0x7f427814c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f426406ae40 0x7f426406d300 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42780af010 0x7f4278148e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 --2- 172.21.15.106:0/658195047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42780aebd0 0x7f4278147310 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 >> 172.21.15.106:0/658195047 conn(0x7f42780cf960 msgr2=0x7f4278159e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 shutdown_connections 2023-11-24T08:09:31.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:31.490+0000 7f427e285640 1 -- 172.21.15.106:0/658195047 wait complete. 2023-11-24T08:09:31.510 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:09:31.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:31.510 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-24T08:09:31.607 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:09:31.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:31.607 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:09:31.674 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:31.674 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:31.735 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:31.735 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:31.787 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:31.787 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:31.836 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:32.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.051+0000 7ff0c985c640 1 Processor -- start 2023-11-24T08:09:32.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.051+0000 7ff0c985c640 1 -- start start 2023-11-24T08:09:32.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.053+0000 7ff0c985c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff0c415df80 0x7ff0c40ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.053+0000 7ff0c985c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c415e3f0 con 0x7ff0c40b2250 2023-11-24T08:09:32.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.053+0000 7ff0c985c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c415ab80 con 0x7ff0c40cd470 2023-11-24T08:09:32.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.053+0000 7ff0c985c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c415a990 con 0x7ff0c415df80 2023-11-24T08:09:32.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.053+0000 7ff0c2ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff0c40b2250 0x7ff0c40b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51726/0 (socket says 172.21.15.106:51726) 2023-11-24T08:09:32.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.053+0000 7ff0c2ffd640 1 -- 172.21.15.106:0/3800676767 learned_addr learned my addr 172.21.15.106:0/3800676767 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1957348992 0 0) 0x7ff0c415e3f0 con 0x7ff0c40b2250 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ac005730 con 0x7ff0c40b2250 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c27fc640 1 --2- 172.21.15.106:0/3800676767 >> v2:172.21.15.156:3300/0 conn(0x7ff0c415df80 0x7ff0c40ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21166402 0 0) 0x7ff0ac005730 con 0x7ff0c40b2250 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 >> v2:172.21.15.156:3300/0 conn(0x7ff0c415df80 msgr2=0x7ff0c40ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 --2- 172.21.15.106:0/3800676767 >> v2:172.21.15.156:3300/0 conn(0x7ff0c415df80 0x7ff0c40ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 >> v1:172.21.15.156:6789/0 conn(0x7ff0c40cd470 legacy=0x7ff0c415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40b2d40 con 0x7ff0c40b2250 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7ff0b4001670 con 0x7ff0c40b2250 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 >> v1:172.21.15.106:6789/0 conn(0x7ff0c40b2250 legacy=0x7ff0c40b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac005b80 0x7ff0ac005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac0069d0 0x7ff0ac006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ac007390 0x7ff0ac00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00e2b0 con 0x7ff0ac007390 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00e420 con 0x7ff0ac005b80 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.054+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00e5e0 con 0x7ff0ac0069d0 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ac007390 0x7ff0ac00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c27fc640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac0069d0 0x7ff0ac006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c2ffd640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac005b80 0x7ff0ac005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c37fe640 1 -- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac0069d0 msgr2=0x7ff0ac006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac0069d0 0x7ff0ac006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c37fe640 1 -- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac005b80 msgr2=0x7ff0ac005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac005b80 0x7ff0ac005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c37fe640 1 -- 172.21.15.106:0/3800676767 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac005730 con 0x7ff0ac007390 2023-11-24T08:09:32.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.055+0000 7ff0c2ffd640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0ac005b80 0x7ff0ac005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.056+0000 7ff0c37fe640 1 -- 172.21.15.106:0/3800676767 reap_dead start 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.056+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ac007390 0x7ff0ac00db90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001e50 tx=0x7ff0ac007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5440f183968adb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.056+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc003b70 con 0x7ff0ac007390 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.056+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc003d10 con 0x7ff0ac007390 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.056+0000 7ff0c1ffb640 1 -- 172.21.15.106:0/3800676767 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc005310 con 0x7ff0ac007390 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 -- 172.21.15.106:0/3800676767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ac007390 msgr2=0x7ff0ac00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ac007390 0x7ff0ac00db90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001e50 tx=0x7ff0ac007a00 comp rx=0 tx=0).stop 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 -- 172.21.15.106:0/3800676767 shutdown_connections 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 --2- 172.21.15.106:0/3800676767 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0ac007390 0x7ff0ac00db90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 -- 172.21.15.106:0/3800676767 >> 172.21.15.106:0/3800676767 conn(0x7ff0c406a920 msgr2=0x7ff0c406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:32.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 -- 172.21.15.106:0/3800676767 shutdown_connections 2023-11-24T08:09:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.057+0000 7ff0c985c640 1 -- 172.21.15.106:0/3800676767 wait complete. 2023-11-24T08:09:32.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.059+0000 7ff0c985c640 1 Processor -- start 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.059+0000 7ff0c985c640 1 -- start start 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c985c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac007390 0x7ff0c4156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c985c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c40c93b0 0x7ff0c40c9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c985c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 0x7ff0c40af0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c985c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40c9d70 con 0x7ff0c40aec30 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c985c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40b0a00 con 0x7ff0c40c93b0 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c985c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c40af960 con 0x7ff0ac007390 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 0x7ff0c40af0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 0x7ff0c40af0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60572/0 (socket says 172.21.15.106:60572) 2023-11-24T08:09:32.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.060+0000 7ff0c37fe640 1 -- 172.21.15.106:0/2192469928 learned_addr learned my addr 172.21.15.106:0/2192469928 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:32.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c2ffd640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac007390 0x7ff0c4156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c27fc640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c40c93b0 0x7ff0c40c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c37fe640 1 -- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac007390 msgr2=0x7ff0c4156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac007390 0x7ff0c4156fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c37fe640 1 -- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c40c93b0 msgr2=0x7ff0c40c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c40c93b0 0x7ff0c40c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c37fe640 1 -- 172.21.15.106:0/2192469928 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40afc40 con 0x7ff0c40aec30 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c37fe640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 0x7ff0c40af0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001f70 tx=0x7ff0bc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc006d50 con 0x7ff0c40aec30 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc003b80 con 0x7ff0c40aec30 2023-11-24T08:09:32.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c40aff20 con 0x7ff0c40aec30 2023-11-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0396c0 con 0x7ff0c40aec30 2023-11-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.061+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c40b04e0 con 0x7ff0c40aec30 2023-11-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.062+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c4067470 con 0x7ff0c40aec30 2023-11-24T08:09:32.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.063+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff0bc039820 con 0x7ff0c40aec30 2023-11-24T08:09:32.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.064+0000 7ff0abfff640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b006ae70 0x7ff0b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.064+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0bc0ab460 con 0x7ff0c40aec30 2023-11-24T08:09:32.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.065+0000 7ff0c2ffd640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b006ae70 0x7ff0b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.065+0000 7ff0c2ffd640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b006ae70 0x7ff0b006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4001eb0 tx=0x7ff0b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.068+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0bc076520 con 0x7ff0c40aec30 2023-11-24T08:09:32.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.259+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0c40adfc0 con 0x7ff0c40aec30 2023-11-24T08:09:32.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.262+0000 7ff0abfff640 1 -- 172.21.15.106:0/2192469928 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16553 (secure 0 0 0) 0x7ff0bc079b40 con 0x7ff0c40aec30 2023-11-24T08:09:32.264 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:32.264 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":22,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:08:49.114397+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:32.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.265+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b006ae70 msgr2=0x7ff0b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.265+0000 7ff0c985c640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b006ae70 0x7ff0b006d330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4001eb0 tx=0x7ff0b4009040 comp rx=0 tx=0).stop 2023-11-24T08:09:32.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.265+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 msgr2=0x7ff0c40af0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.265+0000 7ff0c985c640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 0x7ff0c40af0e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001f70 tx=0x7ff0bc0014a0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 shutdown_connections 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff0c40aec30 0x7ff0c40af0e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c40c93b0 0x7ff0c40c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff0b006ae70 0x7ff0b006d330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 --2- 172.21.15.106:0/2192469928 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff0ac007390 0x7ff0c4156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 >> 172.21.15.106:0/2192469928 conn(0x7ff0c406a920 msgr2=0x7ff0c40ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 shutdown_connections 2023-11-24T08:09:32.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.266+0000 7ff0c985c640 1 -- 172.21.15.106:0/2192469928 wait complete. 2023-11-24T08:09:32.287 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:32.288 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:32.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.496+0000 7f639e69b640 1 Processor -- start 2023-11-24T08:09:32.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.497+0000 7f639e69b640 1 -- start start 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.498+0000 7f639e69b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6398140340 0x7f63980ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.498+0000 7f639e69b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6398161080 0x7f63980c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.498+0000 7f639e69b640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6398161600 con 0x7f6398140720 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.498+0000 7f639e69b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980cbf70 con 0x7f6398140340 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.498+0000 7f639e69b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63981618e0 con 0x7f6398161080 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.498+0000 7f6397fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6398140340 0x7f63980ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.499+0000 7f6397fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6398140340 0x7f63980ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60580/0 (socket says 172.21.15.106:60580) 2023-11-24T08:09:32.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.499+0000 7f6397fff640 1 -- 172.21.15.106:0/767296310 learned_addr learned my addr 172.21.15.106:0/767296310 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:32.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.499+0000 7f6397fff640 1 -- 172.21.15.106:0/767296310 >> v2:172.21.15.156:3300/0 conn(0x7f6398161080 msgr2=0x7f63980c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.499+0000 7f6397fff640 1 --2- 172.21.15.106:0/767296310 >> v2:172.21.15.156:3300/0 conn(0x7f6398161080 0x7f63980c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.499+0000 7f6397fff640 1 -- 172.21.15.106:0/767296310 >> v1:172.21.15.156:6789/0 conn(0x7f6398140720 legacy=0x7f63980cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.499+0000 7f6397fff640 1 -- 172.21.15.106:0/767296310 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6398161bc0 con 0x7f6398140340 2023-11-24T08:09:32.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.500+0000 7f6397fff640 1 --2- 172.21.15.106:0/767296310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980ce410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63980c6d20 tx=0x7f638802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d560f2dfca52401d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.500+0000 7f6396ffd640 1 -- 172.21.15.106:0/767296310 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638802ca80 con 0x7f6398140340 2023-11-24T08:09:32.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.500+0000 7f6396ffd640 1 -- 172.21.15.106:0/767296310 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638802cbe0 con 0x7f6398140340 2023-11-24T08:09:32.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.500+0000 7f6396ffd640 1 -- 172.21.15.106:0/767296310 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63880343f0 con 0x7f6398140340 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 -- 172.21.15.106:0/767296310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 msgr2=0x7f63980ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 --2- 172.21.15.106:0/767296310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980ce410 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f63980c6d20 tx=0x7f638802bee0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 -- 172.21.15.106:0/767296310 shutdown_connections 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 --2- 172.21.15.106:0/767296310 >> v2:172.21.15.156:3300/0 conn(0x7f6398161080 0x7f63980c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 --2- 172.21.15.106:0/767296310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980ce410 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 -- 172.21.15.106:0/767296310 >> 172.21.15.106:0/767296310 conn(0x7f63980b2e40 msgr2=0x7f63980b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 -- 172.21.15.106:0/767296310 shutdown_connections 2023-11-24T08:09:32.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.501+0000 7f639e69b640 1 -- 172.21.15.106:0/767296310 wait complete. 2023-11-24T08:09:32.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.503+0000 7f639e69b640 1 Processor -- start 2023-11-24T08:09:32.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.503+0000 7f639e69b640 1 -- start start 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f639e69b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f639e69b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398140720 0x7f63981f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f639e69b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6398161080 0x7f63981f3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f639e69b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639815e260 con 0x7f6398140340 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f639e69b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980b59c0 con 0x7f6398140720 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f639e69b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63980b56e0 con 0x7f6398161080 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.504+0000 7f6397fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980b4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60582/0 (socket says 172.21.15.106:60582) 2023-11-24T08:09:32.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 -- 172.21.15.106:0/3350818073 learned_addr learned my addr 172.21.15.106:0/3350818073 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f63977fe640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398140720 0x7f63981f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 -- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6398161080 msgr2=0x7f63981f3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f639cc38640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6398161080 0x7f63981f3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6398161080 0x7f63981f3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 -- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398140720 msgr2=0x7f63981f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398140720 0x7f63981f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 -- 172.21.15.106:0/3350818073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388009b30 con 0x7f6398140340 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f63977fe640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398140720 0x7f63981f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.505+0000 7f6397fff640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980b4e50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63880093e0 tx=0x7f6388004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.506+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638803e450 con 0x7f6398140340 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.506+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638803e5b0 con 0x7f6398140340 2023-11-24T08:09:32.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.506+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638803e960 con 0x7f6398140340 2023-11-24T08:09:32.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.506+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639815dc10 con 0x7f6398140340 2023-11-24T08:09:32.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.506+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981f0ac0 con 0x7f6398140340 2023-11-24T08:09:32.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.507+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398067990 con 0x7f6398140340 2023-11-24T08:09:32.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.508+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f638802cd50 con 0x7f6398140340 2023-11-24T08:09:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.508+0000 7f6394ff9640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637806ae40 0x7f637806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.509+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63880ac2f0 con 0x7f6398140340 2023-11-24T08:09:32.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.509+0000 7f63977fe640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637806ae40 0x7f637806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.509+0000 7f63977fe640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637806ae40 0x7f637806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6380007fd0 tx=0x7f6380009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.513+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6388077390 con 0x7f6398140340 2023-11-24T08:09:32.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.701+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f63980c6570 con 0x7f6398140340 2023-11-24T08:09:32.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.702+0000 7f6394ff9640 1 -- 172.21.15.106:0/3350818073 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f638803eac0 con 0x7f6398140340 2023-11-24T08:09:32.703 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637806ae40 msgr2=0x7f637806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637806ae40 0x7f637806d300 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6380007fd0 tx=0x7f6380009040 comp rx=0 tx=0).stop 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 msgr2=0x7f63980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980b4e50 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63880093e0 tx=0x7f6388004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 shutdown_connections 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637806ae40 0x7f637806d300 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6398161080 0x7f63981f3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398140720 0x7f63981f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 --2- 172.21.15.106:0/3350818073 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6398140340 0x7f63980b4e50 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 >> 172.21.15.106:0/3350818073 conn(0x7f63980b2e40 msgr2=0x7f639806cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:32.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 shutdown_connections 2023-11-24T08:09:32.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.705+0000 7f639e69b640 1 -- 172.21.15.106:0/3350818073 wait complete. 2023-11-24T08:09:32.726 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.945+0000 7f6aabfff640 1 Processor -- start 2023-11-24T08:09:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.945+0000 7f6aabfff640 1 -- start start 2023-11-24T08:09:32.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.947+0000 7f6aabfff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6aac0b0b40 0x7f6aac0b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.947+0000 7f6aabfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6aac0cce40 0x7f6aac0cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.947+0000 7f6aabfff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6aac0b14d0 con 0x7f6aac15d310 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.947+0000 7f6aabfff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac06cd10 con 0x7f6aac0cce40 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.947+0000 7f6aabfff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6aac0ae5d0 con 0x7f6aac0b0b40 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.947+0000 7f6aab7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6aac0cce40 0x7f6aac0cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aab7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6aac0cce40 0x7f6aac0cd2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60598/0 (socket says 172.21.15.106:60598) 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aab7fe640 1 -- 172.21.15.106:0/10148227 learned_addr learned my addr 172.21.15.106:0/10148227 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aa3fff640 1 --2- 172.21.15.106:0/10148227 >> v2:172.21.15.156:3300/0 conn(0x7f6aac0b0b40 0x7f6aac0b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aab7fe640 1 -- 172.21.15.106:0/10148227 >> v2:172.21.15.156:3300/0 conn(0x7f6aac0b0b40 msgr2=0x7f6aac0b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aab7fe640 1 --2- 172.21.15.106:0/10148227 >> v2:172.21.15.156:3300/0 conn(0x7f6aac0b0b40 0x7f6aac0b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aab7fe640 1 -- 172.21.15.106:0/10148227 >> v1:172.21.15.156:6789/0 conn(0x7f6aac15d310 legacy=0x7f6aac0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:32.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.948+0000 7f6aab7fe640 1 -- 172.21.15.106:0/10148227 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aac0ae8b0 con 0x7f6aac0cce40 2023-11-24T08:09:32.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.949+0000 7f6aab7fe640 1 --2- 172.21.15.106:0/10148227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac0cce40 0x7f6aac0cd2a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0b1cb0 tx=0x7f6aa402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9038385ff51bcb3a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.950+0000 7f6aaa7fc640 1 -- 172.21.15.106:0/10148227 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa402ca80 con 0x7f6aac0cce40 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.950+0000 7f6aaa7fc640 1 -- 172.21.15.106:0/10148227 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa402cbe0 con 0x7f6aac0cce40 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.950+0000 7f6aaa7fc640 1 -- 172.21.15.106:0/10148227 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa40343f0 con 0x7f6aac0cce40 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 -- 172.21.15.106:0/10148227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac0cce40 msgr2=0x7f6aac0cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 --2- 172.21.15.106:0/10148227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac0cce40 0x7f6aac0cd2a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6aac0b1cb0 tx=0x7f6aa402bee0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 -- 172.21.15.106:0/10148227 shutdown_connections 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 --2- 172.21.15.106:0/10148227 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac0cce40 0x7f6aac0cd2a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 --2- 172.21.15.106:0/10148227 >> v2:172.21.15.156:3300/0 conn(0x7f6aac0b0b40 0x7f6aac0b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 -- 172.21.15.106:0/10148227 >> 172.21.15.106:0/10148227 conn(0x7f6aac0cbb40 msgr2=0x7f6aac0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 -- 172.21.15.106:0/10148227 shutdown_connections 2023-11-24T08:09:32.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.951+0000 7f6aabfff640 1 -- 172.21.15.106:0/10148227 wait complete. 2023-11-24T08:09:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.953+0000 7f6aabfff640 1 Processor -- start 2023-11-24T08:09:32.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.953+0000 7f6aabfff640 1 -- start start 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.954+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6aac0b0b40 0x7f6aac0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.954+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aac0cce40 0x7f6aac14b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.954+0000 7f6aabfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 0x7f6aac0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.954+0000 7f6aabfff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac099eb0 con 0x7f6aac15d310 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aabfff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac09a190 con 0x7f6aac0cce40 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aabfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6aac09a470 con 0x7f6aac0b0b40 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 0x7f6aac0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 0x7f6aac0c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60606/0 (socket says 172.21.15.106:60606) 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 -- 172.21.15.106:0/3385374857 learned_addr learned my addr 172.21.15.106:0/3385374857 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aa3fff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aac0cce40 0x7f6aac14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aaaffd640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6aac0b0b40 0x7f6aac0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 -- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6aac0b0b40 msgr2=0x7f6aac0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6aac0b0b40 0x7f6aac0c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 -- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aac0cce40 msgr2=0x7f6aac14b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aac0cce40 0x7f6aac14b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:32.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.955+0000 7f6aab7fe640 1 -- 172.21.15.106:0/3385374857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa4009b30 con 0x7f6aac15d310 2023-11-24T08:09:32.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.956+0000 7f6aab7fe640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 0x7f6aac0c7c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4036040 tx=0x7f6aa4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.956+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa4009510 con 0x7f6aac15d310 2023-11-24T08:09:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.956+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6aa403e450 con 0x7f6aac15d310 2023-11-24T08:09:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.956+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6aac09a750 con 0x7f6aac15d310 2023-11-24T08:09:32.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.956+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6aa403e7e0 con 0x7f6aac15d310 2023-11-24T08:09:32.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.956+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6aac0c8e60 con 0x7f6aac15d310 2023-11-24T08:09:32.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.957+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6aac09ac70 con 0x7f6aac15d310 2023-11-24T08:09:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.958+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6aa402cd50 con 0x7f6aac15d310 2023-11-24T08:09:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.959+0000 7f6aa8ff9640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6a8c06ae40 0x7f6a8c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.959+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6aa40ac0a0 con 0x7f6aac15d310 2023-11-24T08:09:32.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.959+0000 7f6aaaffd640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6a8c06ae40 0x7f6a8c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:32.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.960+0000 7f6aaaffd640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6a8c06ae40 0x7f6a8c06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c006fd0 tx=0x7f6a9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:32.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:32.964+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6aac09ac70 con 0x7f6aac15d310 2023-11-24T08:09:33.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.152+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6aac0b14d0 con 0x7f6aac15d310 2023-11-24T08:09:33.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.153+0000 7f6aa8ff9640 1 -- 172.21.15.106:0/3385374857 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6aac0b14d0 con 0x7f6aac15d310 2023-11-24T08:09:33.155 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:33.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.155+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6a8c06ae40 msgr2=0x7f6a8c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.155+0000 7f6aabfff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6a8c06ae40 0x7f6a8c06d300 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c006fd0 tx=0x7f6a9c008040 comp rx=0 tx=0).stop 2023-11-24T08:09:33.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.155+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 msgr2=0x7f6aac0c7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.155+0000 7f6aabfff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 0x7f6aac0c7c80 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4036040 tx=0x7f6aa4004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 shutdown_connections 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6aac15d310 0x7f6aac0c7c80 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6aac0cce40 0x7f6aac14b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6a8c06ae40 0x7f6a8c06d300 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 --2- 172.21.15.106:0/3385374857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6aac0b0b40 0x7f6aac0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 >> 172.21.15.106:0/3385374857 conn(0x7f6aac0cbb40 msgr2=0x7f6aac0cd220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 shutdown_connections 2023-11-24T08:09:33.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.156+0000 7f6aabfff640 1 -- 172.21.15.106:0/3385374857 wait complete. 2023-11-24T08:09:33.177 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:33.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.391+0000 7f8e88bcf640 1 Processor -- start 2023-11-24T08:09:33.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.391+0000 7f8e88bcf640 1 -- start start 2023-11-24T08:09:33.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.392+0000 7f8e88bcf640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8e8415b260 0x7f8e8406e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.392+0000 7f8e88bcf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e8406e580 0x7f8e840b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.392+0000 7f8e88bcf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.392+0000 7f8e88bcf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e8415b640 con 0x7f8e8406e580 2023-11-24T08:09:33.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.392+0000 7f8e88bcf640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e840b1510 con 0x7f8e8415b260 2023-11-24T08:09:33.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.392+0000 7f8e88bcf640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8415ed20 con 0x7f8e840b1080 2023-11-24T08:09:33.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e8406e580 0x7f8e840b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e8406e580 0x7f8e840b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60618/0 (socket says 172.21.15.106:60618) 2023-11-24T08:09:33.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 -- 172.21.15.106:0/66777291 learned_addr learned my addr 172.21.15.106:0/66777291 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:33.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e83fff640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 -- 172.21.15.106:0/66777291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 msgr2=0x7f8e840ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:33.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e837fe640 1 --2- 172.21.15.106:0/66777291 >> v2:172.21.15.156:3300/0 conn(0x7f8e8415b260 0x7f8e8406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 -- 172.21.15.106:0/66777291 >> v2:172.21.15.156:3300/0 conn(0x7f8e8415b260 msgr2=0x7f8e8406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 --2- 172.21.15.106:0/66777291 >> v2:172.21.15.156:3300/0 conn(0x7f8e8415b260 0x7f8e8406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e82ffd640 1 -- 172.21.15.106:0/66777291 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e8415f8a0 con 0x7f8e8406e580 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e837fe640 1 --2- 172.21.15.106:0/66777291 >> v2:172.21.15.156:3300/0 conn(0x7f8e8415b260 0x7f8e8406e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.393+0000 7f8e83fff640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.394+0000 7f8e82ffd640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8406e580 0x7f8e840b0b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e840d0160 tx=0x7f8e7802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e4d234cfb5e650e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.395+0000 7f8e827fc640 1 -- 172.21.15.106:0/66777291 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7802c960 con 0x7f8e8406e580 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.395+0000 7f8e827fc640 1 -- 172.21.15.106:0/66777291 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7802cac0 con 0x7f8e8406e580 2023-11-24T08:09:33.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.395+0000 7f8e827fc640 1 -- 172.21.15.106:0/66777291 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7802ce50 con 0x7f8e8406e580 2023-11-24T08:09:33.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.395+0000 7f8e88bcf640 1 -- 172.21.15.106:0/66777291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8406e580 msgr2=0x7f8e840b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.395+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8406e580 0x7f8e840b0b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8e840d0160 tx=0x7f8e7802bdc0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 -- 172.21.15.106:0/66777291 shutdown_connections 2023-11-24T08:09:33.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/66777291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8406e580 0x7f8e840b0b40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/66777291 >> v2:172.21.15.156:3300/0 conn(0x7f8e8415b260 0x7f8e8406e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 -- 172.21.15.106:0/66777291 >> 172.21.15.106:0/66777291 conn(0x7f8e840cbb40 msgr2=0x7f8e840cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:33.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 -- 172.21.15.106:0/66777291 shutdown_connections 2023-11-24T08:09:33.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.396+0000 7f8e88bcf640 1 -- 172.21.15.106:0/66777291 wait complete. 2023-11-24T08:09:33.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.397+0000 7f8e88bcf640 1 Processor -- start 2023-11-24T08:09:33.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.398+0000 7f8e88bcf640 1 -- start start 2023-11-24T08:09:33.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.398+0000 7f8e88bcf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e8406e580 0x7f8e84149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.398+0000 7f8e88bcf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e88bcf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 0x7f8e840b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e88bcf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e840b2170 con 0x7f8e8415b260 2023-11-24T08:09:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e88bcf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e8409c850 con 0x7f8e8406e580 2023-11-24T08:09:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e88bcf640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e84149ab0 con 0x7f8e840b1080 2023-11-24T08:09:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 0x7f8e840b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 0x7f8e840b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60622/0 (socket says 172.21.15.106:60622) 2023-11-24T08:09:33.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 -- 172.21.15.106:0/4134578768 learned_addr learned my addr 172.21.15.106:0/4134578768 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:33.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e837fe640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e8406e580 0x7f8e84149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 -- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 msgr2=0x7f8e840b3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 -- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e8406e580 msgr2=0x7f8e84149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e8406e580 0x7f8e84149570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 -- 172.21.15.106:0/4134578768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e78009b30 con 0x7f8e8415b260 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.399+0000 7f8e83fff640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 0x7f8e840b41c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c7c0 tx=0x7f8e7c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.400+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c004230 con 0x7f8e8415b260 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.400+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c004390 con 0x7f8e8415b260 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.400+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c007860 con 0x7f8e8415b260 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.400+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e8409afc0 con 0x7f8e8415b260 2023-11-24T08:09:33.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.400+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e840b1ab0 con 0x7f8e8415b260 2023-11-24T08:09:33.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.401+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e840b22b0 con 0x7f8e8415b260 2023-11-24T08:09:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.402+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e7c0079c0 con 0x7f8e8415b260 2023-11-24T08:09:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.403+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e7006ae40 0x7f8e7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.403+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e7c089850 con 0x7f8e8415b260 2023-11-24T08:09:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.403+0000 7f8e837fe640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e7006ae40 0x7f8e7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.404+0000 7f8e837fe640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e7006ae40 0x7f8e7006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400b7c0 tx=0x7f8e74008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:33.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.408+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e840b22b0 con 0x7f8e8415b260 2023-11-24T08:09:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.599+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8e840b22b0 con 0x7f8e8415b260 2023-11-24T08:09:33.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.600+0000 7f8e6bfff640 1 -- 172.21.15.106:0/4134578768 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8e840b22b0 con 0x7f8e8415b260 2023-11-24T08:09:33.602 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e7006ae40 msgr2=0x7f8e7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e7006ae40 0x7f8e7006d300 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e7400b7c0 tx=0x7f8e74008040 comp rx=0 tx=0).stop 2023-11-24T08:09:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 msgr2=0x7f8e840b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 0x7f8e840b41c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c7c0 tx=0x7f8e7c00cc80 comp rx=0 tx=0).stop 2023-11-24T08:09:33.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 shutdown_connections 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e8415b260 0x7f8e840b41c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e7006ae40 0x7f8e7006d300 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e840b1080 0x7f8e840b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.602+0000 7f8e88bcf640 1 --2- 172.21.15.106:0/4134578768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e8406e580 0x7f8e84149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.603+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 >> 172.21.15.106:0/4134578768 conn(0x7f8e840cbb40 msgr2=0x7f8e840af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.603+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 shutdown_connections 2023-11-24T08:09:33.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.603+0000 7f8e88bcf640 1 -- 172.21.15.106:0/4134578768 wait complete. 2023-11-24T08:09:33.623 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:09:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.836+0000 7f9454fc9640 1 Processor -- start 2023-11-24T08:09:33.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.836+0000 7f9454fc9640 1 -- start start 2023-11-24T08:09:33.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.837+0000 7f9454fc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f9450161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.837+0000 7f9454fc9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9450161fa0 0x7f945006ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.837+0000 7f9454fc9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94501612f0 con 0x7f94500b2400 2023-11-24T08:09:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.837+0000 7f9454fc9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500cff70 con 0x7f9450161fa0 2023-11-24T08:09:33.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.837+0000 7f9454fc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500afb20 con 0x7f94500cfb90 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9450161fa0 0x7f945006ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9450161fa0 0x7f945006ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:60212/0 (socket says 172.21.15.106:60212) 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 -- 172.21.15.106:0/1286407780 learned_addr learned my addr 172.21.15.106:0/1286407780 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f944e59c640 1 --2- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f9450161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 -- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 msgr2=0x7f9450161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 --2- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f9450161a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 -- 172.21.15.106:0/1286407780 >> v1:172.21.15.156:6789/0 conn(0x7f94500b2400 legacy=0x7f94500cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:33.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.838+0000 7f9447fff640 1 -- 172.21.15.106:0/1286407780 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945006a850 con 0x7f9450161fa0 2023-11-24T08:09:33.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.840+0000 7f9447fff640 1 --2- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9450161fa0 0x7f945006ee50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94380098a0 tx=0x7f943802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=63846763852f7b96 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:33.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.840+0000 7f944dd9b640 1 -- 172.21.15.106:0/1286407780 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438030840 con 0x7f9450161fa0 2023-11-24T08:09:33.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.840+0000 7f944dd9b640 1 -- 172.21.15.106:0/1286407780 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94380309a0 con 0x7f9450161fa0 2023-11-24T08:09:33.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.840+0000 7f944dd9b640 1 -- 172.21.15.106:0/1286407780 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9438030d30 con 0x7f9450161fa0 2023-11-24T08:09:33.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 -- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9450161fa0 msgr2=0x7f945006ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 --2- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9450161fa0 0x7f945006ee50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f94380098a0 tx=0x7f943802b980 comp rx=0 tx=0).stop 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 -- 172.21.15.106:0/1286407780 shutdown_connections 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 --2- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9450161fa0 0x7f945006ee50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 --2- 172.21.15.106:0/1286407780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f9450161a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 -- 172.21.15.106:0/1286407780 >> 172.21.15.106:0/1286407780 conn(0x7f94500999d0 msgr2=0x7f94500b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 -- 172.21.15.106:0/1286407780 shutdown_connections 2023-11-24T08:09:33.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.841+0000 7f9454fc9640 1 -- 172.21.15.106:0/1286407780 wait complete. 2023-11-24T08:09:33.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.843+0000 7f9454fc9640 1 Processor -- start 2023-11-24T08:09:33.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.843+0000 7f9454fc9640 1 -- start start 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f9454fc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94500b2400 0x7f9450152190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f9454fc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f94500c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f9454fc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 0x7f94500b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f9454fc9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015e790 con 0x7f94500b3fb0 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f9454fc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945015e9a0 con 0x7f94500b2400 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f9454fc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945015ec80 con 0x7f94500cfb90 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 0x7f94500b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f944ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 0x7f94500b56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60624/0 (socket says 172.21.15.106:60624) 2023-11-24T08:09:33.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.844+0000 7f944ed9d640 1 -- 172.21.15.106:0/2702024563 learned_addr learned my addr 172.21.15.106:0/2702024563 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944e59c640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94500b2400 0x7f9450152190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f9447fff640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f94500c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944ed9d640 1 -- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 msgr2=0x7f94500c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944ed9d640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f94500c3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944ed9d640 1 -- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94500b2400 msgr2=0x7f9450152190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944ed9d640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94500b2400 0x7f9450152190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944ed9d640 1 -- 172.21.15.106:0/2702024563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94380094d0 con 0x7f94500b3fb0 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.845+0000 7f944ed9d640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 0x7f94500b56c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f944800b9e0 tx=0x7f944800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.846+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448003b70 con 0x7f94500b3fb0 2023-11-24T08:09:33.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.846+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448003cd0 con 0x7f94500b3fb0 2023-11-24T08:09:33.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.846+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f944800cb60 con 0x7f94500b3fb0 2023-11-24T08:09:33.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.846+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945015ef60 con 0x7f94500b3fb0 2023-11-24T08:09:33.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.846+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945015f520 con 0x7f94500b3fb0 2023-11-24T08:09:33.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.847+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94500cb6a0 con 0x7f94500b3fb0 2023-11-24T08:09:33.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.848+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f944800ccc0 con 0x7f94500b3fb0 2023-11-24T08:09:33.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.849+0000 7f9446ffd640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f943006ae40 0x7f943006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:33.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.849+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9448089490 con 0x7f94500b3fb0 2023-11-24T08:09:33.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.849+0000 7f944e59c640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f943006ae40 0x7f943006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:33.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.850+0000 7f944e59c640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f943006ae40 0x7f943006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9440009ad0 tx=0x7f9440008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:33.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:33.853+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9448054410 con 0x7f94500b3fb0 2023-11-24T08:09:34.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.058+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f945004e8b0 con 0x7f94500b3fb0 2023-11-24T08:09:34.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.059+0000 7f9446ffd640 1 -- 172.21.15.106:0/2702024563 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9448057a30 con 0x7f94500b3fb0 2023-11-24T08:09:34.060 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:09:34.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.061+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f943006ae40 msgr2=0x7f943006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.061+0000 7f9454fc9640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f943006ae40 0x7f943006d300 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f9440009ad0 tx=0x7f9440008040 comp rx=0 tx=0).stop 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.061+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 msgr2=0x7f94500b56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.061+0000 7f9454fc9640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 0x7f94500b56c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f944800b9e0 tx=0x7f944800bea0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 shutdown_connections 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f94500b3fb0 0x7f94500b56c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f943006ae40 0x7f943006d300 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f94500cfb90 0x7f94500c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 --2- 172.21.15.106:0/2702024563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94500b2400 0x7f9450152190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 >> 172.21.15.106:0/2702024563 conn(0x7f94500999d0 msgr2=0x7f94500b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 shutdown_connections 2023-11-24T08:09:34.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.062+0000 7f9454fc9640 1 -- 172.21.15.106:0/2702024563 wait complete. 2023-11-24T08:09:34.081 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:09:34.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:34.082 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-24T08:09:34.191 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:09:34.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:34.191 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:09:34.257 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:34.257 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:34.316 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:34.317 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:34.359 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:09:34.359 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:34.459 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:34.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.672+0000 7fdda7c56640 1 Processor -- start 2023-11-24T08:09:34.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.673+0000 7fdda7c56640 1 -- start start 2023-11-24T08:09:34.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.674+0000 7fdda7c56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdda00b2250 0x7fdda015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:34.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.674+0000 7fdda7c56640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fdda00b2630 0x7fdda00ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:34.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.674+0000 7fdda7c56640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdda015ab80 con 0x7fdda00b2aa0 2023-11-24T08:09:34.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.674+0000 7fdda7c56640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda015a990 con 0x7fdda00b2250 2023-11-24T08:09:34.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.674+0000 7fdda7c56640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdda015e680 con 0x7fdda00b2630 2023-11-24T08:09:34.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.674+0000 7fdda59f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdda00b2250 0x7fdda015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:34.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.675+0000 7fdda59f2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdda00b2250 0x7fdda015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51590/0 (socket says 172.21.15.106:51590) 2023-11-24T08:09:34.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.675+0000 7fdda59f2640 1 -- 172.21.15.106:0/3712559209 learned_addr learned my addr 172.21.15.106:0/3712559209 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:34.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.675+0000 7fdda59f2640 1 -- 172.21.15.106:0/3712559209 >> v2:172.21.15.156:3300/0 conn(0x7fdda00b2630 msgr2=0x7fdda00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:34.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.675+0000 7fdda59f2640 1 --2- 172.21.15.106:0/3712559209 >> v2:172.21.15.156:3300/0 conn(0x7fdda00b2630 0x7fdda00ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.675+0000 7fdda59f2640 1 -- 172.21.15.106:0/3712559209 >> v1:172.21.15.156:6789/0 conn(0x7fdda00b2aa0 legacy=0x7fdda015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:34.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.675+0000 7fdda59f2640 1 -- 172.21.15.106:0/3712559209 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda015e960 con 0x7fdda00b2250 2023-11-24T08:09:34.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.676+0000 7fdda59f2640 1 --2- 172.21.15.106:0/3712559209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2250 0x7fdda015df80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdda00ae5c0 tx=0x7fdd9c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc23e2e43f7e5852 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.676+0000 7fdda49f0640 1 -- 172.21.15.106:0/3712559209 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c02ee90 con 0x7fdda00b2250 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.676+0000 7fdda49f0640 1 -- 172.21.15.106:0/3712559209 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd9c036440 con 0x7fdda00b2250 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.676+0000 7fdda49f0640 1 -- 172.21.15.106:0/3712559209 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd9c0367d0 con 0x7fdda00b2250 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 -- 172.21.15.106:0/3712559209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2250 msgr2=0x7fdda015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 --2- 172.21.15.106:0/3712559209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2250 0x7fdda015df80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdda00ae5c0 tx=0x7fdd9c02e0e0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 -- 172.21.15.106:0/3712559209 shutdown_connections 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 --2- 172.21.15.106:0/3712559209 >> v2:172.21.15.156:3300/0 conn(0x7fdda00b2630 0x7fdda00ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 --2- 172.21.15.106:0/3712559209 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2250 0x7fdda015df80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 -- 172.21.15.106:0/3712559209 >> 172.21.15.106:0/3712559209 conn(0x7fdda006a920 msgr2=0x7fdda006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:34.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 -- 172.21.15.106:0/3712559209 shutdown_connections 2023-11-24T08:09:34.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.677+0000 7fdda7c56640 1 -- 172.21.15.106:0/3712559209 wait complete. 2023-11-24T08:09:34.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.679+0000 7fdda7c56640 1 Processor -- start 2023-11-24T08:09:34.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.679+0000 7fdda7c56640 1 -- start start 2023-11-24T08:09:34.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda7c56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdda00b2250 0x7fdda01f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:34.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda7c56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 0x7fdda00aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:34.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda7c56640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda00b2aa0 0x7fdda00cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:34.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda7c56640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda0156ea0 con 0x7fdda00b2630 2023-11-24T08:09:34.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda7c56640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdda01f3a20 con 0x7fdda00b2aa0 2023-11-24T08:09:34.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda7c56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdda014da00 con 0x7fdda00b2250 2023-11-24T08:09:34.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda51f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 0x7fdda00aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:34.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda51f1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 0x7fdda00aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51596/0 (socket says 172.21.15.106:51596) 2023-11-24T08:09:34.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.680+0000 7fdda51f1640 1 -- 172.21.15.106:0/1564505159 learned_addr learned my addr 172.21.15.106:0/1564505159 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda51f1640 1 -- 172.21.15.106:0/1564505159 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdda00b2250 msgr2=0x7fdda01f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda51f1640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdda00b2250 0x7fdda01f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda51f1640 1 -- 172.21.15.106:0/1564505159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda00b2aa0 msgr2=0x7fdda00cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda51f1640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda00b2aa0 0x7fdda00cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda51f1640 1 -- 172.21.15.106:0/1564505159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9c00b770 con 0x7fdda00b2630 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda51f1640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 0x7fdda00aeb50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000b4a0 tx=0x7fdd9000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90010610 con 0x7fdda00b2630 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd90010770 con 0x7fdda00b2630 2023-11-24T08:09:34.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd90010b20 con 0x7fdda00b2630 2023-11-24T08:09:34.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdda0156820 con 0x7fdda00b2630 2023-11-24T08:09:34.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.681+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdda00b1830 con 0x7fdda00b2630 2023-11-24T08:09:34.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.683+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdda0156680 con 0x7fdda00b2630 2023-11-24T08:09:34.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.683+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fdd90010c80 con 0x7fdda00b2630 2023-11-24T08:09:34.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.685+0000 7fdd967fc640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdd8406ae40 0x7fdd8406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:34.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.685+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdd90089360 con 0x7fdda00b2630 2023-11-24T08:09:34.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.685+0000 7fdda59f2640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdd8406ae40 0x7fdd8406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:34.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.686+0000 7fdda59f2640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdd8406ae40 0x7fdd8406d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c038000 tx=0x7fdd9c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:34.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.689+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdda0156680 con 0x7fdda00b2630 2023-11-24T08:09:34.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.880+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdda0156680 con 0x7fdda00b2630 2023-11-24T08:09:34.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.883+0000 7fdd967fc640 1 -- 172.21.15.106:0/1564505159 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16793 (secure 0 0 0) 0x7fdda0156680 con 0x7fdda00b2630 2023-11-24T08:09:34.885 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:34.885 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":23,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:34.270334+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.885+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdd8406ae40 msgr2=0x7fdd8406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.885+0000 7fdda7c56640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdd8406ae40 0x7fdd8406d300 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdd9c038000 tx=0x7fdd9c030000 comp rx=0 tx=0).stop 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.885+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 msgr2=0x7fdda00aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.885+0000 7fdda7c56640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 0x7fdda00aeb50 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fdd9000b4a0 tx=0x7fdd9000b960 comp rx=0 tx=0).stop 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 shutdown_connections 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdda00b2aa0 0x7fdda00cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdda00b2630 0x7fdda00aeb50 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdd8406ae40 0x7fdd8406d300 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 --2- 172.21.15.106:0/1564505159 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdda00b2250 0x7fdda01f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:34.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 >> 172.21.15.106:0/1564505159 conn(0x7fdda006a920 msgr2=0x7fdda015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:34.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 shutdown_connections 2023-11-24T08:09:34.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:34.886+0000 7fdda7c56640 1 -- 172.21.15.106:0/1564505159 wait complete. 2023-11-24T08:09:34.907 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:34.908 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:35.118 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.116+0000 7f5ce2439640 1 Processor -- start 2023-11-24T08:09:35.118 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.117+0000 7f5ce2439640 1 -- start start 2023-11-24T08:09:35.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.118+0000 7f5ce2439640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5cdc0b3650 0x7f5cdc06ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.118+0000 7f5ce2439640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc06cd80 con 0x7f5cdc06bfd0 2023-11-24T08:09:35.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.118+0000 7f5ce2439640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cdc06c7c0 con 0x7f5cdc146b10 2023-11-24T08:09:35.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.118+0000 7f5ce2439640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cdc06caa0 con 0x7f5cdc0b3650 2023-11-24T08:09:35.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.119+0000 7f5ce1c38640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5cdc06bfd0 0x7f5cdc15e230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50718/0 (socket says 172.21.15.106:50718) 2023-11-24T08:09:35.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.119+0000 7f5ce1c38640 1 -- 172.21.15.106:0/3923623959 learned_addr learned my addr 172.21.15.106:0/3923623959 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.119+0000 7f5ce0c36640 1 --2- 172.21.15.106:0/3923623959 >> v2:172.21.15.156:3300/0 conn(0x7f5cdc0b3650 0x7f5cdc06ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.119+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201516312 0 0) 0x7f5cdc06cd80 con 0x7f5cdc06bfd0 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.119+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cc8003520 con 0x7f5cdc06bfd0 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916066620 0 0) 0x7f5cdc06c7c0 con 0x7f5cdc146b10 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cdc06cd80 con 0x7f5cdc146b10 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2997991760 0 0) 0x7f5cc8003520 con 0x7f5cdc06bfd0 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 >> v2:172.21.15.156:3300/0 conn(0x7f5cdc0b3650 msgr2=0x7f5cdc06ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 --2- 172.21.15.106:0/3923623959 >> v2:172.21.15.156:3300/0 conn(0x7f5cdc0b3650 0x7f5cdc06ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 >> v1:172.21.15.156:6789/0 conn(0x7f5cdc146b10 legacy=0x7f5cdc0b2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc15da30 con 0x7f5cdc06bfd0 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f5cd4001670 con 0x7f5cdc06bfd0 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 >> v1:172.21.15.106:6789/0 conn(0x7f5cdc06bfd0 legacy=0x7f5cdc15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc8003b50 0x7f5cc8003f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc80047c0 0x7f5cc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8005180 0x7f5cc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc06caa0 con 0x7f5cc8003b50 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc800c050 con 0x7f5cc80047c0 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc800c210 con 0x7f5cc8005180 2023-11-24T08:09:35.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.120+0000 7f5ce1437640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc8003b50 0x7f5cc8003f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce1437640 1 -- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8005180 msgr2=0x7f5cc800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce1437640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8005180 0x7f5cc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce1437640 1 -- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc80047c0 msgr2=0x7f5cc8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce0c36640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc80047c0 0x7f5cc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce1437640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cc80047c0 0x7f5cc8004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce1437640 1 -- 172.21.15.106:0/3923623959 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc8003520 con 0x7f5cc8003b50 2023-11-24T08:09:35.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.121+0000 7f5ce1c38640 1 -- 172.21.15.106:0/3923623959 reap_dead start 2023-11-24T08:09:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.122+0000 7f5ce1437640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc8003b50 0x7f5cc8003f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80018d0 tx=0x7f5cc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2448b091b34d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.122+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd80052d0 con 0x7f5cc8003b50 2023-11-24T08:09:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.122+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd8005430 con 0x7f5cc8003b50 2023-11-24T08:09:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.122+0000 7f5ccffff640 1 -- 172.21.15.106:0/3923623959 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd8003810 con 0x7f5cc8003b50 2023-11-24T08:09:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 -- 172.21.15.106:0/3923623959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc8003b50 msgr2=0x7f5cc8003f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc8003b50 0x7f5cc8003f50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5cd80018d0 tx=0x7f5cc80057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 -- 172.21.15.106:0/3923623959 shutdown_connections 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 --2- 172.21.15.106:0/3923623959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cc8003b50 0x7f5cc8003f50 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 -- 172.21.15.106:0/3923623959 >> 172.21.15.106:0/3923623959 conn(0x7f5cdc061100 msgr2=0x7f5cdc061510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 -- 172.21.15.106:0/3923623959 shutdown_connections 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.123+0000 7f5ce2439640 1 -- 172.21.15.106:0/3923623959 wait complete. 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.125+0000 7f5ce2439640 1 Processor -- start 2023-11-24T08:09:35.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.125+0000 7f5ce2439640 1 -- start start 2023-11-24T08:09:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce2439640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8003b50 0x7f5cdc1f6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce2439640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cdc141230 0x7f5cdc13a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce2439640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 0x7f5cdc140030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce2439640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc1525a0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce2439640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc0cb450 con 0x7f5cdc141230 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce2439640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cdc1f7460 con 0x7f5cc8003b50 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce1c38640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 0x7f5cdc140030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce1c38640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 0x7f5cdc140030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51624/0 (socket says 172.21.15.106:51624) 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce1c38640 1 -- 172.21.15.106:0/287488649 learned_addr learned my addr 172.21.15.106:0/287488649 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce1c38640 1 -- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8003b50 msgr2=0x7f5cdc1f6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce1437640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8003b50 0x7f5cdc1f6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.126+0000 7f5ce0c36640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cdc141230 0x7f5cdc13a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce1c38640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8003b50 0x7f5cdc1f6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce1c38640 1 -- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cdc141230 msgr2=0x7f5cdc13a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce1c38640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cdc141230 0x7f5cdc13a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce1c38640 1 -- 172.21.15.106:0/287488649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc1417a0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce1c38640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 0x7f5cdc140030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc06d7f0 tx=0x7f5cd400c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4004bf0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cd4004d70 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cdc0cabb0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cd4007270 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.127+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cdc0cb170 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.129+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cdc04e840 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.129+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5cd40073d0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.131+0000 7f5ccdffb640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cc006ae40 0x7f5cc006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.131+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5cd4088a30 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.131+0000 7f5ce1437640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cc006ae40 0x7f5cc006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.132+0000 7f5ce1437640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cc006ae40 0x7f5cc006d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8034000 tx=0x7f5cd8031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:35.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.135+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5cd40539b0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.335+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5cdc15f750 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.336+0000 7f5ccdffb640 1 -- 172.21.15.106:0/287488649 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5cd4056fd0 con 0x7f5cdc1f2ea0 2023-11-24T08:09:35.338 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cc006ae40 msgr2=0x7f5cc006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cc006ae40 0x7f5cc006d300 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5cd8034000 tx=0x7f5cd8031000 comp rx=0 tx=0).stop 2023-11-24T08:09:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 msgr2=0x7f5cdc140030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 0x7f5cdc140030 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc06d7f0 tx=0x7f5cd400c7a0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 shutdown_connections 2023-11-24T08:09:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5cdc1f2ea0 0x7f5cdc140030 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cdc141230 0x7f5cdc13a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5cc006ae40 0x7f5cc006d300 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 --2- 172.21.15.106:0/287488649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5cc8003b50 0x7f5cdc1f6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 >> 172.21.15.106:0/287488649 conn(0x7f5cdc061100 msgr2=0x7f5cdc161620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 shutdown_connections 2023-11-24T08:09:35.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.339+0000 7f5ce2439640 1 -- 172.21.15.106:0/287488649 wait complete. 2023-11-24T08:09:35.361 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.569+0000 7f7723075640 1 Processor -- start 2023-11-24T08:09:35.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.569+0000 7f7723075640 1 -- start start 2023-11-24T08:09:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7723075640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c15b440 0x7f771c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7723075640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f771c0cbf20 0x7f771c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7723075640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771c0cc3a0 con 0x7f771c0c57e0 2023-11-24T08:09:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7723075640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f771c15b820 con 0x7f771c0cbf20 2023-11-24T08:09:35.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7723075640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c15eea0 con 0x7f771c15b440 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7721612640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f771c0c57e0 0x7f771c0c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50732/0 (socket says 172.21.15.106:50732) 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f7721612640 1 -- 172.21.15.106:0/1864275327 learned_addr learned my addr 172.21.15.106:0/1864275327 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.571+0000 7f771bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f771c0cbf20 0x7f771c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.572+0000 7f7720e11640 1 --2- 172.21.15.106:0/1864275327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c15b440 0x7f771c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.572+0000 7f771b7fe640 1 -- 172.21.15.106:0/1864275327 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2753384798 0 0) 0x7f7714001490 con 0x7f771c0c57e0 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.572+0000 7f771bfff640 1 -- 172.21.15.106:0/1864275327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c15b440 msgr2=0x7f771c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.572+0000 7f771bfff640 1 --2- 172.21.15.106:0/1864275327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c15b440 0x7f771c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.572+0000 7f771bfff640 1 -- 172.21.15.106:0/1864275327 >> v1:172.21.15.106:6789/0 conn(0x7f771c0c57e0 legacy=0x7f771c0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.572+0000 7f771bfff640 1 -- 172.21.15.106:0/1864275327 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771c06d600 con 0x7f771c0cbf20 2023-11-24T08:09:35.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.573+0000 7f771bfff640 1 --2- 172.21.15.106:0/1864275327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0cbf20 0x7f771c0c52a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f770802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=317b301157358f9d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:35.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.573+0000 7f771b7fe640 1 -- 172.21.15.106:0/1864275327 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770802e070 con 0x7f771c0cbf20 2023-11-24T08:09:35.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f771b7fe640 1 -- 172.21.15.106:0/1864275327 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080315a0 con 0x7f771c0cbf20 2023-11-24T08:09:35.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f771b7fe640 1 -- 172.21.15.106:0/1864275327 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708031930 con 0x7f771c0cbf20 2023-11-24T08:09:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 -- 172.21.15.106:0/1864275327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0cbf20 msgr2=0x7f771c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 --2- 172.21.15.106:0/1864275327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0cbf20 0x7f771c0c52a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f77080097a0 tx=0x7f770802b840 comp rx=0 tx=0).stop 2023-11-24T08:09:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 -- 172.21.15.106:0/1864275327 shutdown_connections 2023-11-24T08:09:35.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 --2- 172.21.15.106:0/1864275327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0cbf20 0x7f771c0c52a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 --2- 172.21.15.106:0/1864275327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c15b440 0x7f771c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 -- 172.21.15.106:0/1864275327 >> 172.21.15.106:0/1864275327 conn(0x7f771c0b0c50 msgr2=0x7f771c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 -- 172.21.15.106:0/1864275327 shutdown_connections 2023-11-24T08:09:35.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.574+0000 7f7723075640 1 -- 172.21.15.106:0/1864275327 wait complete. 2023-11-24T08:09:35.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.576+0000 7f7723075640 1 Processor -- start 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.576+0000 7f7723075640 1 -- start start 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7723075640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 0x7f771c1e7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7723075640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c0cbf20 0x7f771c1541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7723075640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f771c15b440 0x7f771c158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7723075640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c15bde0 con 0x7f771c0c57e0 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7723075640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771c155f70 con 0x7f771c15b440 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7723075640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771c156250 con 0x7f771c0cbf20 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7720e11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 0x7f771c1e7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7720e11640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 0x7f771c1e7900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51648/0 (socket says 172.21.15.106:51648) 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.577+0000 7f7720e11640 1 -- 172.21.15.106:0/1610505037 learned_addr learned my addr 172.21.15.106:0/1610505037 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:35.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7720e11640 1 -- 172.21.15.106:0/1610505037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c0cbf20 msgr2=0x7f771c1541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7720e11640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c0cbf20 0x7f771c1541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7720e11640 1 -- 172.21.15.106:0/1610505037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f771c15b440 msgr2=0x7f771c158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7720e11640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f771c15b440 0x7f771c158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7720e11640 1 -- 172.21.15.106:0/1610505037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708009400 con 0x7f771c0c57e0 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7720e11640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 0x7f771c1e7900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f771000d5f0 tx=0x7f771000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77100125f0 con 0x7f771c0c57e0 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7710012750 con 0x7f771c0c57e0 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7710012b00 con 0x7f771c0c57e0 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.578+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f771c15ba00 con 0x7f771c0c57e0 2023-11-24T08:09:35.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.579+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f771c158f70 con 0x7f771c0c57e0 2023-11-24T08:09:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.580+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771c04e8b0 con 0x7f771c0c57e0 2023-11-24T08:09:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.581+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7710012c60 con 0x7f771c0c57e0 2023-11-24T08:09:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.581+0000 7f77197fa640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f770006ae40 0x7f770006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:35.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.582+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7710089b00 con 0x7f771c0c57e0 2023-11-24T08:09:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.582+0000 7f771bfff640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f770006ae40 0x7f770006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:35.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.583+0000 7f771bfff640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f770006ae40 0x7f770006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7708036000 tx=0x7f7708030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:35.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.586+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7710054bb0 con 0x7f771c0c57e0 2023-11-24T08:09:35.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.775+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f771c15e990 con 0x7f771c0c57e0 2023-11-24T08:09:35.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.775+0000 7f77197fa640 1 -- 172.21.15.106:0/1610505037 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f77100581d0 con 0x7f771c0c57e0 2023-11-24T08:09:35.777 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:35.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f770006ae40 msgr2=0x7f770006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f770006ae40 0x7f770006d300 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7708036000 tx=0x7f7708030000 comp rx=0 tx=0).stop 2023-11-24T08:09:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 msgr2=0x7f771c1e7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 0x7f771c1e7900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f771000d5f0 tx=0x7f771000dab0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 shutdown_connections 2023-11-24T08:09:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f771c15b440 0x7f771c158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f770006ae40 0x7f770006d300 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f771c0cbf20 0x7f771c1541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 --2- 172.21.15.106:0/1610505037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f771c0c57e0 0x7f771c1e7900 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.778+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 >> 172.21.15.106:0/1610505037 conn(0x7f771c0b0c50 msgr2=0x7f771c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.779+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 shutdown_connections 2023-11-24T08:09:35.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:35.779+0000 7f7723075640 1 -- 172.21.15.106:0/1610505037 wait complete. 2023-11-24T08:09:35.799 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:36.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.005+0000 7f754053e640 1 Processor -- start 2023-11-24T08:09:36.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.006+0000 7f754053e640 1 -- start start 2023-11-24T08:09:36.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.007+0000 7f754053e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f753815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.007+0000 7f754053e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f753806de10 0x7f75380705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.007+0000 7f754053e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753815ec90 con 0x7f75380c49f0 2023-11-24T08:09:36.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.007+0000 7f754053e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75380cd530 con 0x7f753806de10 2023-11-24T08:09:36.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.007+0000 7f754053e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75380c4ed0 con 0x7f7538072160 2023-11-24T08:09:36.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f753806de10 0x7f75380705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f753806de10 0x7f75380705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51650/0 (socket says 172.21.15.106:51650) 2023-11-24T08:09:36.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753dad9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f75380c49f0 0x7f753806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50742/0 (socket says 172.21.15.106:50742) 2023-11-24T08:09:36.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 -- 172.21.15.106:0/223164625 learned_addr learned my addr 172.21.15.106:0/223164625 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753e2da640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f753815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 -- 172.21.15.106:0/223164625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 msgr2=0x7f753815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f753815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 -- 172.21.15.106:0/223164625 >> v1:172.21.15.106:6789/0 conn(0x7f75380c49f0 legacy=0x7f753806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753eadb640 1 -- 172.21.15.106:0/223164625 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75380c51b0 con 0x7f753806de10 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753e2da640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f753815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:36.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.008+0000 7f753d2d8640 1 -- 172.21.15.106:0/223164625 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146477257 0 0) 0x7f752c001110 con 0x7f75380c49f0 2023-11-24T08:09:36.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.009+0000 7f753eadb640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753806de10 0x7f75380705c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7530009e50 tx=0x7f753002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156ebd85dcfeb25f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:36.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f753d2d8640 1 -- 172.21.15.106:0/223164625 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753002cb00 con 0x7f753806de10 2023-11-24T08:09:36.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f753d2d8640 1 -- 172.21.15.106:0/223164625 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753002cc60 con 0x7f753806de10 2023-11-24T08:09:36.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f753d2d8640 1 -- 172.21.15.106:0/223164625 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75300343f0 con 0x7f753806de10 2023-11-24T08:09:36.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 -- 172.21.15.106:0/223164625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753806de10 msgr2=0x7f75380705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753806de10 0x7f75380705c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7530009e50 tx=0x7f753002bf60 comp rx=0 tx=0).stop 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 -- 172.21.15.106:0/223164625 shutdown_connections 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f753806de10 0x7f75380705c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 --2- 172.21.15.106:0/223164625 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f753815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 -- 172.21.15.106:0/223164625 >> 172.21.15.106:0/223164625 conn(0x7f75380b0c50 msgr2=0x7f75380b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 -- 172.21.15.106:0/223164625 shutdown_connections 2023-11-24T08:09:36.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.010+0000 7f754053e640 1 -- 172.21.15.106:0/223164625 wait complete. 2023-11-24T08:09:36.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.012+0000 7f754053e640 1 Processor -- start 2023-11-24T08:09:36.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.012+0000 7f754053e640 1 -- start start 2023-11-24T08:09:36.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.013+0000 7f754053e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753806de10 0x7f753809c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.013+0000 7f754053e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f7538158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.013+0000 7f754053e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 0x7f75380c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.013+0000 7f754053e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380c73e0 con 0x7f75380c49f0 2023-11-24T08:09:36.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.013+0000 7f754053e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75380c7100 con 0x7f753806de10 2023-11-24T08:09:36.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.013+0000 7f754053e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753809a510 con 0x7f7538072160 2023-11-24T08:09:36.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 0x7f75380c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 0x7f75380c60d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51654/0 (socket says 172.21.15.106:51654) 2023-11-24T08:09:36.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 -- 172.21.15.106:0/1235255937 learned_addr learned my addr 172.21.15.106:0/1235255937 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:36.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753dad9640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f7538158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753e2da640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753806de10 0x7f753809c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 -- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 msgr2=0x7f7538158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f7538158b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 -- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753806de10 msgr2=0x7f753809c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753806de10 0x7f753809c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.014+0000 7f753eadb640 1 -- 172.21.15.106:0/1235255937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7530009b30 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.015+0000 7f753eadb640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 0x7f75380c60d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7530036040 tx=0x7f7530004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.015+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530009510 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.015+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753003e450 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.015+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753809a7f0 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.015+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753003e5e0 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.015+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753809c8f0 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.016+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753809ad10 con 0x7f75380c49f0 2023-11-24T08:09:36.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.017+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f753003e740 con 0x7f75380c49f0 2023-11-24T08:09:36.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.018+0000 7f752affd640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f751c06ae40 0x7f751c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.018+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f75300abfe0 con 0x7f75380c49f0 2023-11-24T08:09:36.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.018+0000 7f753e2da640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f751c06ae40 0x7f751c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.019+0000 7f753e2da640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f751c06ae40 0x7f751c06d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f753400d8b0 tx=0x7f7534008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:36.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.022+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f753809ad10 con 0x7f75380c49f0 2023-11-24T08:09:36.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.216+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f753809ad10 con 0x7f75380c49f0 2023-11-24T08:09:36.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.217+0000 7f752affd640 1 -- 172.21.15.106:0/1235255937 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f753809ad10 con 0x7f75380c49f0 2023-11-24T08:09:36.219 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f751c06ae40 msgr2=0x7f751c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f751c06ae40 0x7f751c06d300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f753400d8b0 tx=0x7f7534008040 comp rx=0 tx=0).stop 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 msgr2=0x7f75380c60d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 0x7f75380c60d0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7530036040 tx=0x7f7530004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 shutdown_connections 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f75380c49f0 0x7f75380c60d0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f751c06ae40 0x7f751c06d300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7538072160 0x7f7538158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 --2- 172.21.15.106:0/1235255937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f753806de10 0x7f753809c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 >> 172.21.15.106:0/1235255937 conn(0x7f75380b0c50 msgr2=0x7f75380ae210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 shutdown_connections 2023-11-24T08:09:36.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.220+0000 7f754053e640 1 -- 172.21.15.106:0/1235255937 wait complete. 2023-11-24T08:09:36.239 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:09:36.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.448+0000 7f152576c640 1 Processor -- start 2023-11-24T08:09:36.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.448+0000 7f152576c640 1 -- start start 2023-11-24T08:09:36.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f152576c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f152015d490 0x7f15200cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f152576c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f15200d3940 0x7f15200d1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f152576c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15200cea30 con 0x7f15200b63d0 2023-11-24T08:09:36.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f152576c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200d0450 con 0x7f152015d490 2023-11-24T08:09:36.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f152576c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15200d0bf0 con 0x7f15200d3940 2023-11-24T08:09:36.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f151ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f152015d490 0x7f15200cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f151f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f15200b63d0 0x7f15200d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50750/0 (socket says 172.21.15.106:50750) 2023-11-24T08:09:36.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.450+0000 7f151f7fe640 1 -- 172.21.15.106:0/295926303 learned_addr learned my addr 172.21.15.106:0/295926303 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f151effd640 1 -- 172.21.15.106:0/295926303 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3745877212 0 0) 0x7f15200cea30 con 0x7f15200b63d0 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f151effd640 1 -- 172.21.15.106:0/295926303 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1508003520 con 0x7f15200b63d0 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f151ffff640 1 -- 172.21.15.106:0/295926303 >> v2:172.21.15.156:3300/0 conn(0x7f15200d3940 msgr2=0x7f15200d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f1524f6b640 1 --2- 172.21.15.106:0/295926303 >> v2:172.21.15.156:3300/0 conn(0x7f15200d3940 0x7f15200d1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f151ffff640 1 --2- 172.21.15.106:0/295926303 >> v2:172.21.15.156:3300/0 conn(0x7f15200d3940 0x7f15200d1250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f151ffff640 1 -- 172.21.15.106:0/295926303 >> v1:172.21.15.106:6789/0 conn(0x7f15200b63d0 legacy=0x7f15200d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.451+0000 7f151ffff640 1 -- 172.21.15.106:0/295926303 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15200ce750 con 0x7f152015d490 2023-11-24T08:09:36.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.452+0000 7f151ffff640 1 --2- 172.21.15.106:0/295926303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200cd8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1520161f30 tx=0x7f151002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03ee866a31f69b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.452+0000 7f151effd640 1 -- 172.21.15.106:0/295926303 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510031830 con 0x7f152015d490 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.452+0000 7f151effd640 1 -- 172.21.15.106:0/295926303 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510031990 con 0x7f152015d490 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.452+0000 7f151effd640 1 -- 172.21.15.106:0/295926303 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510031d00 con 0x7f152015d490 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 -- 172.21.15.106:0/295926303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 msgr2=0x7f15200cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 --2- 172.21.15.106:0/295926303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200cd8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1520161f30 tx=0x7f151002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 -- 172.21.15.106:0/295926303 shutdown_connections 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 --2- 172.21.15.106:0/295926303 >> v2:172.21.15.156:3300/0 conn(0x7f15200d3940 0x7f15200d1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 --2- 172.21.15.106:0/295926303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200cd8f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 -- 172.21.15.106:0/295926303 >> 172.21.15.106:0/295926303 conn(0x7f1520061340 msgr2=0x7f1520061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 -- 172.21.15.106:0/295926303 shutdown_connections 2023-11-24T08:09:36.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.453+0000 7f152576c640 1 -- 172.21.15.106:0/295926303 wait complete. 2023-11-24T08:09:36.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.455+0000 7f152576c640 1 Processor -- start 2023-11-24T08:09:36.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.455+0000 7f152576c640 1 -- start start 2023-11-24T08:09:36.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f152576c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15200d3940 0x7f15200d21d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f152576c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f152576c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f152014f550 0x7f15201f2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f152576c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15201603d0 con 0x7f152015d490 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f152576c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15200d2710 con 0x7f152014f550 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f152576c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15201608c0 con 0x7f15200d3940 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51670/0 (socket says 172.21.15.106:51670) 2023-11-24T08:09:36.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 -- 172.21.15.106:0/3701944263 learned_addr learned my addr 172.21.15.106:0/3701944263 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:36.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 -- 172.21.15.106:0/3701944263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15200d3940 msgr2=0x7f15200d21d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15200d3940 0x7f15200d21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 -- 172.21.15.106:0/3701944263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f152014f550 msgr2=0x7f15201f2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f152014f550 0x7f15201f2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.456+0000 7f151f7fe640 1 -- 172.21.15.106:0/3701944263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1510009400 con 0x7f152015d490 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.457+0000 7f151f7fe640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200c6db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f151400c460 tx=0x7f151400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.457+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514005ce0 con 0x7f152015d490 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.457+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1514005e40 con 0x7f152015d490 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.457+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514007290 con 0x7f152015d490 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.457+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15200c72f0 con 0x7f152015d490 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.458+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15200c99a0 con 0x7f152015d490 2023-11-24T08:09:36.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.459+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152004e840 con 0x7f152015d490 2023-11-24T08:09:36.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.460+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f15140073f0 con 0x7f152015d490 2023-11-24T08:09:36.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.461+0000 7f151cff9640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f150c06ae40 0x7f150c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:36.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.461+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1514088d60 con 0x7f152015d490 2023-11-24T08:09:36.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.461+0000 7f151ffff640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f150c06ae40 0x7f150c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:36.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.462+0000 7f151ffff640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f150c06ae40 0x7f150c06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1510036000 tx=0x7f1510030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:36.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.465+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1514053c60 con 0x7f152015d490 2023-11-24T08:09:36.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.668+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f15200d1f30 con 0x7f152015d490 2023-11-24T08:09:36.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.669+0000 7f151cff9640 1 -- 172.21.15.106:0/3701944263 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1514017080 con 0x7f152015d490 2023-11-24T08:09:36.671 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:09:36.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f150c06ae40 msgr2=0x7f150c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f150c06ae40 0x7f150c06d300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f1510036000 tx=0x7f1510030000 comp rx=0 tx=0).stop 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 msgr2=0x7f15200c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200c6db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f151400c460 tx=0x7f151400c920 comp rx=0 tx=0).stop 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 shutdown_connections 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f152014f550 0x7f15201f2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f152015d490 0x7f15200c6db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f150c06ae40 0x7f150c06d300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 --2- 172.21.15.106:0/3701944263 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f15200d3940 0x7f15200d21d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:36.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 >> 172.21.15.106:0/3701944263 conn(0x7f1520061340 msgr2=0x7f15200d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:36.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 shutdown_connections 2023-11-24T08:09:36.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:36.672+0000 7f152576c640 1 -- 172.21.15.106:0/3701944263 wait complete. 2023-11-24T08:09:36.694 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-24T08:09:36.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:36.694 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-24T08:09:36.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:09:36.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:36.793 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-24T08:09:36.845 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:36.846 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:36.905 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:36.906 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:36.963 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:09:36.963 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:09:37.016 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.235+0000 7f562a8a6640 1 Processor -- start 2023-11-24T08:09:37.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.235+0000 7f562a8a6640 1 -- start start 2023-11-24T08:09:37.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a8a6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624144b30 0x7f562415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a8a6640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f562415c410 0x7f56240b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a8a6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f562415bf80 con 0x7f56240b1de0 2023-11-24T08:09:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a8a6640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56240b1560 con 0x7f562415c410 2023-11-24T08:09:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a8a6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56240b0fb0 con 0x7f5624144b30 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a0a5640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f56240b1de0 0x7f562406c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50754/0 (socket says 172.21.15.106:50754) 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.237+0000 7f562a0a5640 1 -- 172.21.15.106:0/4132970056 learned_addr learned my addr 172.21.15.106:0/4132970056 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56290a3640 1 --2- 172.21.15.106:0/4132970056 >> v2:172.21.15.156:3300/0 conn(0x7f562415c410 0x7f56240b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56298a4640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624144b30 0x7f562415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1528172477 0 0) 0x7f562415bf80 con 0x7f56240b1de0 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5610003520 con 0x7f56240b1de0 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 827650278 0 0) 0x7f5610003520 con 0x7f56240b1de0 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624144b30 msgr2=0x7f562415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624144b30 0x7f562415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 >> v2:172.21.15.156:3300/0 conn(0x7f562415c410 msgr2=0x7f56240b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 --2- 172.21.15.106:0/4132970056 >> v2:172.21.15.156:3300/0 conn(0x7f562415c410 0x7f56240b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56240b1280 con 0x7f56240b1de0 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f561c0019f0 con 0x7f56240b1de0 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.238+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 >> v1:172.21.15.106:6789/0 conn(0x7f56240b1de0 legacy=0x7f562406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56288a2640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5610003970 0x7f5610003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56288a2640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56100047c0 0x7f5610004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56288a2640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5610005180 0x7f561000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240b1560 con 0x7f5610005180 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240b0fb0 con 0x7f56100047c0 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561000bf40 con 0x7f5610003970 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56298a4640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624144b30 0x7f562415f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f562a0a5640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5610005180 0x7f561000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56298a4640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5610003970 0x7f5610003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f56290a3640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56100047c0 0x7f5610004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f562a0a5640 1 -- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5610003970 msgr2=0x7f5610003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f562a0a5640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5610003970 0x7f5610003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f562a0a5640 1 -- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56100047c0 msgr2=0x7f5610004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f562a0a5640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56100047c0 0x7f5610004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.239+0000 7f562a0a5640 1 -- 172.21.15.106:0/4132970056 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5610003520 con 0x7f5610005180 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.240+0000 7f562a0a5640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5610005180 0x7f561000b980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f561c00a790 tx=0x7f56100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7487ce39adcac8e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c034630 con 0x7f5610005180 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c02f070 con 0x7f5610005180 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f56288a2640 1 -- 172.21.15.106:0/4132970056 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c0349f0 con 0x7f5610005180 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a8a6640 1 -- 172.21.15.106:0/4132970056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5610005180 msgr2=0x7f561000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a8a6640 1 --2- 172.21.15.106:0/4132970056 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5610005180 0x7f561000b980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f561c00a790 tx=0x7f56100057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a0a5640 1 -- 172.21.15.106:0/4132970056 reap_dead start 2023-11-24T08:09:37.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a8a6640 1 -- 172.21.15.106:0/4132970056 shutdown_connections 2023-11-24T08:09:37.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a8a6640 1 -- 172.21.15.106:0/4132970056 >> 172.21.15.106:0/4132970056 conn(0x7f56240cb830 msgr2=0x7f56240cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:37.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a8a6640 1 -- 172.21.15.106:0/4132970056 shutdown_connections 2023-11-24T08:09:37.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.241+0000 7f562a8a6640 1 -- 172.21.15.106:0/4132970056 wait complete. 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.243+0000 7f562a8a6640 1 Processor -- start 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.243+0000 7f562a8a6640 1 -- start start 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a8a6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5610005180 0x7f5624159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a8a6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624137830 0x7f5624070210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a8a6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 0x7f562413bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a8a6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f562415acb0 con 0x7f562415a3b0 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a8a6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56240708f0 con 0x7f5610005180 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a8a6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5624070bd0 con 0x7f5624137830 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.244+0000 7f562a0a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 0x7f562413bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 0x7f562413bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51676/0 (socket says 172.21.15.106:51676) 2023-11-24T08:09:37.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 -- 172.21.15.106:0/971145135 learned_addr learned my addr 172.21.15.106:0/971145135 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 -- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624137830 msgr2=0x7f5624070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624137830 0x7f5624070210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 -- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5610005180 msgr2=0x7f5624159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f56298a4640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5610005180 0x7f5624159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5610005180 0x7f5624159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 -- 172.21.15.106:0/971145135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624070eb0 con 0x7f562415a3b0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f562a0a5640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 0x7f562413bb50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f561c02cad0 tx=0x7f561c001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.245+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c034840 con 0x7f562415a3b0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.246+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561c0349a0 con 0x7f562415a3b0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.246+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5624071190 con 0x7f562415a3b0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.246+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561c034d10 con 0x7f562415a3b0 2023-11-24T08:09:37.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.246+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f562413c090 con 0x7f562415a3b0 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.247+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5624134110 con 0x7f562415a3b0 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.248+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f561c031030 con 0x7f562415a3b0 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.249+0000 7f561a7fc640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f560806ae60 0x7f560806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.249+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f561c0abbf0 con 0x7f562415a3b0 2023-11-24T08:09:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.249+0000 7f56298a4640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f560806ae60 0x7f560806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.250+0000 7f56298a4640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f560806ae60 0x7f560806d320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5620006fd0 tx=0x7f5620008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:37.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.253+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5624134110 con 0x7f562415a3b0 2023-11-24T08:09:37.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.447+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5624134110 con 0x7f562415a3b0 2023-11-24T08:09:37.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.448+0000 7f561a7fc640 1 -- 172.21.15.106:0/971145135 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16793 (secure 0 0 0) 0x7f5624134110 con 0x7f562415a3b0 2023-11-24T08:09:37.450 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:37.450 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:35.282948+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f560806ae60 msgr2=0x7f560806d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f560806ae60 0x7f560806d320 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f5620006fd0 tx=0x7f5620008040 comp rx=0 tx=0).stop 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 msgr2=0x7f562413bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 0x7f562413bb50 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f561c02cad0 tx=0x7f561c001cb0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 shutdown_connections 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f562415a3b0 0x7f562413bb50 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f560806ae60 0x7f560806d320 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5624137830 0x7f5624070210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 --2- 172.21.15.106:0/971145135 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5610005180 0x7f5624159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 >> 172.21.15.106:0/971145135 conn(0x7f56240cb830 msgr2=0x7f562415d140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:37.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 shutdown_connections 2023-11-24T08:09:37.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.451+0000 7f562a8a6640 1 -- 172.21.15.106:0/971145135 wait complete. 2023-11-24T08:09:37.473 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:37.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.689+0000 7f00d7397640 1 Processor -- start 2023-11-24T08:09:37.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.689+0000 7f00d7397640 1 -- start start 2023-11-24T08:09:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.690+0000 7f00d7397640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f00d00ccbd0 0x7f00d01660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d7397640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f00d00c9650 0x7f00d00c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d7397640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00d01655f0 con 0x7f00d00ccfb0 2023-11-24T08:09:37.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d7397640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d00cc380 con 0x7f00d00c9650 2023-11-24T08:09:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d7397640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00d0162b30 con 0x7f00d00ccbd0 2023-11-24T08:09:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d5934640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f00d00c9650 0x7f00d00c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d5934640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f00d00c9650 0x7f00d00c9ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51680/0 (socket says 172.21.15.106:51680) 2023-11-24T08:09:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d5934640 1 -- 172.21.15.106:0/269846779 learned_addr learned my addr 172.21.15.106:0/269846779 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.691+0000 7f00d5133640 1 --2- 172.21.15.106:0/269846779 >> v2:172.21.15.156:3300/0 conn(0x7f00d00ccbd0 0x7f00d01660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.692+0000 7f00d5934640 1 -- 172.21.15.106:0/269846779 >> v2:172.21.15.156:3300/0 conn(0x7f00d00ccbd0 msgr2=0x7f00d01660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.692+0000 7f00d5934640 1 --2- 172.21.15.106:0/269846779 >> v2:172.21.15.156:3300/0 conn(0x7f00d00ccbd0 0x7f00d01660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.692+0000 7f00d5934640 1 -- 172.21.15.106:0/269846779 >> v1:172.21.15.156:6789/0 conn(0x7f00d00ccfb0 legacy=0x7f00d0162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.692+0000 7f00d5934640 1 -- 172.21.15.106:0/269846779 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d0162e10 con 0x7f00d00c9650 2023-11-24T08:09:37.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.693+0000 7f00d5934640 1 --2- 172.21.15.106:0/269846779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d00c9ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f00d006ea50 tx=0x7f00c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725e967af65a2e1d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:37.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.693+0000 7f00c7fff640 1 -- 172.21.15.106:0/269846779 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c802ca80 con 0x7f00d00c9650 2023-11-24T08:09:37.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.693+0000 7f00c7fff640 1 -- 172.21.15.106:0/269846779 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c802cbe0 con 0x7f00d00c9650 2023-11-24T08:09:37.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.693+0000 7f00c7fff640 1 -- 172.21.15.106:0/269846779 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c80343f0 con 0x7f00d00c9650 2023-11-24T08:09:37.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 -- 172.21.15.106:0/269846779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 msgr2=0x7f00d00c9ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 --2- 172.21.15.106:0/269846779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d00c9ae0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f00d006ea50 tx=0x7f00c802bee0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 -- 172.21.15.106:0/269846779 shutdown_connections 2023-11-24T08:09:37.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 --2- 172.21.15.106:0/269846779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d00c9ae0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 --2- 172.21.15.106:0/269846779 >> v2:172.21.15.156:3300/0 conn(0x7f00d00ccbd0 0x7f00d01660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 -- 172.21.15.106:0/269846779 >> 172.21.15.106:0/269846779 conn(0x7f00d00b0c50 msgr2=0x7f00d00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:37.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 -- 172.21.15.106:0/269846779 shutdown_connections 2023-11-24T08:09:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.694+0000 7f00d7397640 1 -- 172.21.15.106:0/269846779 wait complete. 2023-11-24T08:09:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.696+0000 7f00d7397640 1 Processor -- start 2023-11-24T08:09:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.696+0000 7f00d7397640 1 -- start start 2023-11-24T08:09:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d7397640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d0164460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d7397640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00d00ccbd0 0x7f00d00b3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d7397640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d00ccfb0 0x7f00d00ad1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d7397640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00b3c50 con 0x7f00d00c9650 2023-11-24T08:09:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d7397640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00b3e60 con 0x7f00d00ccfb0 2023-11-24T08:09:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d7397640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00b4140 con 0x7f00d00ccbd0 2023-11-24T08:09:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d0164460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d0164460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51682/0 (socket says 172.21.15.106:51682) 2023-11-24T08:09:37.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 -- 172.21.15.106:0/3937483657 learned_addr learned my addr 172.21.15.106:0/3937483657 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:37.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 -- 172.21.15.106:0/3937483657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00d00ccbd0 msgr2=0x7f00d00b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00d00ccbd0 0x7f00d00b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 -- 172.21.15.106:0/3937483657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d00ccfb0 msgr2=0x7f00d00ad1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d00ccfb0 0x7f00d00ad1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.697+0000 7f00d5133640 1 -- 172.21.15.106:0/3937483657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c8009b30 con 0x7f00d00c9650 2023-11-24T08:09:37.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.698+0000 7f00d5133640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d0164460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00d5f0 tx=0x7f00cc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:37.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.698+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc0125f0 con 0x7f00d00c9650 2023-11-24T08:09:37.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.698+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00cc012750 con 0x7f00d00c9650 2023-11-24T08:09:37.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.698+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d00b4420 con 0x7f00d00c9650 2023-11-24T08:09:37.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.698+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00cc012b00 con 0x7f00d00c9650 2023-11-24T08:09:37.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.698+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d00b49e0 con 0x7f00d00c9650 2023-11-24T08:09:37.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.699+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d00cd3f0 con 0x7f00d00c9650 2023-11-24T08:09:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.700+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f00cc012c60 con 0x7f00d00c9650 2023-11-24T08:09:37.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.701+0000 7f00c5ffb640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f00b406ae40 0x7f00b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:37.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.701+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f00cc089b00 con 0x7f00d00c9650 2023-11-24T08:09:37.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.701+0000 7f00d4932640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f00b406ae40 0x7f00b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:37.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.702+0000 7f00d4932640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f00b406ae40 0x7f00b406d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00d006c5a0 tx=0x7f00c00079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:37.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.705+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f00d00cd3f0 con 0x7f00d00c9650 2023-11-24T08:09:37.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.894+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f00d00cd3f0 con 0x7f00d00c9650 2023-11-24T08:09:37.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.895+0000 7f00c5ffb640 1 -- 172.21.15.106:0/3937483657 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00d00cd3f0 con 0x7f00d00c9650 2023-11-24T08:09:37.896 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:37.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f00b406ae40 msgr2=0x7f00b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f00b406ae40 0x7f00b406d300 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f00d006c5a0 tx=0x7f00c00079b0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 msgr2=0x7f00d0164460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:37.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d0164460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f00cc00d5f0 tx=0x7f00cc00dab0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 shutdown_connections 2023-11-24T08:09:37.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00d00ccfb0 0x7f00d00ad1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f00b406ae40 0x7f00b406d300 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00d00ccbd0 0x7f00d00b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.897+0000 7f00d7397640 1 --2- 172.21.15.106:0/3937483657 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00d00c9650 0x7f00d0164460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:37.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.898+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 >> 172.21.15.106:0/3937483657 conn(0x7f00d00b0c50 msgr2=0x7f00d00aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:37.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.898+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 shutdown_connections 2023-11-24T08:09:37.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:37.898+0000 7f00d7397640 1 -- 172.21.15.106:0/3937483657 wait complete. 2023-11-24T08:09:37.918 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.128+0000 7f971881d640 1 Processor -- start 2023-11-24T08:09:38.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.129+0000 7f971881d640 1 -- start start 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971881d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f9714071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971881d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9714071620 0x7f97140ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971881d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9714070860 con 0x7f971406c7c0 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971881d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9714159f40 con 0x7f9714071620 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971881d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f971415b020 con 0x7f971415aa80 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971359e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9714071620 0x7f97140ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971359e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9714071620 0x7f97140ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51692/0 (socket says 172.21.15.106:51692) 2023-11-24T08:09:38.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.130+0000 7f971359e640 1 -- 172.21.15.106:0/1560951680 learned_addr learned my addr 172.21.15.106:0/1560951680 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.131+0000 7f971259c640 1 --2- 172.21.15.106:0/1560951680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f9714071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.131+0000 7f971359e640 1 -- 172.21.15.106:0/1560951680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 msgr2=0x7f9714071eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.131+0000 7f971359e640 1 --2- 172.21.15.106:0/1560951680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f9714071eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.131+0000 7f971359e640 1 -- 172.21.15.106:0/1560951680 >> v1:172.21.15.156:6789/0 conn(0x7f971406c7c0 legacy=0x7f971406cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.131+0000 7f971359e640 1 -- 172.21.15.106:0/1560951680 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714070b40 con 0x7f9714071620 2023-11-24T08:09:38.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.132+0000 7f971359e640 1 --2- 172.21.15.106:0/1560951680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ccc90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f970c00d150 tx=0x7f970c02e1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9a9057128df18b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:38.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f9711d9b640 1 -- 172.21.15.106:0/1560951680 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c038400 con 0x7f9714071620 2023-11-24T08:09:38.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f9711d9b640 1 -- 172.21.15.106:0/1560951680 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f970c032070 con 0x7f9714071620 2023-11-24T08:09:38.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f9711d9b640 1 -- 172.21.15.106:0/1560951680 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f970c038780 con 0x7f9714071620 2023-11-24T08:09:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 -- 172.21.15.106:0/1560951680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 msgr2=0x7f97140ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 --2- 172.21.15.106:0/1560951680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ccc90 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f970c00d150 tx=0x7f970c02e1f0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 -- 172.21.15.106:0/1560951680 shutdown_connections 2023-11-24T08:09:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 --2- 172.21.15.106:0/1560951680 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ccc90 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 --2- 172.21.15.106:0/1560951680 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f9714071eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 -- 172.21.15.106:0/1560951680 >> 172.21.15.106:0/1560951680 conn(0x7f97140cbb40 msgr2=0x7f97140cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.133+0000 7f971881d640 1 -- 172.21.15.106:0/1560951680 shutdown_connections 2023-11-24T08:09:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.134+0000 7f971881d640 1 -- 172.21.15.106:0/1560951680 wait complete. 2023-11-24T08:09:38.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.135+0000 7f971881d640 1 Processor -- start 2023-11-24T08:09:38.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.135+0000 7f971881d640 1 -- start start 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.136+0000 7f971881d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f971406c7c0 0x7f9714142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.136+0000 7f971881d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.136+0000 7f971881d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f97140ad880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.136+0000 7f971881d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140c5ad0 con 0x7f9714071620 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.136+0000 7f971881d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9714143880 con 0x7f971406c7c0 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.136+0000 7f971881d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97141432b0 con 0x7f971415aa80 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ad340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51700/0 (socket says 172.21.15.106:51700) 2023-11-24T08:09:38.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 -- 172.21.15.106:0/3261606526 learned_addr learned my addr 172.21.15.106:0/3261606526 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971359e640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f97140ad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 -- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 msgr2=0x7f97140ad880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f97140ad880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 -- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f971406c7c0 msgr2=0x7f9714142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f9712d9d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f971406c7c0 0x7f9714142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f971406c7c0 0x7f9714142d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971259c640 1 -- 172.21.15.106:0/3261606526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970c00bd40 con 0x7f9714071620 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.137+0000 7f971359e640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f97140ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f971259c640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ad340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f970800d5d0 tx=0x7f970800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97080125f0 con 0x7f9714071620 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9708012770 con 0x7f9714071620 2023-11-24T08:09:38.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9708012b20 con 0x7f9714071620 2023-11-24T08:09:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97140c2e90 con 0x7f9714071620 2023-11-24T08:09:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97140c5510 con 0x7f9714071620 2023-11-24T08:09:38.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.138+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97141435e0 con 0x7f9714071620 2023-11-24T08:09:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.139+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9708012c80 con 0x7f9714071620 2023-11-24T08:09:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.140+0000 7f96fb7fe640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f970006ae40 0x7f970006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.140+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9708089bd0 con 0x7f9714071620 2023-11-24T08:09:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.140+0000 7f9712d9d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f970006ae40 0x7f970006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.141+0000 7f9712d9d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f970006ae40 0x7f970006d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9704000f60 tx=0x7f9704009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:38.147 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.145+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97141435e0 con 0x7f9714071620 2023-11-24T08:09:38.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.335+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f97141439c0 con 0x7f9714071620 2023-11-24T08:09:38.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.336+0000 7f96fb7fe640 1 -- 172.21.15.106:0/3261606526 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f97141439c0 con 0x7f9714071620 2023-11-24T08:09:38.339 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f970006ae40 msgr2=0x7f970006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f970006ae40 0x7f970006d300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9704000f60 tx=0x7f9704009040 comp rx=0 tx=0).stop 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 msgr2=0x7f97140ad340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ad340 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f970800d5d0 tx=0x7f970800da90 comp rx=0 tx=0).stop 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 shutdown_connections 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f970006ae40 0x7f970006d300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f971415aa80 0x7f97140ad880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9714071620 0x7f97140ad340 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 --2- 172.21.15.106:0/3261606526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f971406c7c0 0x7f9714142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.338+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 >> 172.21.15.106:0/3261606526 conn(0x7f97140cbb40 msgr2=0x7f9714159300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.339+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 shutdown_connections 2023-11-24T08:09:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.339+0000 7f971881d640 1 -- 172.21.15.106:0/3261606526 wait complete. 2023-11-24T08:09:38.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:38.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.575+0000 7ff6099ce640 1 Processor -- start 2023-11-24T08:09:38.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.575+0000 7ff6099ce640 1 -- start start 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.577+0000 7ff6099ce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff60415d720 0x7ff6040d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.577+0000 7ff6099ce640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff6040d3870 0x7ff6040b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.577+0000 7ff6099ce640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60415db00 con 0x7ff6040b1400 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.577+0000 7ff6099ce640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6040d3d20 con 0x7ff60415d720 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.577+0000 7ff6099ce640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040d0ce0 con 0x7ff6040d3870 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.577+0000 7ff6037fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff6040b1400 0x7ff6040b1870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50758/0 (socket says 172.21.15.106:50758) 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff6037fe640 1 -- 172.21.15.106:0/512963596 learned_addr learned my addr 172.21.15.106:0/512963596 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:38.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff602ffd640 1 --2- 172.21.15.106:0/512963596 >> v2:172.21.15.106:3300/0 conn(0x7ff60415d720 0x7ff6040d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff5fa7fc640 1 --2- 172.21.15.106:0/512963596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff6040d3870 0x7ff6040b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff6027fc640 1 -- 172.21.15.106:0/512963596 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1855866221 0 0) 0x7ff60415db00 con 0x7ff6040b1400 2023-11-24T08:09:38.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff6027fc640 1 -- 172.21.15.106:0/512963596 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0003520 con 0x7ff6040b1400 2023-11-24T08:09:38.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff602ffd640 1 -- 172.21.15.106:0/512963596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff6040d3870 msgr2=0x7ff6040b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:38.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff602ffd640 1 --2- 172.21.15.106:0/512963596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff6040d3870 0x7ff6040b0ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff602ffd640 1 -- 172.21.15.106:0/512963596 >> v1:172.21.15.106:6789/0 conn(0x7ff6040b1400 legacy=0x7ff6040b1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.578+0000 7ff602ffd640 1 -- 172.21.15.106:0/512963596 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60406d400 con 0x7ff60415d720 2023-11-24T08:09:38.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.579+0000 7ff602ffd640 1 --2- 172.21.15.106:0/512963596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60415d720 0x7ff6040d3330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40097a0 tx=0x7ff5f402b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24a5435ea93f7db4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:38.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6027fc640 1 -- 172.21.15.106:0/512963596 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4031780 con 0x7ff60415d720 2023-11-24T08:09:38.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6027fc640 1 -- 172.21.15.106:0/512963596 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f40318e0 con 0x7ff60415d720 2023-11-24T08:09:38.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6027fc640 1 -- 172.21.15.106:0/512963596 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f4031c70 con 0x7ff60415d720 2023-11-24T08:09:38.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6099ce640 1 -- 172.21.15.106:0/512963596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60415d720 msgr2=0x7ff6040d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6099ce640 1 --2- 172.21.15.106:0/512963596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60415d720 0x7ff6040d3330 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff5f40097a0 tx=0x7ff5f402b820 comp rx=0 tx=0).stop 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6099ce640 1 -- 172.21.15.106:0/512963596 shutdown_connections 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6099ce640 1 --2- 172.21.15.106:0/512963596 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff6040d3870 0x7ff6040b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6099ce640 1 --2- 172.21.15.106:0/512963596 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff60415d720 0x7ff6040d3330 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.580+0000 7ff6099ce640 1 -- 172.21.15.106:0/512963596 >> 172.21.15.106:0/512963596 conn(0x7ff60408af60 msgr2=0x7ff60409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.581+0000 7ff6099ce640 1 -- 172.21.15.106:0/512963596 shutdown_connections 2023-11-24T08:09:38.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.581+0000 7ff6099ce640 1 -- 172.21.15.106:0/512963596 wait complete. 2023-11-24T08:09:38.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.582+0000 7ff6099ce640 1 Processor -- start 2023-11-24T08:09:38.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.582+0000 7ff6099ce640 1 -- start start 2023-11-24T08:09:38.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.583+0000 7ff6099ce640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6040b1400 0x7ff6041605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.583+0000 7ff6099ce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 0x7ff6040b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.583+0000 7ff6099ce640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff60415d720 0x7ff60415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.583+0000 7ff6099ce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040b9380 con 0x7ff6040d3870 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.583+0000 7ff6099ce640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff604160ae0 con 0x7ff6040b1400 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.583+0000 7ff6099ce640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6040b6f40 con 0x7ff60415d720 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 0x7ff6040b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 0x7ff6040b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51726/0 (socket says 172.21.15.106:51726) 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 -- 172.21.15.106:0/1971914662 learned_addr learned my addr 172.21.15.106:0/1971914662 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 -- 172.21.15.106:0/1971914662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff60415d720 msgr2=0x7ff60415c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff60415d720 0x7ff60415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 -- 172.21.15.106:0/1971914662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6040b1400 msgr2=0x7ff6041605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6040b1400 0x7ff6041605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 -- 172.21.15.106:0/1971914662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f4009400 con 0x7ff6040d3870 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff5fa7fc640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 0x7ff6040b7c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00d800 tx=0x7ff5ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.584+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00b6c0 con 0x7ff6040d3870 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.585+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5ec00b820 con 0x7ff6040d3870 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.585+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec00bbb0 con 0x7ff6040d3870 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.585+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6040b71e0 con 0x7ff6040d3870 2023-11-24T08:09:38.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.585+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6040b7480 con 0x7ff6040d3870 2023-11-24T08:09:38.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.586+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60404e8b0 con 0x7ff6040d3870 2023-11-24T08:09:38.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.587+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7ff5ec00bd10 con 0x7ff6040d3870 2023-11-24T08:09:38.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.588+0000 7ff600ff9640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5e406ae40 0x7ff5e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:38.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.588+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff5ec089cc0 con 0x7ff6040d3870 2023-11-24T08:09:38.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.588+0000 7ff602ffd640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5e406ae40 0x7ff5e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:38.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.589+0000 7ff602ffd640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5e406ae40 0x7ff5e406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4036000 tx=0x7ff5f4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:38.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.592+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5ec090080 con 0x7ff6040d3870 2023-11-24T08:09:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.780+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6040cfb30 con 0x7ff6040d3870 2023-11-24T08:09:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.781+0000 7ff600ff9640 1 -- 172.21.15.106:0/1971914662 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5ec054c40 con 0x7ff6040d3870 2023-11-24T08:09:38.784 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.783+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5e406ae40 msgr2=0x7ff5e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.783+0000 7ff6099ce640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5e406ae40 0x7ff5e406d300 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4036000 tx=0x7ff5f4030000 comp rx=0 tx=0).stop 2023-11-24T08:09:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 msgr2=0x7ff6040b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 0x7ff6040b7c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00d800 tx=0x7ff5ec00dcc0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 shutdown_connections 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5e406ae40 0x7ff5e406d300 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff60415d720 0x7ff60415c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff6040d3870 0x7ff6040b7c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 --2- 172.21.15.106:0/1971914662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6040b1400 0x7ff6041605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 >> 172.21.15.106:0/1971914662 conn(0x7ff60408af60 msgr2=0x7ff6040ce250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 shutdown_connections 2023-11-24T08:09:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:38.784+0000 7ff6099ce640 1 -- 172.21.15.106:0/1971914662 wait complete. 2023-11-24T08:09:38.806 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:09:39.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.009+0000 7f1ba16ee640 1 Processor -- start 2023-11-24T08:09:39.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.009+0000 7f1ba16ee640 1 -- start start 2023-11-24T08:09:39.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1ba16ee640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1b9c072160 0x7f1b9c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1ba16ee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b9c0c49f0 0x7f1b9c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1ba16ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c06dc40 0x7f1b9c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1ba16ee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9c06e0d0 con 0x7f1b9c0c49f0 2023-11-24T08:09:39.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1ba16ee640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9c15ec90 con 0x7f1b9c072160 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1ba16ee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0cd530 con 0x7f1b9c06dc40 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1b9a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b9c0c49f0 0x7f1b9c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1b9a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1b9c0c49f0 0x7f1b9c06d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51738/0 (socket says 172.21.15.106:51738) 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.011+0000 7f1b9a7fc640 1 -- 172.21.15.106:0/1849351766 learned_addr learned my addr 172.21.15.106:0/1849351766 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.012+0000 7f1b9a7fc640 1 -- 172.21.15.106:0/1849351766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c06dc40 msgr2=0x7f1b9c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.012+0000 7f1b9a7fc640 1 --2- 172.21.15.106:0/1849351766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c06dc40 0x7f1b9c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.012+0000 7f1b9a7fc640 1 -- 172.21.15.106:0/1849351766 >> v2:172.21.15.156:3300/0 conn(0x7f1b9c072160 msgr2=0x7f1b9c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.012+0000 7f1b9a7fc640 1 --2- 172.21.15.106:0/1849351766 >> v2:172.21.15.156:3300/0 conn(0x7f1b9c072160 0x7f1b9c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.012+0000 7f1b9a7fc640 1 -- 172.21.15.106:0/1849351766 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c0c4ed0 con 0x7f1b9c0c49f0 2023-11-24T08:09:39.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.013+0000 7f1b9a7fc640 1 --2- 172.21.15.106:0/1849351766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c0c49f0 0x7f1b9c06d700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0ccae0 tx=0x7f1b9002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66f29d29688f5c57 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:39.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.013+0000 7f1b99ffb640 1 -- 172.21.15.106:0/1849351766 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9002f8e0 con 0x7f1b9c0c49f0 2023-11-24T08:09:39.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.013+0000 7f1b99ffb640 1 -- 172.21.15.106:0/1849351766 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b9002fa40 con 0x7f1b9c0c49f0 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.013+0000 7f1b99ffb640 1 -- 172.21.15.106:0/1849351766 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b9002fdd0 con 0x7f1b9c0c49f0 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 -- 172.21.15.106:0/1849351766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c0c49f0 msgr2=0x7f1b9c06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/1849351766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c0c49f0 0x7f1b9c06d700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f1b9c0ccae0 tx=0x7f1b9002db20 comp rx=0 tx=0).stop 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 -- 172.21.15.106:0/1849351766 shutdown_connections 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/1849351766 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c06dc40 0x7f1b9c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/1849351766 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c0c49f0 0x7f1b9c06d700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/1849351766 >> v2:172.21.15.156:3300/0 conn(0x7f1b9c072160 0x7f1b9c15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 -- 172.21.15.106:0/1849351766 >> 172.21.15.106:0/1849351766 conn(0x7f1b9c0b0c50 msgr2=0x7f1b9c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 -- 172.21.15.106:0/1849351766 shutdown_connections 2023-11-24T08:09:39.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.014+0000 7f1ba16ee640 1 -- 172.21.15.106:0/1849351766 wait complete. 2023-11-24T08:09:39.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.016+0000 7f1ba16ee640 1 Processor -- start 2023-11-24T08:09:39.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.016+0000 7f1ba16ee640 1 -- start start 2023-11-24T08:09:39.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1ba16ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 0x7f1b9c149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1ba16ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c072160 0x7f1b9c0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1ba16ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b9c0c49f0 0x7f1b9c0c6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1ba16ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0c6610 con 0x7f1b9c06dc40 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1ba16ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b3130 con 0x7f1b9c0c49f0 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1ba16ee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b33d0 con 0x7f1b9c072160 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 0x7f1b9c149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 0x7f1b9c149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51750/0 (socket says 172.21.15.106:51750) 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 -- 172.21.15.106:0/3626509650 learned_addr learned my addr 172.21.15.106:0/3626509650 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 -- 172.21.15.106:0/3626509650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c072160 msgr2=0x7f1b9c0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c072160 0x7f1b9c0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 -- 172.21.15.106:0/3626509650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b9c0c49f0 msgr2=0x7f1b9c0c6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b9c0c49f0 0x7f1b9c0c6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.017+0000 7f1b9affd640 1 -- 172.21.15.106:0/3626509650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900094d0 con 0x7f1b9c06dc40 2023-11-24T08:09:39.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.018+0000 7f1b9affd640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 0x7f1b9c149210 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b4a0 tx=0x7f1b8c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:39.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.018+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c010610 con 0x7f1b9c06dc40 2023-11-24T08:09:39.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.018+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c010770 con 0x7f1b9c06dc40 2023-11-24T08:09:39.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.018+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c0b36b0 con 0x7f1b9c06dc40 2023-11-24T08:09:39.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.018+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c010b20 con 0x7f1b9c06dc40 2023-11-24T08:09:39.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.018+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c0b3c70 con 0x7f1b9c06dc40 2023-11-24T08:09:39.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.019+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c04e8b0 con 0x7f1b9c06dc40 2023-11-24T08:09:39.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.020+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f1b8c010c80 con 0x7f1b9c06dc40 2023-11-24T08:09:39.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.021+0000 7f1b837fe640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b8806ae40 0x7f1b8806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.021+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1b8c0893a0 con 0x7f1b9c06dc40 2023-11-24T08:09:39.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.021+0000 7f1b9a7fc640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b8806ae40 0x7f1b8806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.022+0000 7f1b9a7fc640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b8806ae40 0x7f1b8806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b90038000 tx=0x7f1b90032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:39.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.025+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1b8c054320 con 0x7f1b9c06dc40 2023-11-24T08:09:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.211+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1b9c15dab0 con 0x7f1b9c06dc40 2023-11-24T08:09:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.212+0000 7f1b837fe640 1 -- 172.21.15.106:0/3626509650 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1b8c057940 con 0x7f1b9c06dc40 2023-11-24T08:09:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.214+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b8806ae40 msgr2=0x7f1b8806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.214+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b8806ae40 0x7f1b8806d300 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f1b90038000 tx=0x7f1b90032000 comp rx=0 tx=0).stop 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.214+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 msgr2=0x7f1b9c149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.214+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 0x7f1b9c149210 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c00b4a0 tx=0x7f1b8c00b960 comp rx=0 tx=0).stop 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 shutdown_connections 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b9c0c49f0 0x7f1b9c0c6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b8806ae40 0x7f1b8806d300 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1b9c072160 0x7f1b9c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 --2- 172.21.15.106:0/3626509650 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1b9c06dc40 0x7f1b9c149210 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 >> 172.21.15.106:0/3626509650 conn(0x7f1b9c0b0c50 msgr2=0x7f1b9c15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 shutdown_connections 2023-11-24T08:09:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.215+0000 7f1ba16ee640 1 -- 172.21.15.106:0/3626509650 wait complete. 2023-11-24T08:09:39.235 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-24T08:09:39.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:39.236 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-24T08:09:39.346 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:09:39.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:09:39.347 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-24T08:09:39.396 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:09:39.397 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:39.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.611+0000 7f3ff07a9640 1 Processor -- start 2023-11-24T08:09:39.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.612+0000 7f3ff07a9640 1 -- start start 2023-11-24T08:09:39.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.613+0000 7f3ff07a9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3fe81609e0 0x7f3fe806bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.613+0000 7f3ff07a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe80ceb70 con 0x7f3fe80ce010 2023-11-24T08:09:39.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.613+0000 7f3ff07a9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe806c1f0 con 0x7f3fe80d31a0 2023-11-24T08:09:39.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.613+0000 7f3ff07a9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe80d1070 con 0x7f3fe81609e0 2023-11-24T08:09:39.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3feed46640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3fe80ce010 0x7f3fe80ce460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50760/0 (socket says 172.21.15.106:50760) 2023-11-24T08:09:39.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3feed46640 1 -- 172.21.15.106:0/3705312370 learned_addr learned my addr 172.21.15.106:0/3705312370 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:39.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3fee545640 1 --2- 172.21.15.106:0/3705312370 >> v2:172.21.15.156:3300/0 conn(0x7f3fe81609e0 0x7f3fe806bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817221841 0 0) 0x7f3fe80ceb70 con 0x7f3fe80ce010 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4003520 con 0x7f3fe80ce010 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 336834313 0 0) 0x7f3fd4003520 con 0x7f3fe80ce010 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 >> v2:172.21.15.156:3300/0 conn(0x7f3fe81609e0 msgr2=0x7f3fe806bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.614+0000 7f3fed543640 1 --2- 172.21.15.106:0/3705312370 >> v2:172.21.15.156:3300/0 conn(0x7f3fe81609e0 0x7f3fe806bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 >> v1:172.21.15.156:6789/0 conn(0x7f3fe80d31a0 legacy=0x7f3fe80d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe80cedc0 con 0x7f3fe80ce010 2023-11-24T08:09:39.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f3fe00019f0 con 0x7f3fe80ce010 2023-11-24T08:09:39.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 >> v1:172.21.15.106:6789/0 conn(0x7f3fe80ce010 legacy=0x7f3fe80ce460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd4003970 0x7f3fd4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fd40047c0 0x7f3fd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fd4005180 0x7f3fd400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80d1070 con 0x7f3fd40047c0 2023-11-24T08:09:39.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd400c020 con 0x7f3fd4003970 2023-11-24T08:09:39.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd400c1e0 con 0x7f3fd4005180 2023-11-24T08:09:39.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fedd44640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fd40047c0 0x7f3fd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fee545640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd4003970 0x7f3fd4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3feed46640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fd4005180 0x7f3fd400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fedd44640 1 -- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fd4005180 msgr2=0x7f3fd400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:39.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fedd44640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fd4005180 0x7f3fd400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fedd44640 1 -- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd4003970 msgr2=0x7f3fd4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:39.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fedd44640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd4003970 0x7f3fd4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fedd44640 1 -- 172.21.15.106:0/3705312370 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4003520 con 0x7f3fd40047c0 2023-11-24T08:09:39.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3fee545640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd4003970 0x7f3fd4003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:39.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3feed46640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fd4005180 0x7f3fd400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:39.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.615+0000 7f3feed46640 1 -- 172.21.15.106:0/3705312370 reap_dead start 2023-11-24T08:09:39.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.616+0000 7f3fedd44640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fd40047c0 0x7f3fd4004c40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8001720 tx=0x7f3fd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f3b769b74fa582 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:39.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd80052d0 con 0x7f3fd40047c0 2023-11-24T08:09:39.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd802e050 con 0x7f3fd40047c0 2023-11-24T08:09:39.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3fed543640 1 -- 172.21.15.106:0/3705312370 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8003830 con 0x7f3fd40047c0 2023-11-24T08:09:39.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 -- 172.21.15.106:0/3705312370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fd40047c0 msgr2=0x7f3fd4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fd40047c0 0x7f3fd4004c40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8001720 tx=0x7f3fd40057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 -- 172.21.15.106:0/3705312370 shutdown_connections 2023-11-24T08:09:39.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/3705312370 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fd40047c0 0x7f3fd4004c40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 -- 172.21.15.106:0/3705312370 >> 172.21.15.106:0/3705312370 conn(0x7f3fe808af60 msgr2=0x7f3fe809c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:39.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 -- 172.21.15.106:0/3705312370 shutdown_connections 2023-11-24T08:09:39.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.617+0000 7f3ff07a9640 1 -- 172.21.15.106:0/3705312370 wait complete. 2023-11-24T08:09:39.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.619+0000 7f3ff07a9640 1 Processor -- start 2023-11-24T08:09:39.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.619+0000 7f3ff07a9640 1 -- start start 2023-11-24T08:09:39.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3ff07a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd40047c0 0x7f3fe81f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3ff07a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fe814c1b0 0x7f3fe81f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3ff07a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 0x7f3fe815f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3ff07a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe814c7e0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3ff07a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe814d080 con 0x7f3fd40047c0 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3ff07a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe80b4b10 con 0x7f3fe814c1b0 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3feed46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 0x7f3fe815f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3feed46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 0x7f3fe815f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51762/0 (socket says 172.21.15.106:51762) 2023-11-24T08:09:39.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.620+0000 7f3feed46640 1 -- 172.21.15.106:0/1788880381 learned_addr learned my addr 172.21.15.106:0/1788880381 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:39.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3feed46640 1 -- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fe814c1b0 msgr2=0x7f3fe81f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3feed46640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fe814c1b0 0x7f3fe81f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3feed46640 1 -- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd40047c0 msgr2=0x7f3fe81f6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3fee545640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd40047c0 0x7f3fe81f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3feed46640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd40047c0 0x7f3fe81f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3feed46640 1 -- 172.21.15.106:0/1788880381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe814c9c0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3feed46640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 0x7f3fe815f650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3fe808bc10 tx=0x7f3fe000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0004d60 con 0x7f3fe815f1f0 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe000bb80 con 0x7f3fe815f1f0 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe814cca0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.621+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0007070 con 0x7f3fe815f1f0 2023-11-24T08:09:39.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.622+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe815fb90 con 0x7f3fe815f1f0 2023-11-24T08:09:39.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.623+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe814d1c0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.624+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f3fe00071d0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.625+0000 7f3fdeffd640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fcc06ae40 0x7f3fcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:39.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.625+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3fe0089ab0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.625+0000 7f3fee545640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fcc06ae40 0x7f3fcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:39.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.626+0000 7f3fee545640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fcc06ae40 0x7f3fcc06d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009a40 tx=0x7f3fe4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:39.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.629+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3fe814d1c0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.853+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fe814d1c0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.855+0000 7f3fdeffd640 1 -- 172.21.15.106:0/1788880381 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f3fe814d1c0 con 0x7f3fe815f1f0 2023-11-24T08:09:39.857 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:39.858 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:39.858 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:39.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fcc06ae40 msgr2=0x7f3fcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fcc06ae40 0x7f3fcc06d300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3fe4009a40 tx=0x7f3fe4008040 comp rx=0 tx=0).stop 2023-11-24T08:09:39.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 msgr2=0x7f3fe815f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:39.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 0x7f3fe815f650 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f3fe808bc10 tx=0x7f3fe000b4a0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 shutdown_connections 2023-11-24T08:09:39.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3fe815f1f0 0x7f3fe815f650 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3fcc06ae40 0x7f3fcc06d300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3fe814c1b0 0x7f3fe81f7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 --2- 172.21.15.106:0/1788880381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3fd40047c0 0x7f3fe81f6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 >> 172.21.15.106:0/1788880381 conn(0x7f3fe808af60 msgr2=0x7f3fe80ca4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 shutdown_connections 2023-11-24T08:09:39.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:39.857+0000 7f3ff07a9640 1 -- 172.21.15.106:0/1788880381 wait complete. 2023-11-24T08:09:39.877 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:40.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.084+0000 7f8e25bc9640 1 Processor -- start 2023-11-24T08:09:40.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.084+0000 7f8e25bc9640 1 -- start start 2023-11-24T08:09:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.085+0000 7f8e25bc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e2015b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.085+0000 7f8e25bc9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2015e3f0 con 0x7f8e2015df80 2023-11-24T08:09:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.085+0000 7f8e25bc9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e2015ab80 con 0x7f8e200b2250 2023-11-24T08:09:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.085+0000 7f8e25bc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2015a990 con 0x7f8e2015b080 2023-11-24T08:09:40.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.086+0000 7f8e16ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8e2015df80 0x7f8e200ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50776/0 (socket says 172.21.15.106:50776) 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.086+0000 7f8e16ffd640 1 -- 172.21.15.106:0/1220843753 learned_addr learned my addr 172.21.15.106:0/1220843753 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.086+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3104798111 0 0) 0x7f8e2015e3f0 con 0x7f8e2015df80 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.086+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e0c003520 con 0x7f8e2015df80 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.086+0000 7f8e1ffff640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e2015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.086+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3210689173 0 0) 0x7f8e0c003520 con 0x7f8e2015df80 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 msgr2=0x7f8e2015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e2015b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 >> v1:172.21.15.156:6789/0 conn(0x7f8e200b2250 legacy=0x7f8e200b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e200b2d40 con 0x7f8e2015df80 2023-11-24T08:09:40.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f8e08001670 con 0x7f8e2015df80 2023-11-24T08:09:40.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 >> v1:172.21.15.106:6789/0 conn(0x7f8e2015df80 legacy=0x7f8e200ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e0c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e0c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e0c004d00 0x7f8e0c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c00bb60 con 0x7f8e0c0039a0 2023-11-24T08:09:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c00bca0 con 0x7f8e0c004d00 2023-11-24T08:09:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0c00be60 con 0x7f8e2015b080 2023-11-24T08:09:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e0c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1ffff640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e0c004d00 0x7f8e0c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e16ffd640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e0c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 msgr2=0x7f8e0c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e0c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e0c004d00 msgr2=0x7f8e0c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e0c004d00 0x7f8e0c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/1220843753 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e0c003520 con 0x7f8e0c0039a0 2023-11-24T08:09:40.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.087+0000 7f8e16ffd640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2015b080 0x7f8e0c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:40.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.088+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e0c003dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8e10001750 tx=0x7f8e0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916f5806043ce475 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100052d0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1002e050 con 0x7f8e0c0039a0 2023-11-24T08:09:40.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e1effd640 1 -- 172.21.15.106:0/1220843753 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100035f0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e25bc9640 1 -- 172.21.15.106:0/1220843753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 msgr2=0x7f8e0c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/1220843753 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e0c003dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8e10001750 tx=0x7f8e0c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e1ffff640 1 -- 172.21.15.106:0/1220843753 reap_dead start 2023-11-24T08:09:40.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e25bc9640 1 -- 172.21.15.106:0/1220843753 shutdown_connections 2023-11-24T08:09:40.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e25bc9640 1 -- 172.21.15.106:0/1220843753 >> 172.21.15.106:0/1220843753 conn(0x7f8e2006a920 msgr2=0x7f8e2006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:40.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e25bc9640 1 -- 172.21.15.106:0/1220843753 shutdown_connections 2023-11-24T08:09:40.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.089+0000 7f8e25bc9640 1 -- 172.21.15.106:0/1220843753 wait complete. 2023-11-24T08:09:40.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.091+0000 7f8e25bc9640 1 Processor -- start 2023-11-24T08:09:40.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.091+0000 7f8e25bc9640 1 -- start start 2023-11-24T08:09:40.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e25bc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e200ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e25bc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2014ea90 0x7f8e200adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e25bc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2009b9c0 0x7f8e2009be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e25bc9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2009ce20 con 0x7f8e0c0039a0 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e25bc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2009d100 con 0x7f8e2014ea90 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e25bc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e20136b00 con 0x7f8e2009b9c0 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e1f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e200ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e1f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e200ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51790/0 (socket says 172.21.15.106:51790) 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.092+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/861379870 learned_addr learned my addr 172.21.15.106:0/861379870 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e16ffd640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2014ea90 0x7f8e200adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1ffff640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2009b9c0 0x7f8e2009be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2009b9c0 msgr2=0x7f8e2009be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2009b9c0 0x7f8e2009be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2014ea90 msgr2=0x7f8e200adaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2014ea90 0x7f8e200adaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1f7fe640 1 -- 172.21.15.106:0/861379870 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e20136de0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1ffff640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2009b9c0 0x7f8e2009be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e16ffd640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2014ea90 0x7f8e200adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.093+0000 7f8e1f7fe640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e200ad5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e10008fd0 tx=0x7f8e10001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.094+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e100076c0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.094+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e10007a30 con 0x7f8e0c0039a0 2023-11-24T08:09:40.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.094+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e201370c0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.094+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e10006c00 con 0x7f8e0c0039a0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.094+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2014e540 con 0x7f8e0c0039a0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.095+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e20137250 con 0x7f8e0c0039a0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.096+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8e100035f0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.096+0000 7f8e1cff9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e0006ae40 0x7f8e0006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.096+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8e100ab880 con 0x7f8e0c0039a0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.097+0000 7f8e16ffd640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e0006ae40 0x7f8e0006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.097+0000 7f8e16ffd640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e0006ae40 0x7f8e0006d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e08000c00 tx=0x7f8e08009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.101+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e1007a3b0 con 0x7f8e0c0039a0 2023-11-24T08:09:40.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.278+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e20067470 con 0x7f8e0c0039a0 2023-11-24T08:09:40.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.281+0000 7f8e1cff9640 1 -- 172.21.15.106:0/861379870 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16793 (secure 0 0 0) 0x7f8e10079f40 con 0x7f8e0c0039a0 2023-11-24T08:09:40.284 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:40.284 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:35.282948+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e0006ae40 msgr2=0x7f8e0006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e0006ae40 0x7f8e0006d300 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8e08000c00 tx=0x7f8e08009040 comp rx=0 tx=0).stop 2023-11-24T08:09:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 msgr2=0x7f8e200ad5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e200ad5b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f8e10008fd0 tx=0x7f8e10001120 comp rx=0 tx=0).stop 2023-11-24T08:09:40.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 shutdown_connections 2023-11-24T08:09:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e0006ae40 0x7f8e0006d300 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e2009b9c0 0x7f8e2009be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e2014ea90 0x7f8e200adaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 --2- 172.21.15.106:0/861379870 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e0c0039a0 0x7f8e200ad5b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 >> 172.21.15.106:0/861379870 conn(0x7f8e2006a920 msgr2=0x7f8e200cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 shutdown_connections 2023-11-24T08:09:40.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.283+0000 7f8e25bc9640 1 -- 172.21.15.106:0/861379870 wait complete. 2023-11-24T08:09:40.304 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:08:45.474425+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:08:46.647137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:40.305 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:40.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.512+0000 7fd8bf583640 1 Processor -- start 2023-11-24T08:09:40.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.512+0000 7fd8bf583640 1 -- start start 2023-11-24T08:09:40.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.514+0000 7fd8bf583640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd8b80cc0f0 0x7fd8b80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.514+0000 7fd8bf583640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b815eea0 con 0x7fd8b815b440 2023-11-24T08:09:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.514+0000 7fd8bf583640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80abf30 con 0x7fd8b80c57e0 2023-11-24T08:09:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.514+0000 7fd8bf583640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b815b820 con 0x7fd8b80cc0f0 2023-11-24T08:09:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.514+0000 7fd8bd31f640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd8b815b440 0x7fd8b80cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50786/0 (socket says 172.21.15.106:50786) 2023-11-24T08:09:40.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.514+0000 7fd8bd31f640 1 -- 172.21.15.106:0/3822535417 learned_addr learned my addr 172.21.15.106:0/3822535417 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8bcb1e640 1 --2- 172.21.15.106:0/3822535417 >> v2:172.21.15.156:3300/0 conn(0x7fd8b80cc0f0 0x7fd8b80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488863588 0 0) 0x7fd8b815eea0 con 0x7fd8b815b440 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a4003520 con 0x7fd8b815b440 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 244895192 0 0) 0x7fd8a4003520 con 0x7fd8b815b440 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 >> v2:172.21.15.156:3300/0 conn(0x7fd8b80cc0f0 msgr2=0x7fd8b80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 --2- 172.21.15.106:0/3822535417 >> v2:172.21.15.156:3300/0 conn(0x7fd8b80cc0f0 0x7fd8b80c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 >> v1:172.21.15.156:6789/0 conn(0x7fd8b80c57e0 legacy=0x7fd8b806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b815f0f0 con 0x7fd8b815b440 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fd8b4001670 con 0x7fd8b815b440 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 >> v1:172.21.15.106:6789/0 conn(0x7fd8b815b440 legacy=0x7fd8b80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8a40039c0 0x7fd8a4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40047c0 0x7fd8a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a40051d0 0x7fd8a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b815b820 con 0x7fd8a40039c0 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a400c120 con 0x7fd8a40051d0 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.515+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a400c2e0 con 0x7fd8a40047c0 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8a40039c0 0x7fd8a4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bdb20640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a40051d0 0x7fd8a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bcb1e640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40047c0 0x7fd8a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bd31f640 1 -- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40047c0 msgr2=0x7fd8a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40047c0 0x7fd8a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bd31f640 1 -- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a40051d0 msgr2=0x7fd8a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8a40051d0 0x7fd8a400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bdb20640 1 -- 172.21.15.106:0/3822535417 reap_dead start 2023-11-24T08:09:40.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bd31f640 1 -- 172.21.15.106:0/3822535417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a4003520 con 0x7fd8a40039c0 2023-11-24T08:09:40.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.516+0000 7fd8bcb1e640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40047c0 0x7fd8a4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:40.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.517+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8a40039c0 0x7fd8a4003de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4001eb0 tx=0x7fd8a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b663638014a80b46 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.517+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b4005ad0 con 0x7fd8a40039c0 2023-11-24T08:09:40.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.517+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b4005c30 con 0x7fd8a40039c0 2023-11-24T08:09:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.517+0000 7fd8affff640 1 -- 172.21.15.106:0/3822535417 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b402ca00 con 0x7fd8a40039c0 2023-11-24T08:09:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 -- 172.21.15.106:0/3822535417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8a40039c0 msgr2=0x7fd8a4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8a40039c0 0x7fd8a4003de0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4001eb0 tx=0x7fd8a40057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 -- 172.21.15.106:0/3822535417 shutdown_connections 2023-11-24T08:09:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 --2- 172.21.15.106:0/3822535417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8a40039c0 0x7fd8a4003de0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 -- 172.21.15.106:0/3822535417 >> 172.21.15.106:0/3822535417 conn(0x7fd8b80b0c50 msgr2=0x7fd8b80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:40.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 -- 172.21.15.106:0/3822535417 shutdown_connections 2023-11-24T08:09:40.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.518+0000 7fd8bf583640 1 -- 172.21.15.106:0/3822535417 wait complete. 2023-11-24T08:09:40.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.520+0000 7fd8bf583640 1 Processor -- start 2023-11-24T08:09:40.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.520+0000 7fd8bf583640 1 -- start start 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bf583640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40039c0 0x7fd8b809cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bf583640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b80c96c0 0x7fd8b815e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bf583640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 0x7fd8b815bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bf583640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b809bd30 con 0x7fd8b8155db0 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bf583640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b815ec40 con 0x7fd8b80c96c0 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bf583640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80c9b40 con 0x7fd8a40039c0 2023-11-24T08:09:40.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bdb20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 0x7fd8b815bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bdb20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 0x7fd8b815bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51808/0 (socket says 172.21.15.106:51808) 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bdb20640 1 -- 172.21.15.106:0/2782910349 learned_addr learned my addr 172.21.15.106:0/2782910349 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40039c0 0x7fd8b809cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.521+0000 7fd8bcb1e640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b80c96c0 0x7fd8b815e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bdb20640 1 -- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40039c0 msgr2=0x7fd8b809cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bdb20640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40039c0 0x7fd8b809cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bdb20640 1 -- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b80c96c0 msgr2=0x7fd8b815e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bdb20640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b80c96c0 0x7fd8b815e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bdb20640 1 -- 172.21.15.106:0/2782910349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b809bf80 con 0x7fd8b8155db0 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bcb1e640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b80c96c0 0x7fd8b815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bdb20640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 0x7fd8b815bd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8b809d280 tx=0x7fd8b000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0006d60 con 0x7fd8b8155db0 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8b0003970 con 0x7fd8b8155db0 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b809c260 con 0x7fd8b8155db0 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0003d20 con 0x7fd8b8155db0 2023-11-24T08:09:40.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.522+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b809c820 con 0x7fd8b8155db0 2023-11-24T08:09:40.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.524+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b809ca40 con 0x7fd8b8155db0 2023-11-24T08:09:40.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.524+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd8b0005070 con 0x7fd8b8155db0 2023-11-24T08:09:40.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.525+0000 7fd8adffb640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd89c06ae70 0x7fd89c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.525+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd8b0088f10 con 0x7fd8b8155db0 2023-11-24T08:09:40.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.526+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd89c06ae70 0x7fd89c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.526+0000 7fd8bd31f640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd89c06ae70 0x7fd89c06d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4034000 tx=0x7fd8b4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.530+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd8b0053ea0 con 0x7fd8b8155db0 2023-11-24T08:09:40.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.718+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8b804e8b0 con 0x7fd8b8155db0 2023-11-24T08:09:40.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.718+0000 7fd8adffb640 1 -- 172.21.15.106:0/2782910349 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8b0019070 con 0x7fd8b8155db0 2023-11-24T08:09:40.721 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:40.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd89c06ae70 msgr2=0x7fd89c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd89c06ae70 0x7fd89c06d330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd8b4034000 tx=0x7fd8b4009030 comp rx=0 tx=0).stop 2023-11-24T08:09:40.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 msgr2=0x7fd8b815bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 0x7fd8b815bd50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd8b809d280 tx=0x7fd8b000ea00 comp rx=0 tx=0).stop 2023-11-24T08:09:40.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 shutdown_connections 2023-11-24T08:09:40.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd8b8155db0 0x7fd8b815bd50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8b80c96c0 0x7fd8b815e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd89c06ae70 0x7fd89c06d330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 --2- 172.21.15.106:0/2782910349 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd8a40039c0 0x7fd8b809cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 >> 172.21.15.106:0/2782910349 conn(0x7fd8b80b0c50 msgr2=0x7fd8b8071d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:40.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 shutdown_connections 2023-11-24T08:09:40.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.721+0000 7fd8bf583640 1 -- 172.21.15.106:0/2782910349 wait complete. 2023-11-24T08:09:40.744 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:09:40.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.954+0000 7f2f3f59e640 1 Processor -- start 2023-11-24T08:09:40.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.954+0000 7f2f3f59e640 1 -- start start 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3f59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2f400cb580 0x7f2f400c2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3f59e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f400cb960 con 0x7f2f400c25d0 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3f59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f400c29e0 con 0x7f2f400ca940 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3f59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f40161e40 con 0x7f2f400cb580 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2f400c25d0 0x7f2f40165420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50792/0 (socket says 172.21.15.106:50792) 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3ed9d640 1 -- 172.21.15.106:0/3873614632 learned_addr learned my addr 172.21.15.106:0/3873614632 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3873614632 >> v2:172.21.15.156:3300/0 conn(0x7f2f400cb580 0x7f2f400c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.956+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 78790981 0 0) 0x7f2f400cb960 con 0x7f2f400c25d0 2023-11-24T08:09:40.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c003520 con 0x7f2f400c25d0 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3950607744 0 0) 0x7f2f2c003520 con 0x7f2f400c25d0 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 >> v2:172.21.15.156:3300/0 conn(0x7f2f400cb580 msgr2=0x7f2f400c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 --2- 172.21.15.106:0/3873614632 >> v2:172.21.15.156:3300/0 conn(0x7f2f400cb580 0x7f2f400c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 >> v1:172.21.15.156:6789/0 conn(0x7f2f400ca940 legacy=0x7f2f400b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f40162120 con 0x7f2f400c25d0 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2f380019f0 con 0x7f2f400c25d0 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 >> v1:172.21.15.106:6789/0 conn(0x7f2f400c25d0 legacy=0x7f2f40165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f2c0039c0 0x7f2f2c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0047c0 0x7f2f2c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f2c005180 0x7f2f2c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40161e40 con 0x7f2f2c0039c0 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00c080 con 0x7f2f2c005180 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00c240 con 0x7f2f2c0047c0 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.957+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f2c0039c0 0x7f2f2c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3ed9d640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f2c005180 0x7f2f2c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f35d9b640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0047c0 0x7f2f2c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3e59c640 1 -- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0047c0 msgr2=0x7f2f2c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0047c0 0x7f2f2c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3e59c640 1 -- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f2c005180 msgr2=0x7f2f2c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f2c005180 0x7f2f2c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3e59c640 1 -- 172.21.15.106:0/3873614632 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c003520 con 0x7f2f2c0039c0 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.958+0000 7f2f3ed9d640 1 -- 172.21.15.106:0/3873614632 reap_dead start 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.959+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f2c0039c0 0x7f2f2c003de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009ac0 tx=0x7f2f2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=717b92af89c9f1e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.959+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3002c6d0 con 0x7f2f2c0039c0 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.959+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3002c830 con 0x7f2f2c0039c0 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.959+0000 7f2f3dd9b640 1 -- 172.21.15.106:0/3873614632 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3002cbc0 con 0x7f2f2c0039c0 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3873614632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f2c0039c0 msgr2=0x7f2f2c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f2c0039c0 0x7f2f2c003de0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009ac0 tx=0x7f2f2c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3873614632 shutdown_connections 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3873614632 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f2c0039c0 0x7f2f2c003de0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3873614632 >> 172.21.15.106:0/3873614632 conn(0x7f2f400cbb40 msgr2=0x7f2f400cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3873614632 shutdown_connections 2023-11-24T08:09:40.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.960+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3873614632 wait complete. 2023-11-24T08:09:40.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.962+0000 7f2f3f59e640 1 Processor -- start 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.962+0000 7f2f3f59e640 1 -- start start 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.963+0000 7f2f3f59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0039c0 0x7f2f400999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.963+0000 7f2f3f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 0x7f2f40157aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f3f59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f400c4f50 0x7f2f40165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f3f59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400b25f0 con 0x7f2f40158300 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f3f59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f400b1ce0 con 0x7f2f400c4f50 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f3f59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f40157fe0 con 0x7f2f2c0039c0 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 0x7f2f40157aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 0x7f2f40157aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51828/0 (socket says 172.21.15.106:51828) 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 -- 172.21.15.106:0/3731629160 learned_addr learned my addr 172.21.15.106:0/3731629160 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0039c0 0x7f2f400999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f3ed9d640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f400c4f50 0x7f2f40165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 -- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0039c0 msgr2=0x7f2f400999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0039c0 0x7f2f400999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 -- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f400c4f50 msgr2=0x7f2f40165300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f400c4f50 0x7f2f40165300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.964+0000 7f2f35d9b640 1 -- 172.21.15.106:0/3731629160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f400c58a0 con 0x7f2f40158300 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.965+0000 7f2f35d9b640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 0x7f2f40157aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800d4f0 tx=0x7f2f2800d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.965+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28005480 con 0x7f2f40158300 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.965+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f28003610 con 0x7f2f40158300 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.965+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f400b1f30 con 0x7f2f40158300 2023-11-24T08:09:40.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.965+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f28003980 con 0x7f2f40158300 2023-11-24T08:09:40.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.965+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f400c53d0 con 0x7f2f40158300 2023-11-24T08:09:40.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.966+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f400b2730 con 0x7f2f40158300 2023-11-24T08:09:40.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.967+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2f28003ae0 con 0x7f2f40158300 2023-11-24T08:09:40.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.968+0000 7f2f377fe640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f2006ae40 0x7f2f2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:40.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.968+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f2f280895c0 con 0x7f2f40158300 2023-11-24T08:09:40.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.968+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f2006ae40 0x7f2f2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:40.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.969+0000 7f2f3e59c640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f2006ae40 0x7f2f2006d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f30036000 tx=0x7f2f30030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:40.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:40.973+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2f400b2730 con 0x7f2f40158300 2023-11-24T08:09:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.165+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f400b2730 con 0x7f2f40158300 2023-11-24T08:09:41.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.166+0000 7f2f377fe640 1 -- 172.21.15.106:0/3731629160 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f400b2730 con 0x7f2f40158300 2023-11-24T08:09:41.169 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:09:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.168+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f2006ae40 msgr2=0x7f2f2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.168+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f2006ae40 0x7f2f2006d300 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f2f30036000 tx=0x7f2f30030000 comp rx=0 tx=0).stop 2023-11-24T08:09:41.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.168+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 msgr2=0x7f2f40157aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.168+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 0x7f2f40157aa0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800d4f0 tx=0x7f2f2800d9b0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 shutdown_connections 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f400c4f50 0x7f2f40165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f40158300 0x7f2f40157aa0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f2006ae40 0x7f2f2006d300 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 --2- 172.21.15.106:0/3731629160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f2c0039c0 0x7f2f400999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 >> 172.21.15.106:0/3731629160 conn(0x7f2f400cbb40 msgr2=0x7f2f400b0550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 shutdown_connections 2023-11-24T08:09:41.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.169+0000 7f2f3f59e640 1 -- 172.21.15.106:0/3731629160 wait complete. 2023-11-24T08:09:41.190 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:09:41.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.402+0000 7fb0c9147640 1 Processor -- start 2023-11-24T08:09:41.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.402+0000 7fb0c9147640 1 -- start start 2023-11-24T08:09:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.403+0000 7fb0c9147640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 0x7fb0c41681a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.403+0000 7fb0c9147640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c415daa0 con 0x7fb0c40ce7f0 2023-11-24T08:09:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.403+0000 7fb0c9147640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c40ced40 con 0x7fb0c415d6c0 2023-11-24T08:09:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.403+0000 7fb0c9147640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40d41c0 con 0x7fb0c40b4700 2023-11-24T08:09:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.404+0000 7fb0c37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb0c40ce7f0 0x7fb0c40d17f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50796/0 (socket says 172.21.15.106:50796) 2023-11-24T08:09:41.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.404+0000 7fb0c37fe640 1 -- 172.21.15.106:0/2518695118 learned_addr learned my addr 172.21.15.106:0/2518695118 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:41.408 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.404+0000 7fb0c8946640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 0x7fb0c41681a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.404+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664858232 0 0) 0x7fb0c415daa0 con 0x7fb0c40ce7f0 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.404+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0ac003520 con 0x7fb0c40ce7f0 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3066570065 0 0) 0x7fb0ac003520 con 0x7fb0c40ce7f0 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 msgr2=0x7fb0c41681a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 0x7fb0c41681a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 >> v1:172.21.15.156:6789/0 conn(0x7fb0c415d6c0 legacy=0x7fb0c40d3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40b2c10 con 0x7fb0c40ce7f0 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fb0b8001690 con 0x7fb0c40ce7f0 2023-11-24T08:09:41.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 >> v1:172.21.15.106:6789/0 conn(0x7fb0c40ce7f0 legacy=0x7fb0c40d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0ac0039a0 0x7fb0ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac0047c0 0x7fb0ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 0x7fb0ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40d41c0 con 0x7fb0ac0039a0 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00bae0 con 0x7fb0ac0047c0 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ac00bca0 con 0x7fb0c40b4700 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.405+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0ac0039a0 0x7fb0ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c8946640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 0x7fb0ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c37fe640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac0047c0 0x7fb0ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 msgr2=0x7fb0ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:41.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c40b4700 0x7fb0ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac0047c0 msgr2=0x7fb0ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:41.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac0047c0 0x7fb0ac004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2518695118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ac003520 con 0x7fb0ac0039a0 2023-11-24T08:09:41.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.406+0000 7fb0c37fe640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ac0047c0 0x7fb0ac004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:41.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.407+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0ac0039a0 0x7fb0ac003dc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4001750 tx=0x7fb0ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb0bb0170cb44c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:41.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.407+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40052d0 con 0x7fb0ac0039a0 2023-11-24T08:09:41.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.407+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b402e050 con 0x7fb0ac0039a0 2023-11-24T08:09:41.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.407+0000 7fb0c2ffd640 1 -- 172.21.15.106:0/2518695118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b40035f0 con 0x7fb0ac0039a0 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c9147640 1 -- 172.21.15.106:0/2518695118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0ac0039a0 msgr2=0x7fb0ac003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c9147640 1 --2- 172.21.15.106:0/2518695118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0ac0039a0 0x7fb0ac003dc0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4001750 tx=0x7fb0ac0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c8946640 1 -- 172.21.15.106:0/2518695118 reap_dead start 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c9147640 1 -- 172.21.15.106:0/2518695118 shutdown_connections 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c9147640 1 -- 172.21.15.106:0/2518695118 >> 172.21.15.106:0/2518695118 conn(0x7fb0c409c5b0 msgr2=0x7fb0c40d3210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c9147640 1 -- 172.21.15.106:0/2518695118 shutdown_connections 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.408+0000 7fb0c9147640 1 -- 172.21.15.106:0/2518695118 wait complete. 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.410+0000 7fb0c9147640 1 Processor -- start 2023-11-24T08:09:41.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.410+0000 7fb0c9147640 1 -- start start 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c9147640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c41f5be0 0x7fb0c40cbd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c9147640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 0x7fb0c415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c9147640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c415bb00 0x7fb0c415ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c9147640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cc260 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c9147640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415bf80 con 0x7fb0c41f5be0 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c9147640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cd050 con 0x7fb0c415bb00 2023-11-24T08:09:41.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 0x7fb0c415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 0x7fb0c415b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51854/0 (socket says 172.21.15.106:51854) 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.411+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1891141254 learned_addr learned my addr 172.21.15.106:0/1891141254 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c415bb00 msgr2=0x7fb0c415ecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c37fe640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c415bb00 0x7fb0c415ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c41f5be0 0x7fb0c40cbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c41f5be0 msgr2=0x7fb0c40cbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c37fe640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c41f5be0 0x7fb0c40cbd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1891141254 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c41f7320 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c37fe640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 0x7fb0c415b5c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800da90 tx=0x7fb0b800df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:41.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8004cb0 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b8004e10 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c40cc440 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.412+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b80072f0 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.413+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c40cca00 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.414+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c404e840 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.415+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fb0b8007450 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.416+0000 7fb0c0ff9640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0b006ae70 0x7fb0b006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.416+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb0b8088eb0 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.416+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0b006ae70 0x7fb0b006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.417+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0b006ae70 0x7fb0b006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4033000 tx=0x7fb0b4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:41.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.420+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0b8053ec0 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.615+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0c40cd190 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.616+0000 7fb0c0ff9640 1 -- 172.21.15.106:0/1891141254 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb0b8019020 con 0x7fb0c41f6ea0 2023-11-24T08:09:41.619 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0b006ae70 msgr2=0x7fb0b006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0b006ae70 0x7fb0b006d330 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb0b4033000 tx=0x7fb0b4009030 comp rx=0 tx=0).stop 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 msgr2=0x7fb0c415b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 0x7fb0c415b5c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800da90 tx=0x7fb0b800df50 comp rx=0 tx=0).stop 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 shutdown_connections 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0b006ae70 0x7fb0b006d330 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c415bb00 0x7fb0c415ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c41f6ea0 0x7fb0c415b5c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 --2- 172.21.15.106:0/1891141254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c41f5be0 0x7fb0c40cbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 >> 172.21.15.106:0/1891141254 conn(0x7fb0c409c5b0 msgr2=0x7fb0c406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 shutdown_connections 2023-11-24T08:09:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.619+0000 7fb0c9147640 1 -- 172.21.15.106:0/1891141254 wait complete. 2023-11-24T08:09:41.641 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:41.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.851+0000 7fbd8eee6640 1 Processor -- start 2023-11-24T08:09:41.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.852+0000 7fbd8eee6640 1 -- start start 2023-11-24T08:09:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.853+0000 7fbd8eee6640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd8815c410 0x7fbd8815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.853+0000 7fbd8eee6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd880b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.853+0000 7fbd8eee6640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd8815bf80 con 0x7fbd88144b30 2023-11-24T08:09:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.853+0000 7fbd8eee6640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd880b27b0 con 0x7fbd8815c410 2023-11-24T08:09:41.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.853+0000 7fbd8eee6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd880b1560 con 0x7fbd880b1de0 2023-11-24T08:09:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.854+0000 7fbd8e6e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd880b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.854+0000 7fbd8e6e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd880b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:60564/0 (socket says 172.21.15.106:60564) 2023-11-24T08:09:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.854+0000 7fbd8d6e3640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbd8815c410 0x7fbd8815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.854+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/3588761001 learned_addr learned my addr 172.21.15.106:0/3588761001 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.855+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/3588761001 >> v2:172.21.15.156:3300/0 conn(0x7fbd8815c410 msgr2=0x7fbd8815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:41.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.855+0000 7fbd8e6e5640 1 --2- 172.21.15.106:0/3588761001 >> v2:172.21.15.156:3300/0 conn(0x7fbd8815c410 0x7fbd8815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.855+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/3588761001 >> v1:172.21.15.156:6789/0 conn(0x7fbd88144b30 legacy=0x7fbd880b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.855+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/3588761001 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd880b1280 con 0x7fbd880b1de0 2023-11-24T08:09:41.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.856+0000 7fbd8e6e5640 1 --2- 172.21.15.106:0/3588761001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd880b2270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009e50 tx=0x7fbd8002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ce0e29accf82743 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:41.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.857+0000 7fbd8cee2640 1 -- 172.21.15.106:0/3588761001 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8002caf0 con 0x7fbd880b1de0 2023-11-24T08:09:41.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.857+0000 7fbd8cee2640 1 -- 172.21.15.106:0/3588761001 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8002cc50 con 0x7fbd880b1de0 2023-11-24T08:09:41.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.857+0000 7fbd8cee2640 1 -- 172.21.15.106:0/3588761001 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800343f0 con 0x7fbd880b1de0 2023-11-24T08:09:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.857+0000 7fbd8eee6640 1 -- 172.21.15.106:0/3588761001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 msgr2=0x7fbd880b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.857+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/3588761001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd880b2270 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009e50 tx=0x7fbd8002bf50 comp rx=0 tx=0).stop 2023-11-24T08:09:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.858+0000 7fbd8eee6640 1 -- 172.21.15.106:0/3588761001 shutdown_connections 2023-11-24T08:09:41.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.858+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/3588761001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd880b2270 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.858+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/3588761001 >> v2:172.21.15.156:3300/0 conn(0x7fbd8815c410 0x7fbd8815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.858+0000 7fbd8eee6640 1 -- 172.21.15.106:0/3588761001 >> 172.21.15.106:0/3588761001 conn(0x7fbd880cb830 msgr2=0x7fbd880cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:41.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.858+0000 7fbd8eee6640 1 -- 172.21.15.106:0/3588761001 shutdown_connections 2023-11-24T08:09:41.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.858+0000 7fbd8eee6640 1 -- 172.21.15.106:0/3588761001 wait complete. 2023-11-24T08:09:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.860+0000 7fbd8eee6640 1 Processor -- start 2023-11-24T08:09:41.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.860+0000 7fbd8eee6640 1 -- start start 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8eee6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd88140b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8eee6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd88144b30 0x7fbd881e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8eee6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 0x7fbd881436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8eee6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88142090 con 0x7fbd8815c410 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8eee6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88141480 con 0x7fbd88144b30 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8eee6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88141760 con 0x7fbd880b1de0 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8e6e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 0x7fbd881436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8e6e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 0x7fbd881436d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51866/0 (socket says 172.21.15.106:51866) 2023-11-24T08:09:41.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/230844037 learned_addr learned my addr 172.21.15.106:0/230844037 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8dee4640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd88140b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.861+0000 7fbd8d6e3640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd88144b30 0x7fbd881e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 msgr2=0x7fbd88140b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8e6e5640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd88140b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd88144b30 msgr2=0x7fbd881e5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8e6e5640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd88144b30 0x7fbd881e5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8e6e5640 1 -- 172.21.15.106:0/230844037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd80009b30 con 0x7fbd8815c410 2023-11-24T08:09:41.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8d6e3640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd88144b30 0x7fbd881e5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8e6e5640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 0x7fbd881436d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009e50 tx=0x7fbd80004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80009510 con 0x7fbd8815c410 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd8003e450 con 0x7fbd8815c410 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8003e800 con 0x7fbd8815c410 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd88141a40 con 0x7fbd8815c410 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.862+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd88141050 con 0x7fbd8815c410 2023-11-24T08:09:41.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.863+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd88142200 con 0x7fbd8815c410 2023-11-24T08:09:41.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.864+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbd8003e960 con 0x7fbd8815c410 2023-11-24T08:09:41.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.865+0000 7fbd7e7fc640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd6c06ae40 0x7fbd6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:41.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.865+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbd800ac270 con 0x7fbd8815c410 2023-11-24T08:09:41.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.866+0000 7fbd8dee4640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd6c06ae40 0x7fbd6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:41.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.866+0000 7fbd8dee4640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd6c06ae40 0x7fbd6c06d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd880b2630 tx=0x7fbd84009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:41.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:41.870+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd80077320 con 0x7fbd8815c410 2023-11-24T08:09:42.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.095+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd8804e840 con 0x7fbd8815c410 2023-11-24T08:09:42.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.096+0000 7fbd7e7fc640 1 -- 172.21.15.106:0/230844037 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fbd8007a940 con 0x7fbd8815c410 2023-11-24T08:09:42.099 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:42.099 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:42.099 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd6c06ae40 msgr2=0x7fbd6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd6c06ae40 0x7fbd6c06d300 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbd880b2630 tx=0x7fbd84009040 comp rx=0 tx=0).stop 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 msgr2=0x7fbd881436d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 0x7fbd881436d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fbd80009e50 tx=0x7fbd80004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 shutdown_connections 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd8815c410 0x7fbd881436d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd88144b30 0x7fbd881e5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd6c06ae40 0x7fbd6c06d300 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 --2- 172.21.15.106:0/230844037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd880b1de0 0x7fbd88140b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 >> 172.21.15.106:0/230844037 conn(0x7fbd880cb830 msgr2=0x7fbd8815d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:42.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.099+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 shutdown_connections 2023-11-24T08:09:42.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.100+0000 7fbd8eee6640 1 -- 172.21.15.106:0/230844037 wait complete. 2023-11-24T08:09:42.122 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:42.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.328+0000 7f784780e640 1 Processor -- start 2023-11-24T08:09:42.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.329+0000 7f784780e640 1 -- start start 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.330+0000 7f784780e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78400cfb60 0x7f7840161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.330+0000 7f784780e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f784015dc80 0x7f784015e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.330+0000 7f784780e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78400ce0b0 con 0x7f7840157c70 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.330+0000 7f784780e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7840161960 con 0x7f78400cfb60 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.330+0000 7f784780e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f784015e670 con 0x7f784015dc80 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78400cfb60 0x7f7840161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f78400cfb60 0x7f7840161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51882/0 (socket says 172.21.15.106:51882) 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 -- 172.21.15.106:0/3798881910 learned_addr learned my addr 172.21.15.106:0/3798881910 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:42.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7845dab640 1 --2- 172.21.15.106:0/3798881910 >> v2:172.21.15.156:3300/0 conn(0x7f784015dc80 0x7f784015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 -- 172.21.15.106:0/3798881910 >> v2:172.21.15.156:3300/0 conn(0x7f784015dc80 msgr2=0x7f784015e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:42.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 --2- 172.21.15.106:0/3798881910 >> v2:172.21.15.156:3300/0 conn(0x7f784015dc80 0x7f784015e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 -- 172.21.15.106:0/3798881910 >> v1:172.21.15.156:6789/0 conn(0x7f7840157c70 legacy=0x7f78400cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:42.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.331+0000 7f7844da9640 1 -- 172.21.15.106:0/3798881910 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784015e8d0 con 0x7f78400cfb60 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.332+0000 7f7844da9640 1 --2- 172.21.15.106:0/3798881910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78400cfb60 0x7f7840161df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78400d0130 tx=0x7f783402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=130e998dfacd274 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.332+0000 7f783ffff640 1 -- 172.21.15.106:0/3798881910 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78340317e0 con 0x7f78400cfb60 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.332+0000 7f783ffff640 1 -- 172.21.15.106:0/3798881910 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834031940 con 0x7f78400cfb60 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.332+0000 7f783ffff640 1 -- 172.21.15.106:0/3798881910 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834031cd0 con 0x7f78400cfb60 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 -- 172.21.15.106:0/3798881910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78400cfb60 msgr2=0x7f7840161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 --2- 172.21.15.106:0/3798881910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78400cfb60 0x7f7840161df0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f78400d0130 tx=0x7f783402b880 comp rx=0 tx=0).stop 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 -- 172.21.15.106:0/3798881910 shutdown_connections 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 --2- 172.21.15.106:0/3798881910 >> v2:172.21.15.156:3300/0 conn(0x7f784015dc80 0x7f784015e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 --2- 172.21.15.106:0/3798881910 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f78400cfb60 0x7f7840161df0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 -- 172.21.15.106:0/3798881910 >> 172.21.15.106:0/3798881910 conn(0x7f784006a920 msgr2=0x7f784006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:42.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 -- 172.21.15.106:0/3798881910 shutdown_connections 2023-11-24T08:09:42.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.333+0000 7f784780e640 1 -- 172.21.15.106:0/3798881910 wait complete. 2023-11-24T08:09:42.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.335+0000 7f784780e640 1 Processor -- start 2023-11-24T08:09:42.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.335+0000 7f784780e640 1 -- start start 2023-11-24T08:09:42.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f784780e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78400cfb60 0x7f78401f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f784780e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840157c70 0x7f78401553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f784780e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 0x7f78401f5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f784780e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78401f4580 con 0x7f784015dc80 2023-11-24T08:09:42.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f784780e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78401f4860 con 0x7f7840157c70 2023-11-24T08:09:42.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f784780e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78401f4b40 con 0x7f78400cfb60 2023-11-24T08:09:42.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f7845dab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 0x7f78401f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f7845dab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 0x7f78401f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51890/0 (socket says 172.21.15.106:51890) 2023-11-24T08:09:42.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.336+0000 7f7845dab640 1 -- 172.21.15.106:0/638223477 learned_addr learned my addr 172.21.15.106:0/638223477 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:42.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f7845dab640 1 -- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78400cfb60 msgr2=0x7f78401f3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f78455aa640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78400cfb60 0x7f78401f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f7845dab640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78400cfb60 0x7f78401f3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f7845dab640 1 -- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840157c70 msgr2=0x7f78401553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f7845dab640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840157c70 0x7f78401553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f7845dab640 1 -- 172.21.15.106:0/638223477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78340094d0 con 0x7f784015dc80 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f78455aa640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78400cfb60 0x7f78401f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f7845dab640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 0x7f78401f5620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f783800c540 tx=0x7f783800ca00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838004220 con 0x7f784015dc80 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7838004380 con 0x7f784015dc80 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78401f4e20 con 0x7f784015dc80 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838003850 con 0x7f784015dc80 2023-11-24T08:09:42.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.337+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78401f4190 con 0x7f784015dc80 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.339+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78401f5340 con 0x7f784015dc80 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.339+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f78380039b0 con 0x7f784015dc80 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.340+0000 7f783dffb640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f782c06ae40 0x7f782c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.341+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f78380893b0 con 0x7f784015dc80 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.341+0000 7f78455aa640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f782c06ae40 0x7f782c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.341+0000 7f78455aa640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f782c06ae40 0x7f782c06d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7830007eb0 tx=0x7f78300073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:42.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.345+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f78401f5340 con 0x7f784015dc80 2023-11-24T08:09:42.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.535+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f78401f5340 con 0x7f784015dc80 2023-11-24T08:09:42.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.537+0000 7f783dffb640 1 -- 172.21.15.106:0/638223477 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16793 (secure 0 0 0) 0x7f78401f5340 con 0x7f784015dc80 2023-11-24T08:09:42.540 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:42.540 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:35.282948+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f782c06ae40 msgr2=0x7f782c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f782c06ae40 0x7f782c06d300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7830007eb0 tx=0x7f78300073c0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 msgr2=0x7f78401f5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 0x7f78401f5620 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f783800c540 tx=0x7f783800ca00 comp rx=0 tx=0).stop 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 shutdown_connections 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f784015dc80 0x7f78401f5620 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7840157c70 0x7f78401553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f782c06ae40 0x7f782c06d300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 --2- 172.21.15.106:0/638223477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f78400cfb60 0x7f78401f3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 >> 172.21.15.106:0/638223477 conn(0x7f784006a920 msgr2=0x7f78400cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:42.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 shutdown_connections 2023-11-24T08:09:42.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.540+0000 7f784780e640 1 -- 172.21.15.106:0/638223477 wait complete. 2023-11-24T08:09:42.561 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:09:42.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:42.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.774+0000 7fd4cba5c640 1 Processor -- start 2023-11-24T08:09:42.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.774+0000 7fd4cba5c640 1 -- start start 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.775+0000 7fd4cba5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cf450 0x7fd4c40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.775+0000 7fd4cba5c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4c40cdfb0 0x7fd4c40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.775+0000 7fd4cba5c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4c40cece0 con 0x7fd4c415c7f0 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.775+0000 7fd4cba5c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4c4156010 con 0x7fd4c40cdfb0 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.775+0000 7fd4cba5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c406ce30 con 0x7fd4c40cf450 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4c40cdfb0 0x7fd4c40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4c40cdfb0 0x7fd4c40ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51896/0 (socket says 172.21.15.106:51896) 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/153406308 learned_addr learned my addr 172.21.15.106:0/153406308 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c97f8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd4c415c7f0 0x7fd4c406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50798/0 (socket says 172.21.15.106:50798) 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c8ff7640 1 --2- 172.21.15.106:0/153406308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cf450 0x7fd4c40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/153406308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cf450 msgr2=0x7fd4c40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:42.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 --2- 172.21.15.106:0/153406308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cf450 0x7fd4c40cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/153406308 >> v1:172.21.15.106:6789/0 conn(0x7fd4c415c7f0 legacy=0x7fd4c406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.776+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/153406308 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c406d040 con 0x7fd4c40cdfb0 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.777+0000 7fd4c9ff9640 1 --2- 172.21.15.106:0/153406308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c40cdfb0 0x7fd4c40ce440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009e50 tx=0x7fd4bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59295d39f0ad101 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4bbfff640 1 -- 172.21.15.106:0/153406308 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc02cb00 con 0x7fd4c40cdfb0 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4bbfff640 1 -- 172.21.15.106:0/153406308 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc02cc60 con 0x7fd4c40cdfb0 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4bbfff640 1 -- 172.21.15.106:0/153406308 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc0343f0 con 0x7fd4c40cdfb0 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 -- 172.21.15.106:0/153406308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c40cdfb0 msgr2=0x7fd4c40ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/153406308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c40cdfb0 0x7fd4c40ce440 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009e50 tx=0x7fd4bc02bf60 comp rx=0 tx=0).stop 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 -- 172.21.15.106:0/153406308 shutdown_connections 2023-11-24T08:09:42.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/153406308 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c40cdfb0 0x7fd4c40ce440 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/153406308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cf450 0x7fd4c40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 -- 172.21.15.106:0/153406308 >> 172.21.15.106:0/153406308 conn(0x7fd4c406a920 msgr2=0x7fd4c406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:42.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.778+0000 7fd4cba5c640 1 -- 172.21.15.106:0/153406308 shutdown_connections 2023-11-24T08:09:42.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.779+0000 7fd4cba5c640 1 -- 172.21.15.106:0/153406308 wait complete. 2023-11-24T08:09:42.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.780+0000 7fd4cba5c640 1 Processor -- start 2023-11-24T08:09:42.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.780+0000 7fd4cba5c640 1 -- start start 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.781+0000 7fd4cba5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cdfb0 0x7fd4c41f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.781+0000 7fd4cba5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c40cf450 0x7fd4c40c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.781+0000 7fd4cba5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 0x7fd4c415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.781+0000 7fd4cba5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40ad8e0 con 0x7fd4c415c7f0 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.781+0000 7fd4cba5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c41f33b0 con 0x7fd4c40cf450 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.781+0000 7fd4cba5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c40ac4c0 con 0x7fd4c40cdfb0 2023-11-24T08:09:42.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 0x7fd4c415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 0x7fd4c415a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51912/0 (socket says 172.21.15.106:51912) 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/583857553 learned_addr learned my addr 172.21.15.106:0/583857553 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c97f8640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cdfb0 0x7fd4c41f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c8ff7640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c40cf450 0x7fd4c40c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cdfb0 msgr2=0x7fd4c41f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cdfb0 0x7fd4c41f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c40cf450 msgr2=0x7fd4c40c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c40cf450 0x7fd4c40c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:42.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 -- 172.21.15.106:0/583857553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc009b30 con 0x7fd4c415c7f0 2023-11-24T08:09:42.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c97f8640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cdfb0 0x7fd4c41f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:42.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.782+0000 7fd4c9ff9640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 0x7fd4c415a990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009e50 tx=0x7fd4bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.783+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc009510 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.783+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4bc03e450 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.783+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4bc03e800 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.783+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c40ac7a0 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.783+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c40acd60 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.784+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4c41f3520 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.785+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fd4bc03e960 con 0x7fd4c415c7f0 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.786+0000 7fd4b9ffb640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4a806ae70 0x7fd4a806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:42.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.786+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd4bc0ac2e0 con 0x7fd4c415c7f0 2023-11-24T08:09:42.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.786+0000 7fd4c97f8640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4a806ae70 0x7fd4a806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:42.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.787+0000 7fd4c97f8640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4a806ae70 0x7fd4a806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000f60 tx=0x7fd4c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:42.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:42.790+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4bc077370 con 0x7fd4c415c7f0 2023-11-24T08:09:43.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.017+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4c4067470 con 0x7fd4c415c7f0 2023-11-24T08:09:43.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.018+0000 7fd4b9ffb640 1 -- 172.21.15.106:0/583857553 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7fd4bc07a990 con 0x7fd4c415c7f0 2023-11-24T08:09:43.021 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:43.021 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:43.021 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:43.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4a806ae70 msgr2=0x7fd4a806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4a806ae70 0x7fd4a806d330 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0000f60 tx=0x7fd4c0009040 comp rx=0 tx=0).stop 2023-11-24T08:09:43.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 msgr2=0x7fd4c415a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 0x7fd4c415a990 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009e50 tx=0x7fd4bc004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:43.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 shutdown_connections 2023-11-24T08:09:43.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4c415c7f0 0x7fd4c415a990 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c40cf450 0x7fd4c40c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4a806ae70 0x7fd4a806d330 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 --2- 172.21.15.106:0/583857553 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4c40cdfb0 0x7fd4c41f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.021+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 >> 172.21.15.106:0/583857553 conn(0x7fd4c406a920 msgr2=0x7fd4c415cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:43.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.022+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 shutdown_connections 2023-11-24T08:09:43.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.022+0000 7fd4cba5c640 1 -- 172.21.15.106:0/583857553 wait complete. 2023-11-24T08:09:43.043 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:09:43.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.251+0000 7f8d712da640 1 Processor -- start 2023-11-24T08:09:43.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.252+0000 7f8d712da640 1 -- start start 2023-11-24T08:09:43.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.253+0000 7f8d712da640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8d6c0b2250 0x7f8d6c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.253+0000 7f8d712da640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8d6c0b2aa0 0x7f8d6c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.253+0000 7f8d712da640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d6c15ab80 con 0x7f8d6c0b2630 2023-11-24T08:09:43.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.253+0000 7f8d712da640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c15a990 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.253+0000 7f8d712da640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d6c15e680 con 0x7f8d6c0b2250 2023-11-24T08:09:43.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.253+0000 7f8d6b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8d6c0b2aa0 0x7f8d6c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8d6c0b2630 0x7f8d6c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50804/0 (socket says 172.21.15.106:50804) 2023-11-24T08:09:43.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6a7fc640 1 -- 172.21.15.106:0/1163690789 learned_addr learned my addr 172.21.15.106:0/1163690789 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:43.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6affd640 1 --2- 172.21.15.106:0/1163690789 >> v2:172.21.15.156:3300/0 conn(0x7f8d6c0b2250 0x7f8d6c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/1163690789 >> v2:172.21.15.156:3300/0 conn(0x7f8d6c0b2250 msgr2=0x7f8d6c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:43.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6b7fe640 1 --2- 172.21.15.106:0/1163690789 >> v2:172.21.15.156:3300/0 conn(0x7f8d6c0b2250 0x7f8d6c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/1163690789 >> v1:172.21.15.106:6789/0 conn(0x7f8d6c0b2630 legacy=0x7f8d6c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/1163690789 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d6c15e960 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.254+0000 7f8d6affd640 1 --2- 172.21.15.106:0/1163690789 >> v2:172.21.15.156:3300/0 conn(0x7f8d6c0b2250 0x7f8d6c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:43.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.255+0000 7f8d6b7fe640 1 --2- 172.21.15.106:0/1163690789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c15b080 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c072a70 tx=0x7f8d6402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9563b7e3362bff6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:43.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.255+0000 7f8d69ffb640 1 -- 172.21.15.106:0/1163690789 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64031830 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.255+0000 7f8d69ffb640 1 -- 172.21.15.106:0/1163690789 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d64031990 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.255+0000 7f8d69ffb640 1 -- 172.21.15.106:0/1163690789 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d64031d20 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 -- 172.21.15.106:0/1163690789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 msgr2=0x7f8d6c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 --2- 172.21.15.106:0/1163690789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c15b080 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c072a70 tx=0x7f8d6402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 -- 172.21.15.106:0/1163690789 shutdown_connections 2023-11-24T08:09:43.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 --2- 172.21.15.106:0/1163690789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c15b080 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 --2- 172.21.15.106:0/1163690789 >> v2:172.21.15.156:3300/0 conn(0x7f8d6c0b2250 0x7f8d6c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 -- 172.21.15.106:0/1163690789 >> 172.21.15.106:0/1163690789 conn(0x7f8d6c06a920 msgr2=0x7f8d6c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:43.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 -- 172.21.15.106:0/1163690789 shutdown_connections 2023-11-24T08:09:43.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.256+0000 7f8d712da640 1 -- 172.21.15.106:0/1163690789 wait complete. 2023-11-24T08:09:43.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.258+0000 7f8d712da640 1 Processor -- start 2023-11-24T08:09:43.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.258+0000 7f8d712da640 1 -- start start 2023-11-24T08:09:43.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.259+0000 7f8d712da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d6c0b2250 0x7f8d6c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.259+0000 7f8d712da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d6c0b2630 0x7f8d6c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.259+0000 7f8d712da640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.259+0000 7f8d712da640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0b10a0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.259+0000 7f8d712da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0b0ae0 con 0x7f8d6c0b2250 2023-11-24T08:09:43.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.259+0000 7f8d712da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d6c0b0dc0 con 0x7f8d6c0b2630 2023-11-24T08:09:43.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c157660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51928/0 (socket says 172.21.15.106:51928) 2023-11-24T08:09:43.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/480301037 learned_addr learned my addr 172.21.15.106:0/480301037 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:43.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6affd640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d6c0b2250 0x7f8d6c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6a7fc640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d6c0b2630 0x7f8d6c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d6c0b2630 msgr2=0x7f8d6c157120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d6c0b2630 0x7f8d6c157120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d6c0b2250 msgr2=0x7f8d6c0cbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d6c0b2250 0x7f8d6c0cbce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 -- 172.21.15.106:0/480301037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d64009400 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6affd640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d6c0b2250 0x7f8d6c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:43.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6a7fc640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d6c0b2630 0x7f8d6c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:43.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.260+0000 7f8d6b7fe640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c157660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d64034040 tx=0x7f8d640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:43.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.261+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d640317d0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.261+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d6403a9e0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.261+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d6403ad90 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.261+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d6c0cd7c0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.261+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d6c0c86e0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.262+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d6c067470 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.263+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8d640443f0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.264+0000 7f8d637fe640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8d4c06ae70 0x7f8d4c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.264+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f8d640ab970 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.264+0000 7f8d6affd640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8d4c06ae70 0x7f8d4c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.265+0000 7f8d6affd640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8d4c06ae70 0x7f8d4c06d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00d430 tx=0x7f8d5c007530 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:43.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.268+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8d64076a10 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.501+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d6c0ae0e0 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.502+0000 7f8d637fe640 1 -- 172.21.15.106:0/480301037 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4916 (secure 0 0 0) 0x7f8d64035020 con 0x7f8d6c0b2aa0 2023-11-24T08:09:43.505 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:43.505 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/4239086385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4239086385},{"type":"v1","addr":"172.21.15.106:6837","nonce":4239086385}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3262429992","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3262429992},{"type":"v1","addr":"172.21.15.106:6836","nonce":3262429992}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/756255921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":756255921},{"type":"v1","addr":"172.21.15.156:6833","nonce":756255921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:08:47.586538+0000","modified":"2023-11-24T08:08:50.630789+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4276},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4276":{"gid":4276,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.156:6835/4220970412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":4220970412},{"type":"v1","addr":"172.21.15.156:6835","nonce":4220970412}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-24T08:09:43.505 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 7 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8d4c06ae70 msgr2=0x7f8d4c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8d4c06ae70 0x7f8d4c06d330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8d5c00d430 tx=0x7f8d5c007530 comp rx=0 tx=0).stop 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 msgr2=0x7f8d6c157660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c157660 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f8d64034040 tx=0x7f8d640046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 shutdown_connections 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8d6c0b2aa0 0x7f8d6c157660 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8d4c06ae70 0x7f8d4c06d330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8d6c0b2630 0x7f8d6c157120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 --2- 172.21.15.106:0/480301037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d6c0b2250 0x7f8d6c0cbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 >> 172.21.15.106:0/480301037 conn(0x7f8d6c06a920 msgr2=0x7f8d6c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:43.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 shutdown_connections 2023-11-24T08:09:43.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.505+0000 7f8d712da640 1 -- 172.21.15.106:0/480301037 wait complete. 2023-11-24T08:09:43.527 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:43.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.733+0000 7f9cc7933640 1 Processor -- start 2023-11-24T08:09:43.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.733+0000 7f9cc7933640 1 -- start start 2023-11-24T08:09:43.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc7933640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc7933640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cc00cf450 0x7f9cc00cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc7933640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc00cece0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc7933640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc0156010 con 0x7f9cc00cf450 2023-11-24T08:09:43.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc7933640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc006ce30 con 0x7f9cc015c7f0 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc4ece640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cc00cf450 0x7f9cc00cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc4ece640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9cc00cf450 0x7f9cc00cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51944/0 (socket says 172.21.15.106:51944) 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc4ece640 1 -- 172.21.15.106:0/3881650602 learned_addr learned my addr 172.21.15.106:0/3881650602 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.735+0000 7f9cc5ed0640 1 --1- 172.21.15.106:0/3881650602 >> v1:172.21.15.106:6789/0 conn(0x7f9cc00cdfb0 0x7f9cc00ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50806/0 (socket says 172.21.15.106:50806) 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.736+0000 7f9cc56cf640 1 --2- 172.21.15.106:0/3881650602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.736+0000 7f9cc4ece640 1 -- 172.21.15.106:0/3881650602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 msgr2=0x7f9cc006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.736+0000 7f9cc4ece640 1 --2- 172.21.15.106:0/3881650602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.736+0000 7f9cc4ece640 1 -- 172.21.15.106:0/3881650602 >> v1:172.21.15.106:6789/0 conn(0x7f9cc00cdfb0 legacy=0x7f9cc00ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.736+0000 7f9cc4ece640 1 -- 172.21.15.106:0/3881650602 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc006d040 con 0x7f9cc00cf450 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.737+0000 7f9cc4ece640 1 --2- 172.21.15.106:0/3881650602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cf450 0x7f9cc00cf8d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009870 tx=0x7f9cb002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6360f56e579fe9fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.737+0000 7f9cb7fff640 1 -- 172.21.15.106:0/3881650602 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0030840 con 0x7f9cc00cf450 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.737+0000 7f9cb7fff640 1 -- 172.21.15.106:0/3881650602 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb00309a0 con 0x7f9cc00cf450 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.737+0000 7f9cb7fff640 1 -- 172.21.15.106:0/3881650602 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0030d30 con 0x7f9cc00cf450 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 -- 172.21.15.106:0/3881650602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cf450 msgr2=0x7f9cc00cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 --2- 172.21.15.106:0/3881650602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cf450 0x7f9cc00cf8d0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0009870 tx=0x7f9cb002b980 comp rx=0 tx=0).stop 2023-11-24T08:09:43.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 -- 172.21.15.106:0/3881650602 shutdown_connections 2023-11-24T08:09:43.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 --2- 172.21.15.106:0/3881650602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cf450 0x7f9cc00cf8d0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 --2- 172.21.15.106:0/3881650602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 -- 172.21.15.106:0/3881650602 >> 172.21.15.106:0/3881650602 conn(0x7f9cc006a920 msgr2=0x7f9cc006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:43.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 -- 172.21.15.106:0/3881650602 shutdown_connections 2023-11-24T08:09:43.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.738+0000 7f9cc7933640 1 -- 172.21.15.106:0/3881650602 wait complete. 2023-11-24T08:09:43.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.740+0000 7f9cc7933640 1 Processor -- start 2023-11-24T08:09:43.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.740+0000 7f9cc7933640 1 -- start start 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc7933640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 0x7f9cc014b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc7933640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cf450 0x7f9cc0157830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc7933640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc7933640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0156150 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc7933640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc01582d0 con 0x7f9cc00cf450 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc7933640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0156410 con 0x7f9cc015c7f0 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc56cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 0x7f9cc014b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc56cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 0x7f9cc014b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:51950/0 (socket says 172.21.15.106:51950) 2023-11-24T08:09:43.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.741+0000 7f9cc56cf640 1 -- 172.21.15.106:0/1793825343 learned_addr learned my addr 172.21.15.106:0/1793825343 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:43.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc56cf640 1 -- 172.21.15.106:0/1793825343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 msgr2=0x7f9cc015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc56cf640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc56cf640 1 -- 172.21.15.106:0/1793825343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cf450 msgr2=0x7f9cc0157830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc56cf640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cf450 0x7f9cc0157830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc56cf640 1 -- 172.21.15.106:0/1793825343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00094d0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc56cf640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 0x7f9cc014b500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00d5f0 tx=0x7f9cbc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc0125f0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cbc012750 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.742+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc01566f0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.743+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cbc012ae0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.743+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01f23a0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.744+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc0067470 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.745+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9cbc012c40 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.745+0000 7f9cb5ffb640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9ca406ae40 0x7f9ca406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:43.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.746+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9cbc089b90 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.746+0000 7f9cc4ece640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9ca406ae40 0x7f9ca406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:43.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.747+0000 7f9cc4ece640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9ca406ae40 0x7f9ca406d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0036000 tx=0x7f9cb003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:43.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.752+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9cbc054ba0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.942+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cc00cd730 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.944+0000 7f9cb5ffb640 1 -- 172.21.15.106:0/1793825343 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+16793 (secure 0 0 0) 0x7f9cbc0581c0 con 0x7f9cc00cdfb0 2023-11-24T08:09:43.947 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:43.947 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":24,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:35.282948+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:08:45.474425+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-11-24T08:08:46.647137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.2","mappings":[{"from":3,"to":2}]},{"pgid":"3.1d","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3b","mappings":[{"from":7,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9ca406ae40 msgr2=0x7f9ca406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9ca406ae40 0x7f9ca406d300 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0036000 tx=0x7f9cb003b040 comp rx=0 tx=0).stop 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 msgr2=0x7f9cc014b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 0x7f9cc014b500 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc00d5f0 tx=0x7f9cbc00dab0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 shutdown_connections 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9ca406ae40 0x7f9ca406d300 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc015c7f0 0x7f9cc015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cc00cf450 0x7f9cc0157830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 --2- 172.21.15.106:0/1793825343 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc00cdfb0 0x7f9cc014b500 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 >> 172.21.15.106:0/1793825343 conn(0x7f9cc006a920 msgr2=0x7f9cc00cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 shutdown_connections 2023-11-24T08:09:43.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:43.947+0000 7f9cc7933640 1 -- 172.21.15.106:0/1793825343 wait complete. 2023-11-24T08:09:43.970 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:09:44.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.175+0000 7f7c8876f640 1 Processor -- start 2023-11-24T08:09:44.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.175+0000 7f7c8876f640 1 -- start start 2023-11-24T08:09:44.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.176+0000 7f7c8876f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c8015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.176+0000 7f7c8876f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c8015ec90 con 0x7f7c8006de10 2023-11-24T08:09:44.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.176+0000 7f7c8876f640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800cd530 con 0x7f7c80072160 2023-11-24T08:09:44.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.177+0000 7f7c8876f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800c4ed0 con 0x7f7c800c49f0 2023-11-24T08:09:44.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.177+0000 7f7c85d0a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7c8006de10 0x7f7c800705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35410/0 (socket says 172.21.15.106:35410) 2023-11-24T08:09:44.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.177+0000 7f7c85d0a640 1 -- 172.21.15.106:0/42987248 learned_addr learned my addr 172.21.15.106:0/42987248 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:44.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.177+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730094855 0 0) 0x7f7c70001110 con 0x7f7c8006de10 2023-11-24T08:09:44.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.177+0000 7f7c86d0c640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c8015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.177+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c005730 con 0x7f7c8006de10 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622575676 0 0) 0x7f7c6c005730 con 0x7f7c8006de10 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 msgr2=0x7f7c8015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c8015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 >> v1:172.21.15.156:6789/0 conn(0x7f7c80072160 legacy=0x7f7c8006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800c51b0 con 0x7f7c8006de10 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f7c70001800 con 0x7f7c8006de10 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 >> v1:172.21.15.106:6789/0 conn(0x7f7c8006de10 legacy=0x7f7c800705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6c005bb0 0x7f7c6c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0069d0 0x7f7c6c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c6c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800c4ed0 con 0x7f7c6c0069d0 2023-11-24T08:09:44.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c00dc40 con 0x7f7c6c005bb0 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c00de00 con 0x7f7c800c49f0 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85d0a640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0069d0 0x7f7c6c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c8650b640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6c005bb0 0x7f7c6c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c86d0c640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c6c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85d0a640 1 -- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 msgr2=0x7f7c6c007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.178+0000 7f7c85d0a640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c6c007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.179+0000 7f7c85d0a640 1 -- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6c005bb0 msgr2=0x7f7c6c005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.179+0000 7f7c85d0a640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c6c005bb0 0x7f7c6c005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.179+0000 7f7c85d0a640 1 -- 172.21.15.106:0/42987248 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c005730 con 0x7f7c6c0069d0 2023-11-24T08:09:44.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.179+0000 7f7c86d0c640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c800c49f0 0x7f7c6c007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c85d0a640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0069d0 0x7f7c6c006e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c160 tx=0x7f7c6c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d171c15bc0ec672 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70005ad0 con 0x7f7c6c0069d0 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70005c50 con 0x7f7c6c0069d0 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c85509640 1 -- 172.21.15.106:0/42987248 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7002c790 con 0x7f7c6c0069d0 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c8876f640 1 -- 172.21.15.106:0/42987248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0069d0 msgr2=0x7f7c6c006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c8876f640 1 --2- 172.21.15.106:0/42987248 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c6c0069d0 0x7f7c6c006e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c160 tx=0x7f7c6c007a00 comp rx=0 tx=0).stop 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c86d0c640 1 -- 172.21.15.106:0/42987248 reap_dead start 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c8876f640 1 -- 172.21.15.106:0/42987248 shutdown_connections 2023-11-24T08:09:44.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.180+0000 7f7c8876f640 1 -- 172.21.15.106:0/42987248 >> 172.21.15.106:0/42987248 conn(0x7f7c800b0c50 msgr2=0x7f7c800b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:44.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.181+0000 7f7c8876f640 1 -- 172.21.15.106:0/42987248 shutdown_connections 2023-11-24T08:09:44.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.181+0000 7f7c8876f640 1 -- 172.21.15.106:0/42987248 wait complete. 2023-11-24T08:09:44.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.182+0000 7f7c8876f640 1 Processor -- start 2023-11-24T08:09:44.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.182+0000 7f7c8876f640 1 -- start start 2023-11-24T08:09:44.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.183+0000 7f7c8876f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c6c0069d0 0x7f7c801e3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.183+0000 7f7c8876f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c801e51c0 0x7f7c80144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.183+0000 7f7c8876f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 0x7f7c800aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.183+0000 7f7c8876f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c80144f90 con 0x7f7c800cbc10 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.183+0000 7f7c8876f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801e4120 con 0x7f7c801e51c0 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.183+0000 7f7c8876f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801e4400 con 0x7f7c6c0069d0 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 0x7f7c800aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 0x7f7c800aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32856/0 (socket says 172.21.15.106:32856) 2023-11-24T08:09:44.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 -- 172.21.15.106:0/3333858012 learned_addr learned my addr 172.21.15.106:0/3333858012 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c8650b640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c6c0069d0 0x7f7c801e3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c85d0a640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c801e51c0 0x7f7c80144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 -- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c6c0069d0 msgr2=0x7f7c801e3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c6c0069d0 0x7f7c801e3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 -- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c801e51c0 msgr2=0x7f7c80144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c801e51c0 0x7f7c80144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 -- 172.21.15.106:0/3333858012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c801e46e0 con 0x7f7c800cbc10 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.184+0000 7f7c86d0c640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 0x7f7c800aeca0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c800c4e00 tx=0x7f7c7800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.185+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78017400 con 0x7f7c800cbc10 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.185+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78011040 con 0x7f7c800cbc10 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.185+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c780177c0 con 0x7f7c800cbc10 2023-11-24T08:09:44.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.185+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c801e49c0 con 0x7f7c800cbc10 2023-11-24T08:09:44.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.185+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c800cc090 con 0x7f7c800cbc10 2023-11-24T08:09:44.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.186+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c8004e8b0 con 0x7f7c800cbc10 2023-11-24T08:09:44.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.187+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7c780203b0 con 0x7f7c800cbc10 2023-11-24T08:09:44.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.188+0000 7f7c76ffd640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c6406ae70 0x7f7c6406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.188+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7c780899d0 con 0x7f7c800cbc10 2023-11-24T08:09:44.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.188+0000 7f7c8650b640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c6406ae70 0x7f7c6406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.189+0000 7f7c8650b640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c6406ae70 0x7f7c6406d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c005490 tx=0x7f7c7c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:44.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.193+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c78054950 con 0x7f7c800cbc10 2023-11-24T08:09:44.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.420+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7c8015dab0 con 0x7f7c800cbc10 2023-11-24T08:09:44.423 INFO:tasks.ceph.mon.a.smithi106.stderr:2023-11-24T08:09:44.421+0000 7fb1127dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:09:44.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.425+0000 7f7c76ffd640 1 -- 172.21.15.106:0/3333858012 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v8) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7c78057f70 con 0x7f7c800cbc10 2023-11-24T08:09:44.427 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:09:44.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c6406ae70 msgr2=0x7f7c6406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c6406ae70 0x7f7c6406d330 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c005490 tx=0x7f7c7c009040 comp rx=0 tx=0).stop 2023-11-24T08:09:44.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 msgr2=0x7f7c800aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 0x7f7c800aeca0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c800c4e00 tx=0x7f7c7800f4a0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 shutdown_connections 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c800cbc10 0x7f7c800aeca0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c801e51c0 0x7f7c80144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c6406ae70 0x7f7c6406d330 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 --2- 172.21.15.106:0/3333858012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c6c0069d0 0x7f7c801e3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.427+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 >> 172.21.15.106:0/3333858012 conn(0x7f7c800b0c50 msgr2=0x7f7c800b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.428+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 shutdown_connections 2023-11-24T08:09:44.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.428+0000 7f7c8876f640 1 -- 172.21.15.106:0/3333858012 wait complete. 2023-11-24T08:09:44.450 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:09:44.470 INFO:tasks.ceph.mds.d.smithi156.stderr: -7416> 2023-11-24T08:08:51.531+0000 7f3d5eb4f640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:09:44.560 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:09:44.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.662+0000 7f6e9c960640 1 Processor -- start 2023-11-24T08:09:44.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.663+0000 7f6e9c960640 1 -- start start 2023-11-24T08:09:44.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9c960640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e980d1a00 0x7f6e980b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9c960640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e98160dd0 0x7f6e980b3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9c960640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e980b41b0 con 0x7f6e981609f0 2023-11-24T08:09:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9c960640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e980b4490 con 0x7f6e980d1a00 2023-11-24T08:09:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9c960640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980b0090 con 0x7f6e98160dd0 2023-11-24T08:09:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e980d1a00 0x7f6e980b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9659c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e980d1a00 0x7f6e980b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32866/0 (socket says 172.21.15.106:32866) 2023-11-24T08:09:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e9659c640 1 -- 172.21.15.106:0/1193487178 learned_addr learned my addr 172.21.15.106:0/1193487178 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.664+0000 7f6e96d9d640 1 --1- 172.21.15.106:0/1193487178 >> v1:172.21.15.106:6789/0 conn(0x7f6e981609f0 0x7f6e980d12f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35416/0 (socket says 172.21.15.106:35416) 2023-11-24T08:09:44.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e9659c640 1 -- 172.21.15.106:0/1193487178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e98160dd0 msgr2=0x7f6e980b3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e9759e640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e98160dd0 0x7f6e980b3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e9659c640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e98160dd0 0x7f6e980b3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e9659c640 1 -- 172.21.15.106:0/1193487178 >> v1:172.21.15.106:6789/0 conn(0x7f6e981609f0 legacy=0x7f6e980d12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e95d9b640 1 -- 172.21.15.106:0/1193487178 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967943393 0 0) 0x7f6e980b41b0 con 0x7f6e981609f0 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e9659c640 1 -- 172.21.15.106:0/1193487178 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980b0370 con 0x7f6e980d1a00 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.665+0000 7f6e9759e640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e98160dd0 0x7f6e980b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.666+0000 7f6e9659c640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980b2be0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009870 tx=0x7f6e8002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6978a6ab3a640f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.666+0000 7f6e95d9b640 1 -- 172.21.15.106:0/1193487178 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80030840 con 0x7f6e980d1a00 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e95d9b640 1 -- 172.21.15.106:0/1193487178 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e800309a0 con 0x7f6e980d1a00 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e95d9b640 1 -- 172.21.15.106:0/1193487178 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80030d30 con 0x7f6e980d1a00 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 -- 172.21.15.106:0/1193487178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 msgr2=0x7f6e980b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980b2be0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009870 tx=0x7f6e8002b980 comp rx=0 tx=0).stop 2023-11-24T08:09:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 -- 172.21.15.106:0/1193487178 shutdown_connections 2023-11-24T08:09:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e98160dd0 0x7f6e980b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 --2- 172.21.15.106:0/1193487178 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980b2be0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 -- 172.21.15.106:0/1193487178 >> 172.21.15.106:0/1193487178 conn(0x7f6e9809c5b0 msgr2=0x7f6e980d3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 -- 172.21.15.106:0/1193487178 shutdown_connections 2023-11-24T08:09:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.667+0000 7f6e9c960640 1 -- 172.21.15.106:0/1193487178 wait complete. 2023-11-24T08:09:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.669+0000 7f6e9c960640 1 Processor -- start 2023-11-24T08:09:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.669+0000 7f6e9c960640 1 -- start start 2023-11-24T08:09:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.670+0000 7f6e9c960640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.670+0000 7f6e9c960640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e981609f0 0x7f6e980ccde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.670+0000 7f6e9c960640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e98160dd0 0x7f6e980ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.670+0000 7f6e9c960640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980c6560 con 0x7f6e980d1a00 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.670+0000 7f6e9c960640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e980cd320 con 0x7f6e98160dd0 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.670+0000 7f6e9c960640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9815c110 con 0x7f6e981609f0 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32878/0 (socket says 172.21.15.106:32878) 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 -- 172.21.15.106:0/765948220 learned_addr learned my addr 172.21.15.106:0/765948220 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e9659c640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e981609f0 0x7f6e980ccde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e9759e640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e98160dd0 0x7f6e980ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 -- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e981609f0 msgr2=0x7f6e980ccde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e981609f0 0x7f6e980ccde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 -- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e98160dd0 msgr2=0x7f6e980ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e98160dd0 0x7f6e980ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e96d9d640 1 -- 172.21.15.106:0/765948220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e800094d0 con 0x7f6e980d1a00 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.671+0000 7f6e9659c640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e981609f0 0x7f6e980ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.672+0000 7f6e96d9d640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980c94a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c890 tx=0x7f6e8800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.672+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800a640 con 0x7f6e980d1a00 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.672+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8800a7c0 con 0x7f6e980d1a00 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.672+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8800ab70 con 0x7f6e980d1a00 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.672+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9815c3f0 con 0x7f6e980d1a00 2023-11-24T08:09:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.672+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9815c9b0 con 0x7f6e980d1a00 2023-11-24T08:09:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.673+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9804e840 con 0x7f6e980d1a00 2023-11-24T08:09:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.674+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6e8800acd0 con 0x7f6e980d1a00 2023-11-24T08:09:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.676+0000 7f6e8f7fe640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e7806ae40 0x7f6e7806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.676+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f6e88088cc0 con 0x7f6e980d1a00 2023-11-24T08:09:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.676+0000 7f6e9659c640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e7806ae40 0x7f6e7806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.677+0000 7f6e9659c640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e7806ae40 0x7f6e7806d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e80036000 tx=0x7f6e80008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:44.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.680+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e8808f080 con 0x7f6e980d1a00 2023-11-24T08:09:44.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:44.912+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6e980cd510 con 0x7f6e980d1a00 2023-11-24T08:09:45.373 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:45.382 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:09:45.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.496+0000 7f6e8f7fe640 1 -- 172.21.15.106:0/765948220 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6e880180a0 con 0x7f6e980d1a00 2023-11-24T08:09:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e7806ae40 msgr2=0x7f6e7806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:45.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e7806ae40 0x7f6e7806d300 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e80036000 tx=0x7f6e80008030 comp rx=0 tx=0).stop 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 msgr2=0x7f6e980c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980c94a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6e8800c890 tx=0x7f6e8800cd50 comp rx=0 tx=0).stop 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 shutdown_connections 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e98160dd0 0x7f6e980ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e7806ae40 0x7f6e7806d300 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e981609f0 0x7f6e980ccde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 --2- 172.21.15.106:0/765948220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e980d1a00 0x7f6e980c94a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.499+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 >> 172.21.15.106:0/765948220 conn(0x7f6e9809c5b0 msgr2=0x7f6e9806c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.500+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 shutdown_connections 2023-11-24T08:09:45.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.500+0000 7f6e9c960640 1 -- 172.21.15.106:0/765948220 wait complete. 2023-11-24T08:09:45.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:09:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.736+0000 7f4b4d615640 1 Processor -- start 2023-11-24T08:09:45.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.736+0000 7f4b4d615640 1 -- start start 2023-11-24T08:09:45.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b4d615640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2250 0x7f4b4815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:45.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b4d615640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b480b2a70 0x7f4b4815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:45.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b4d615640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b4815ab80 con 0x7f4b480b2630 2023-11-24T08:09:45.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b4d615640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b4815a990 con 0x7f4b480b2a70 2023-11-24T08:09:45.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b4d615640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b4815e680 con 0x7f4b480b2250 2023-11-24T08:09:45.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b480b2a70 0x7f4b4815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:45.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4b480b2a70 0x7f4b4815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32888/0 (socket says 172.21.15.106:32888) 2023-11-24T08:09:45.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 -- 172.21.15.106:0/2668422117 learned_addr learned my addr 172.21.15.106:0/2668422117 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:45.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b46ffd640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2250 0x7f4b4815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:45.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 -- 172.21.15.106:0/2668422117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2250 msgr2=0x7f4b4815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:45.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2250 0x7f4b4815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 -- 172.21.15.106:0/2668422117 >> v1:172.21.15.156:6789/0 conn(0x7f4b480b2630 legacy=0x7f4b480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:45.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b477fe640 1 -- 172.21.15.106:0/2668422117 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4815e960 con 0x7f4b480b2a70 2023-11-24T08:09:45.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.738+0000 7f4b46ffd640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2250 0x7f4b4815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:45.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.740+0000 7f4b477fe640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815b080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000d150 tx=0x7f4b4002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=425020412babe068 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:45.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.740+0000 7f4b45ffb640 1 -- 172.21.15.106:0/2668422117 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b4002ea80 con 0x7f4b480b2a70 2023-11-24T08:09:45.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.740+0000 7f4b45ffb640 1 -- 172.21.15.106:0/2668422117 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b4002ebe0 con 0x7f4b480b2a70 2023-11-24T08:09:45.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.740+0000 7f4b45ffb640 1 -- 172.21.15.106:0/2668422117 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40009600 con 0x7f4b480b2a70 2023-11-24T08:09:45.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 -- 172.21.15.106:0/2668422117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 msgr2=0x7f4b4815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:45.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815b080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000d150 tx=0x7f4b4002e0e0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 -- 172.21.15.106:0/2668422117 shutdown_connections 2023-11-24T08:09:45.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815b080 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 --2- 172.21.15.106:0/2668422117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2250 0x7f4b4815df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 -- 172.21.15.106:0/2668422117 >> 172.21.15.106:0/2668422117 conn(0x7f4b4806a920 msgr2=0x7f4b4806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:45.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 -- 172.21.15.106:0/2668422117 shutdown_connections 2023-11-24T08:09:45.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.741+0000 7f4b4d615640 1 -- 172.21.15.106:0/2668422117 wait complete. 2023-11-24T08:09:45.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.743+0000 7f4b4d615640 1 Processor -- start 2023-11-24T08:09:45.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.743+0000 7f4b4d615640 1 -- start start 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b4d615640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b480b2250 0x7f4b4814a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b4d615640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2630 0x7f4b4814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b4d615640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b4d615640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481500e0 con 0x7f4b480b2a70 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b4d615640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b481526a0 con 0x7f4b480b2250 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b4d615640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b48152980 con 0x7f4b480b2630 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815c9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32892/0 (socket says 172.21.15.106:32892) 2023-11-24T08:09:45.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 -- 172.21.15.106:0/4134594252 learned_addr learned my addr 172.21.15.106:0/4134594252 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b46ffd640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b480b2250 0x7f4b4814a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 -- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2630 msgr2=0x7f4b4814ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2630 0x7f4b4814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 -- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b480b2250 msgr2=0x7f4b4814a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b480b2250 0x7f4b4814a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.744+0000 7f4b477fe640 1 -- 172.21.15.106:0/4134594252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b4000bd40 con 0x7f4b480b2a70 2023-11-24T08:09:45.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.745+0000 7f4b477fe640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815c9a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000d150 tx=0x7f4b40004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:45.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.745+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40036800 con 0x7f4b480b2a70 2023-11-24T08:09:45.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.745+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b40036980 con 0x7f4b480b2a70 2023-11-24T08:09:45.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.745+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b40036d70 con 0x7f4b480b2a70 2023-11-24T08:09:45.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.745+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b480cd7c0 con 0x7f4b480b2a70 2023-11-24T08:09:45.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.745+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b48152f40 con 0x7f4b480b2a70 2023-11-24T08:09:45.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.746+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b48067470 con 0x7f4b480b2a70 2023-11-24T08:09:45.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.747+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4b4002ed50 con 0x7f4b480b2a70 2023-11-24T08:09:45.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.748+0000 7f4b2f7fe640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae90 0x7f4b3406d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:45.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.748+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f4b400ac870 con 0x7f4b480b2a70 2023-11-24T08:09:45.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.748+0000 7f4b46ffd640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae90 0x7f4b3406d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:45.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.749+0000 7f4b46ffd640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae90 0x7f4b3406d350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b38008850 tx=0x7f4b38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:45.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.752+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4b40077940 con 0x7f4b480b2a70 2023-11-24T08:09:45.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:45.946+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4b48067470 con 0x7f4b480b2a70 2023-11-24T08:09:46.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.500+0000 7f4b2f7fe640 1 -- 172.21.15.106:0/4134594252 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4b40035e80 con 0x7f4b480b2a70 2023-11-24T08:09:46.503 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:09:46.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.503+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae90 msgr2=0x7f4b3406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:46.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.504+0000 7f4b4d615640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae90 0x7f4b3406d350 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4b38008850 tx=0x7f4b38008040 comp rx=0 tx=0).stop 2023-11-24T08:09:46.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.504+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 msgr2=0x7f4b4815c9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:46.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.504+0000 7f4b4d615640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815c9a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f4b4000d150 tx=0x7f4b40004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:46.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.505+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 shutdown_connections 2023-11-24T08:09:46.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.505+0000 7f4b4d615640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4b480b2a70 0x7f4b4815c9a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.505+0000 7f4b4d615640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4b3406ae90 0x7f4b3406d350 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.505+0000 7f4b4d615640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4b480b2630 0x7f4b4814ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.505+0000 7f4b4d615640 1 --2- 172.21.15.106:0/4134594252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b480b2250 0x7f4b4814a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.505+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 >> 172.21.15.106:0/4134594252 conn(0x7f4b4806a920 msgr2=0x7f4b48159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:46.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.506+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 shutdown_connections 2023-11-24T08:09:46.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.507+0000 7f4b4d615640 1 -- 172.21.15.106:0/4134594252 wait complete. 2023-11-24T08:09:46.531 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:09:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.734+0000 7f4d09a2b640 1 Processor -- start 2023-11-24T08:09:46.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.734+0000 7f4d09a2b640 1 -- start start 2023-11-24T08:09:46.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.736+0000 7f4d09a2b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d040cf450 0x7f4d040cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.736+0000 7f4d09a2b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d040cdfb0 0x7f4d040ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.736+0000 7f4d09a2b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d040cece0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.736+0000 7f4d09a2b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d040cfe40 con 0x7f4d040cf450 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.736+0000 7f4d09a2b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d04156010 con 0x7f4d040cdfb0 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d040cf450 0x7f4d040cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d040cf450 0x7f4d040cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32902/0 (socket says 172.21.15.106:32902) 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 -- 172.21.15.106:0/3804021078 learned_addr learned my addr 172.21.15.106:0/3804021078 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4d02ffd640 1 --1- 172.21.15.106:0/3804021078 >> v1:172.21.15.106:6789/0 conn(0x7f4d0415c7f0 0x7f4d0406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35430/0 (socket says 172.21.15.106:35430) 2023-11-24T08:09:46.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 -- 172.21.15.106:0/3804021078 >> v2:172.21.15.156:3300/0 conn(0x7f4d040cdfb0 msgr2=0x7f4d040ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:46.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 --2- 172.21.15.106:0/3804021078 >> v2:172.21.15.156:3300/0 conn(0x7f4d040cdfb0 0x7f4d040ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 -- 172.21.15.106:0/3804021078 >> v1:172.21.15.106:6789/0 conn(0x7f4d0415c7f0 legacy=0x7f4d0406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:46.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.737+0000 7f4cfbfff640 1 -- 172.21.15.106:0/3804021078 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d0406d040 con 0x7f4d040cf450 2023-11-24T08:09:46.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.738+0000 7f4cfbfff640 1 --2- 172.21.15.106:0/3804021078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d040cf450 0x7f4d040cf900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d040d00e0 tx=0x7f4cec02b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=581e1d7361c7fedf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:46.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d027fc640 1 -- 172.21.15.106:0/3804021078 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec030870 con 0x7f4d040cf450 2023-11-24T08:09:46.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d027fc640 1 -- 172.21.15.106:0/3804021078 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cec0309d0 con 0x7f4d040cf450 2023-11-24T08:09:46.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d027fc640 1 -- 172.21.15.106:0/3804021078 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cec030d60 con 0x7f4d040cf450 2023-11-24T08:09:46.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 -- 172.21.15.106:0/3804021078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d040cf450 msgr2=0x7f4d040cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/3804021078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d040cf450 0x7f4d040cf900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f4d040d00e0 tx=0x7f4cec02b9b0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 -- 172.21.15.106:0/3804021078 shutdown_connections 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/3804021078 >> v2:172.21.15.156:3300/0 conn(0x7f4d040cdfb0 0x7f4d040ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/3804021078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d040cf450 0x7f4d040cf900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 -- 172.21.15.106:0/3804021078 >> 172.21.15.106:0/3804021078 conn(0x7f4d0406a920 msgr2=0x7f4d0406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 -- 172.21.15.106:0/3804021078 shutdown_connections 2023-11-24T08:09:46.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.739+0000 7f4d09a2b640 1 -- 172.21.15.106:0/3804021078 wait complete. 2023-11-24T08:09:46.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.741+0000 7f4d09a2b640 1 Processor -- start 2023-11-24T08:09:46.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.741+0000 7f4d09a2b640 1 -- start start 2023-11-24T08:09:46.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d09a2b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cdfb0 0x7f4d041538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:46.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d09a2b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d040cf450 0x7f4d040cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:46.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d09a2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 0x7f4d041540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d09a2b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d04156f60 con 0x7f4d0415c7f0 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d09a2b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041566c0 con 0x7f4d040cf450 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d09a2b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d041569a0 con 0x7f4d040cdfb0 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 0x7f4d041540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d037fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 0x7f4d041540f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32912/0 (socket says 172.21.15.106:32912) 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.742+0000 7f4d037fe640 1 -- 172.21.15.106:0/2125257988 learned_addr learned my addr 172.21.15.106:0/2125257988 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d037fe640 1 -- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cdfb0 msgr2=0x7f4d041538e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d02ffd640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cdfb0 0x7f4d041538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4cfbfff640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d040cf450 0x7f4d040cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:46.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d037fe640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cdfb0 0x7f4d041538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d037fe640 1 -- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d040cf450 msgr2=0x7f4d040cbac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d037fe640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d040cf450 0x7f4d040cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d037fe640 1 -- 172.21.15.106:0/2125257988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec0094d0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.743+0000 7f4d037fe640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 0x7f4d041540f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00b9e0 tx=0x7f4cfc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.744+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc00caa0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.744+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cfc00cc20 con 0x7f4d0415c7f0 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.744+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cfc014440 con 0x7f4d0415c7f0 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.744+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d04156c80 con 0x7f4d0415c7f0 2023-11-24T08:09:46.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.744+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0414baa0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.745+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d040b20e0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.746+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4cfc01d3b0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.747+0000 7f4d00ff9640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4ce406ae40 0x7f4ce406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.747+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f4cfc0893d0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.748+0000 7f4d02ffd640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4ce406ae40 0x7f4ce406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:46.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.748+0000 7f4d02ffd640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4ce406ae40 0x7f4ce406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:46.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.752+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d040b20e0 con 0x7f4d0415c7f0 2023-11-24T08:09:46.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:46.948+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4d040b24c0 con 0x7f4d0415c7f0 2023-11-24T08:09:47.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.510+0000 7f4d00ff9640 1 -- 172.21.15.106:0/2125257988 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4d040b24c0 con 0x7f4d0415c7f0 2023-11-24T08:09:47.514 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2023-11-24T08:09:47.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.513+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4ce406ae40 msgr2=0x7f4ce406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:47.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.513+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4ce406ae40 0x7f4ce406d300 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4cf4007fd0 tx=0x7f4cf4009040 comp rx=0 tx=0).stop 2023-11-24T08:09:47.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.513+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 msgr2=0x7f4d041540f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:47.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.513+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 0x7f4d041540f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc00b9e0 tx=0x7f4cfc00bea0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.515+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 shutdown_connections 2023-11-24T08:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.515+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d0415c7f0 0x7f4d041540f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.515+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d040cf450 0x7f4d040cbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.515+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4ce406ae40 0x7f4ce406d300 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.515+0000 7f4d09a2b640 1 --2- 172.21.15.106:0/2125257988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d040cdfb0 0x7f4d041538e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.515+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 >> 172.21.15.106:0/2125257988 conn(0x7f4d0406a920 msgr2=0x7f4d040aede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.516+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 shutdown_connections 2023-11-24T08:09:47.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.516+0000 7f4d09a2b640 1 -- 172.21.15.106:0/2125257988 wait complete. 2023-11-24T08:09:47.542 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:09:47.542 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:09:47.543 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:09:47.543 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:47.543 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:09:47.543 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:09:47.543 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:09:47.544 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:47.544 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:09:47.544 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:09:47.544 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:09:47.544 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:47.544 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:09:47.544 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:09:47.545 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:09:47.545 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:09:47.562 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:09:47.563 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:09:47.564 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:09:47.565 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:09:47.565 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:09:47.565 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:09:47.568 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:09:47.569 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:09:47.612 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:09:47.613 DEBUG:teuthology.parallel:result is None 2023-11-24T08:09:47.616 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:09:47.616 DEBUG:teuthology.parallel:result is None 2023-11-24T08:09:47.619 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:09:47.620 DEBUG:teuthology.parallel:result is None 2023-11-24T08:09:47.624 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:09:47.624 DEBUG:teuthology.parallel:result is None 2023-11-24T08:09:47.624 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:09:47.774 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-11-24T08:09:47.780 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:09:47.780 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:09:47.781 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:09:47.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.819+0000 7f97b9b89640 1 Processor -- start 2023-11-24T08:09:47.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.819+0000 7f97b9b89640 1 -- start start 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b9b89640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f97b40cfb90 0x7f97b4161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b9b89640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b4161fa0 0x7f97b406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b9b89640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b41612f0 con 0x7f97b40b2400 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b9b89640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b40cff70 con 0x7f97b40cfb90 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b9b89640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b40afb20 con 0x7f97b4161fa0 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f97b40b2400 0x7f97b40cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35432/0 (socket says 172.21.15.106:35432) 2023-11-24T08:09:47.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.820+0000 7f97b3fff640 1 -- 172.21.15.106:0/2753201965 learned_addr learned my addr 172.21.15.106:0/2753201965 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b37fe640 1 --2- 172.21.15.106:0/2753201965 >> v2:172.21.15.156:3300/0 conn(0x7f97b40cfb90 0x7f97b4161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97abfff640 1 --2- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b4161fa0 0x7f97b406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b2ffd640 1 -- 172.21.15.106:0/2753201965 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2947755343 0 0) 0x7f97b41612f0 con 0x7f97b40b2400 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b2ffd640 1 -- 172.21.15.106:0/2753201965 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a0003520 con 0x7f97b40b2400 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b37fe640 1 -- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b4161fa0 msgr2=0x7f97b406ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b37fe640 1 --2- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b4161fa0 0x7f97b406ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b37fe640 1 -- 172.21.15.106:0/2753201965 >> v1:172.21.15.106:6789/0 conn(0x7f97b40b2400 legacy=0x7f97b40cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:47.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.821+0000 7f97b37fe640 1 -- 172.21.15.106:0/2753201965 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b406a850 con 0x7f97b40cfb90 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.822+0000 7f97b37fe640 1 --2- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40cfb90 0x7f97b4161a60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97a40097d0 tx=0x7f97a402b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=265e1c623fbb04fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b2ffd640 1 -- 172.21.15.106:0/2753201965 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4031890 con 0x7f97b40cfb90 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b2ffd640 1 -- 172.21.15.106:0/2753201965 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a40319f0 con 0x7f97b40cfb90 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b2ffd640 1 -- 172.21.15.106:0/2753201965 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4031d80 con 0x7f97b40cfb90 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 -- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40cfb90 msgr2=0x7f97b4161a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 --2- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40cfb90 0x7f97b4161a60 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f97a40097d0 tx=0x7f97a402b930 comp rx=0 tx=0).stop 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 -- 172.21.15.106:0/2753201965 shutdown_connections 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 --2- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b4161fa0 0x7f97b406ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 --2- 172.21.15.106:0/2753201965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40cfb90 0x7f97b4161a60 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 -- 172.21.15.106:0/2753201965 >> 172.21.15.106:0/2753201965 conn(0x7f97b40999d0 msgr2=0x7f97b40b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:47.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 -- 172.21.15.106:0/2753201965 shutdown_connections 2023-11-24T08:09:47.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.823+0000 7f97b9b89640 1 -- 172.21.15.106:0/2753201965 wait complete. 2023-11-24T08:09:47.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.825+0000 7f97b9b89640 1 Processor -- start 2023-11-24T08:09:47.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.825+0000 7f97b9b89640 1 -- start start 2023-11-24T08:09:47.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b9b89640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40b2400 0x7f97b415de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b9b89640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b40cfb90 0x7f97b415e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b9b89640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 0x7f97b4159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b9b89640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b415eca0 con 0x7f97b40b2de0 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b9b89640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b415f200 con 0x7f97b40b2400 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b9b89640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b415ef20 con 0x7f97b40cfb90 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.826+0000 7f97b3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 0x7f97b4159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 0x7f97b4159500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32932/0 (socket says 172.21.15.106:32932) 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 -- 172.21.15.106:0/3264529602 learned_addr learned my addr 172.21.15.106:0/3264529602 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b37fe640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40b2400 0x7f97b415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97abfff640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b40cfb90 0x7f97b415e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 -- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b40cfb90 msgr2=0x7f97b415e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:47.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b40cfb90 0x7f97b415e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 -- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40b2400 msgr2=0x7f97b415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:47.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40b2400 0x7f97b415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:47.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 -- 172.21.15.106:0/3264529602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a4009400 con 0x7f97b40b2de0 2023-11-24T08:09:47.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b37fe640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40b2400 0x7f97b415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:09:47.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.827+0000 7f97b3fff640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 0x7f97b4159500 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00ca70 tx=0x7f97ac00cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.828+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac00abd0 con 0x7f97b40b2de0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.828+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b40c8b50 con 0x7f97b40b2de0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.828+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97ac00ad30 con 0x7f97b40b2de0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.828+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97ac012600 con 0x7f97b40b2de0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.828+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b40c9110 con 0x7f97b40b2de0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.829+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b40c89b0 con 0x7f97b40b2de0 2023-11-24T08:09:47.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.830+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f97ac007070 con 0x7f97b40b2de0 2023-11-24T08:09:47.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.831+0000 7f97b0ff9640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f979406ae40 0x7f979406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:47.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.831+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f97ac089140 con 0x7f97b40b2de0 2023-11-24T08:09:47.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.832+0000 7f97b37fe640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f979406ae40 0x7f979406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:47.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.832+0000 7f97b37fe640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f979406ae40 0x7f979406d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97a4036000 tx=0x7f97a403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:47.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:47.836+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97b40c89b0 con 0x7f97b40b2de0 2023-11-24T08:09:48.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:48.050+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f97b40c89b0 con 0x7f97b40b2de0 2023-11-24T08:09:49.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.304+0000 7f97b0ff9640 1 -- 172.21.15.106:0/3264529602 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v32) v1 ==== 97+0+0 (secure 0 0 0) 0x7f97b40c89b0 con 0x7f97b40b2de0 2023-11-24T08:09:49.307 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2023-11-24T08:09:49.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f979406ae40 msgr2=0x7f979406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f979406ae40 0x7f979406d300 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97a4036000 tx=0x7f97a403c040 comp rx=0 tx=0).stop 2023-11-24T08:09:49.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 msgr2=0x7f97b4159500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 0x7f97b4159500 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f97ac00ca70 tx=0x7f97ac00cf30 comp rx=0 tx=0).stop 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 shutdown_connections 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97b40b2de0 0x7f97b4159500 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f979406ae40 0x7f979406d300 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97b40cfb90 0x7f97b415e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 --2- 172.21.15.106:0/3264529602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97b40b2400 0x7f97b415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 >> 172.21.15.106:0/3264529602 conn(0x7f97b40999d0 msgr2=0x7f97b40ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 shutdown_connections 2023-11-24T08:09:49.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.308+0000 7f97b9b89640 1 -- 172.21.15.106:0/3264529602 wait complete. 2023-11-24T08:09:49.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:09:49.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.536+0000 7fe68a421640 1 Processor -- start 2023-11-24T08:09:49.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.536+0000 7fe68a421640 1 -- start start 2023-11-24T08:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe68a421640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6840b8e00 0x7fe6840d1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe68a421640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 0x7fe6840d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe68a421640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6840d2170 con 0x7fe68415d5a0 2023-11-24T08:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe68a421640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6840b6960 con 0x7fe6840b8e00 2023-11-24T08:09:49.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe68a421640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68415d980 con 0x7fe6840d17a0 2023-11-24T08:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe688c1e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe6840b8e00 0x7fe6840d1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe689c20640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 0x7fe6840d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe689c20640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 0x7fe6840d1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:44198/0 (socket says 172.21.15.106:44198) 2023-11-24T08:09:49.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.538+0000 7fe689c20640 1 -- 172.21.15.106:0/1520071591 learned_addr learned my addr 172.21.15.106:0/1520071591 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.539+0000 7fe689c20640 1 -- 172.21.15.106:0/1520071591 >> v2:172.21.15.156:3300/0 conn(0x7fe6840b8e00 msgr2=0x7fe6840d1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.539+0000 7fe689c20640 1 --2- 172.21.15.106:0/1520071591 >> v2:172.21.15.156:3300/0 conn(0x7fe6840b8e00 0x7fe6840d1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.539+0000 7fe689c20640 1 -- 172.21.15.106:0/1520071591 >> v1:172.21.15.156:6789/0 conn(0x7fe68415d5a0 legacy=0x7fe6840ce820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.539+0000 7fe689c20640 1 -- 172.21.15.106:0/1520071591 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6840b6050 con 0x7fe6840d17a0 2023-11-24T08:09:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.541+0000 7fe689c20640 1 --2- 172.21.15.106:0/1520071591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 0x7fe6840d1c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009e50 tx=0x7fe67c02bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7fdc33d6a45a1c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.541+0000 7fe67bfff640 1 -- 172.21.15.106:0/1520071591 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c02cb00 con 0x7fe6840d17a0 2023-11-24T08:09:49.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.541+0000 7fe67bfff640 1 -- 172.21.15.106:0/1520071591 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c02cc60 con 0x7fe6840d17a0 2023-11-24T08:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.541+0000 7fe67bfff640 1 -- 172.21.15.106:0/1520071591 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c0343f0 con 0x7fe6840d17a0 2023-11-24T08:09:49.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 -- 172.21.15.106:0/1520071591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 msgr2=0x7fe6840d1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 --2- 172.21.15.106:0/1520071591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 0x7fe6840d1c30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009e50 tx=0x7fe67c02bf60 comp rx=0 tx=0).stop 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 -- 172.21.15.106:0/1520071591 shutdown_connections 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 --2- 172.21.15.106:0/1520071591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840d17a0 0x7fe6840d1c30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 --2- 172.21.15.106:0/1520071591 >> v2:172.21.15.156:3300/0 conn(0x7fe6840b8e00 0x7fe6840d1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 -- 172.21.15.106:0/1520071591 >> 172.21.15.106:0/1520071591 conn(0x7fe68409c550 msgr2=0x7fe6840d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 -- 172.21.15.106:0/1520071591 shutdown_connections 2023-11-24T08:09:49.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.542+0000 7fe68a421640 1 -- 172.21.15.106:0/1520071591 wait complete. 2023-11-24T08:09:49.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.544+0000 7fe68a421640 1 Processor -- start 2023-11-24T08:09:49.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.544+0000 7fe68a421640 1 -- start start 2023-11-24T08:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68a421640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840b8e00 0x7fe684153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:49.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68a421640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6840d17a0 0x7fe68415e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68a421640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 0x7fe684154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68a421640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe684154b10 con 0x7fe68415d5a0 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68a421640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68415eda0 con 0x7fe6840d17a0 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68a421640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68415c3a0 con 0x7fe6840b8e00 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 0x7fe684154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 0x7fe684154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32934/0 (socket says 172.21.15.106:32934) 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 -- 172.21.15.106:0/3647709845 learned_addr learned my addr 172.21.15.106:0/3647709845 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe68941f640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840b8e00 0x7fe684153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:49.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe688c1e640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6840d17a0 0x7fe68415e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 -- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840b8e00 msgr2=0x7fe684153c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840b8e00 0x7fe684153c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 -- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6840d17a0 msgr2=0x7fe68415e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6840d17a0 0x7fe68415e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.546+0000 7fe689c20640 1 -- 172.21.15.106:0/3647709845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c009b30 con 0x7fe68415d5a0 2023-11-24T08:09:49.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.547+0000 7fe689c20640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 0x7fe684154fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009e50 tx=0x7fe67c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.547+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c009510 con 0x7fe68415d5a0 2023-11-24T08:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.547+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67c03e450 con 0x7fe68415d5a0 2023-11-24T08:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.547+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67c03e800 con 0x7fe68415d5a0 2023-11-24T08:09:49.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.547+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe68415c680 con 0x7fe68415d5a0 2023-11-24T08:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.547+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6841547a0 con 0x7fe68415d5a0 2023-11-24T08:09:49.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.549+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe684154960 con 0x7fe68415d5a0 2023-11-24T08:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.550+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fe67c03e960 con 0x7fe68415d5a0 2023-11-24T08:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.550+0000 7fe679ffb640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe66806ae40 0x7fe66806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.551+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe67c0abff0 con 0x7fe68415d5a0 2023-11-24T08:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.551+0000 7fe68941f640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe66806ae40 0x7fe66806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:49.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.551+0000 7fe68941f640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe66806ae40 0x7fe66806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe680000f60 tx=0x7fe680009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:49.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.555+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe67c0773b0 con 0x7fe68415d5a0 2023-11-24T08:09:49.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.817+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe684167040 con 0x7fe68415d5a0 2023-11-24T08:09:49.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.819+0000 7fe679ffb640 1 -- 172.21.15.106:0/3647709845 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v3) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe67c07a9d0 con 0x7fe68415d5a0 2023-11-24T08:09:49.821 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A==", 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.822 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg==", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:09:49.823 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q==", 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.824 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw==", 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.825 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2023-11-24T08:09:49.826 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA==", 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA==", 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA==", 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.828 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw==", 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2023-11-24T08:09:49.829 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw==", 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw==", 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA==", 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.831 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug==", 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2023-11-24T08:09:49.832 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ==", 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg==", 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg==", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:09:49.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w==", 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.835 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBludbKAhAARxKqsuG6+yx4M/G/4IloBw==", 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mds", 2023-11-24T08:09:49.836 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQD8WWBl232tDxAAy6cpyo67rAqs8QOm2lHCWw==", 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mds" 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-mgr", 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQD8WWBl+4itDxAAXhiNAw4Us3GbClcby9jnrw==", 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.837 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-osd", 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQD8WWBlcpGtDxAA5EHA8bvBdTfuMlwFM7RV2g==", 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-osd" 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd", 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQD8WWBloJmtDxAAKPrqwHwwDvdddVrTBWJs4A==", 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.838 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQD8WWBlQqWtDxAAUvIe0G3PC81bDfhyovGuXQ==", 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-24T08:09:49.839 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.bootstrap-rgw", 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQD8WWBl2LCtDxAAy9x/X/BD3K1mcAtMODtbaA==", 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.840 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ==", 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA==", 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:09:49.841 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:09:49.842 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:09:49.842 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:09:49.842 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.842 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:49.842 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2023-11-24T08:09:49.842 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.821+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe66806ae40 msgr2=0x7fe66806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.821+0000 7fe68a421640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe66806ae40 0x7fe66806d300 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe680000f60 tx=0x7fe680009040 comp rx=0 tx=0).stop 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.821+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 msgr2=0x7fe684154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 0x7fe684154fa0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe67c009e50 tx=0x7fe67c004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 shutdown_connections 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe68415d5a0 0x7fe684154fa0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6840d17a0 0x7fe68415e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe66806ae40 0x7fe66806d300 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 --2- 172.21.15.106:0/3647709845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe6840b8e00 0x7fe684153c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 >> 172.21.15.106:0/3647709845 conn(0x7fe68409c550 msgr2=0x7fe6840cf370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:49.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 shutdown_connections 2023-11-24T08:09:49.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:49.822+0000 7fe68a421640 1 -- 172.21.15.106:0/3647709845 wait complete. 2023-11-24T08:09:49.845 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-24T08:09:50.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.047+0000 7f56e653c640 1 Processor -- start 2023-11-24T08:09:50.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.047+0000 7f56e653c640 1 -- start start 2023-11-24T08:09:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.048+0000 7f56e653c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56e015c7f0 0x7f56e006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.048+0000 7f56e653c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f56e00ce6c0 0x7f56e00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.048+0000 7f56e653c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56e00cece0 con 0x7f56e00cf450 2023-11-24T08:09:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.048+0000 7f56e653c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e00cfe40 con 0x7f56e015c7f0 2023-11-24T08:09:50.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.048+0000 7f56e653c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56e00ceaf0 con 0x7f56e00ce6c0 2023-11-24T08:09:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56dffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56e015c7f0 0x7f56e006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56d7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f56e00cf450 0x7f56e00cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35438/0 (socket says 172.21.15.106:35438) 2023-11-24T08:09:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56d7fff640 1 -- 172.21.15.106:0/2524170601 learned_addr learned my addr 172.21.15.106:0/2524170601 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:50.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56e4ad9640 1 --2- 172.21.15.106:0/2524170601 >> v2:172.21.15.156:3300/0 conn(0x7f56e00ce6c0 0x7f56e00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56dffff640 1 -- 172.21.15.106:0/2524170601 >> v2:172.21.15.156:3300/0 conn(0x7f56e00ce6c0 msgr2=0x7f56e00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56dffff640 1 --2- 172.21.15.106:0/2524170601 >> v2:172.21.15.156:3300/0 conn(0x7f56e00ce6c0 0x7f56e00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56dffff640 1 -- 172.21.15.106:0/2524170601 >> v1:172.21.15.106:6789/0 conn(0x7f56e00cf450 legacy=0x7f56e00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.049+0000 7f56dffff640 1 -- 172.21.15.106:0/2524170601 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56e006d040 con 0x7f56e015c7f0 2023-11-24T08:09:50.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.050+0000 7f56dffff640 1 --2- 172.21.15.106:0/2524170601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e015c7f0 0x7f56e006db20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f56e00d00e0 tx=0x7f56d002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c4266bcc5c4d8af server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56df7fe640 1 -- 172.21.15.106:0/2524170601 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0031830 con 0x7f56e015c7f0 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56df7fe640 1 -- 172.21.15.106:0/2524170601 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0031990 con 0x7f56e015c7f0 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56df7fe640 1 -- 172.21.15.106:0/2524170601 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0031d20 con 0x7f56e015c7f0 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56e653c640 1 -- 172.21.15.106:0/2524170601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e015c7f0 msgr2=0x7f56e006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56e653c640 1 --2- 172.21.15.106:0/2524170601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e015c7f0 0x7f56e006db20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f56e00d00e0 tx=0x7f56d002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56e653c640 1 -- 172.21.15.106:0/2524170601 shutdown_connections 2023-11-24T08:09:50.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56e653c640 1 --2- 172.21.15.106:0/2524170601 >> v2:172.21.15.156:3300/0 conn(0x7f56e00ce6c0 0x7f56e00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56e653c640 1 --2- 172.21.15.106:0/2524170601 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e015c7f0 0x7f56e006db20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.051+0000 7f56e653c640 1 -- 172.21.15.106:0/2524170601 >> 172.21.15.106:0/2524170601 conn(0x7f56e006a920 msgr2=0x7f56e006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:50.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.052+0000 7f56e653c640 1 -- 172.21.15.106:0/2524170601 shutdown_connections 2023-11-24T08:09:50.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.052+0000 7f56e653c640 1 -- 172.21.15.106:0/2524170601 wait complete. 2023-11-24T08:09:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.053+0000 7f56e653c640 1 Processor -- start 2023-11-24T08:09:50.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.053+0000 7f56e653c640 1 -- start start 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.054+0000 7f56e653c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 0x7f56e0157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.054+0000 7f56e653c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56e00cf450 0x7f56e014bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.054+0000 7f56e653c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e015c7f0 0x7f56e00ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.054+0000 7f56e653c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e0158c90 con 0x7f56e00ce6c0 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.054+0000 7f56e653c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e0158010 con 0x7f56e015c7f0 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.054+0000 7f56e653c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56e01582f0 con 0x7f56e00cf450 2023-11-24T08:09:50.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 0x7f56e0157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 0x7f56e0157a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32948/0 (socket says 172.21.15.106:32948) 2023-11-24T08:09:50.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 -- 172.21.15.106:0/1428852189 learned_addr learned my addr 172.21.15.106:0/1428852189 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56d7fff640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56e00cf450 0x7f56e014bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56e4ad9640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e015c7f0 0x7f56e00ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 -- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56e00cf450 msgr2=0x7f56e014bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56e00cf450 0x7f56e014bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 -- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e015c7f0 msgr2=0x7f56e00ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e015c7f0 0x7f56e00ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 -- 172.21.15.106:0/1428852189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d0009400 con 0x7f56e00ce6c0 2023-11-24T08:09:50.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.055+0000 7f56dffff640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 0x7f56e0157a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.056+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d003aa60 con 0x7f56e00ce6c0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.056+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d003abe0 con 0x7f56e00ce6c0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.056+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00443f0 con 0x7f56e00ce6c0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.056+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e01585d0 con 0x7f56e00ce6c0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.056+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e00ce1e0 con 0x7f56e00ce6c0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.057+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56e0067470 con 0x7f56e00ce6c0 2023-11-24T08:09:50.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.058+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f56d0044550 con 0x7f56e00ce6c0 2023-11-24T08:09:50.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.059+0000 7f56dd7fa640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f56c006ae70 0x7f56c006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.059+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f56d00ab630 con 0x7f56e00ce6c0 2023-11-24T08:09:50.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.059+0000 7f56d7fff640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f56c006ae70 0x7f56c006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.060+0000 7f56d7fff640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f56c006ae70 0x7f56c006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f56c8007fd0 tx=0x7f56c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:50.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.063+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56d0076b40 con 0x7f56e00ce6c0 2023-11-24T08:09:50.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.320+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f56e00cd730 con 0x7f56e00ce6c0 2023-11-24T08:09:50.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.324+0000 7f56dd7fa640 1 -- 172.21.15.106:0/1428852189 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f56d0035020 con 0x7f56e00ce6c0 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f56c006ae70 msgr2=0x7f56c006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f56c006ae70 0x7f56c006d330 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f56c8007fd0 tx=0x7f56c8009040 comp rx=0 tx=0).stop 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 msgr2=0x7f56e0157a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 0x7f56e0157a10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f56d0000c00 tx=0x7f56d00046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 shutdown_connections 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e015c7f0 0x7f56e00ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f56c006ae70 0x7f56c006d330 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56e00cf450 0x7f56e014bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 --2- 172.21.15.106:0/1428852189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56e00ce6c0 0x7f56e0157a10 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 >> 172.21.15.106:0/1428852189 conn(0x7f56e006a920 msgr2=0x7f56e0071890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 shutdown_connections 2023-11-24T08:09:50.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.327+0000 7f56e653c640 1 -- 172.21.15.106:0/1428852189 wait complete. 2023-11-24T08:09:50.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-24T08:09:50.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.542+0000 7f4a50cc2640 1 Processor -- start 2023-11-24T08:09:50.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.542+0000 7f4a50cc2640 1 -- start start 2023-11-24T08:09:50.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a50cc2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4a4c159ce0 0x7f4a4c06de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a50cc2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a4c0b21d0 0x7f4a4c0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a50cc2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c0cbf80 0x7f4a4c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a50cc2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c0cc410 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a50cc2640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a4c0b3a40 con 0x7f4a4c159ce0 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a50cc2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0cdfa0 con 0x7f4a4c0cbf80 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a4c0b21d0 0x7f4a4c0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4a4c0b21d0 0x7f4a4c0cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32964/0 (socket says 172.21.15.106:32964) 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 -- 172.21.15.106:0/574106675 learned_addr learned my addr 172.21.15.106:0/574106675 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a4a59c640 1 --2- 172.21.15.106:0/574106675 >> v2:172.21.15.156:3300/0 conn(0x7f4a4c159ce0 0x7f4a4c06de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a4ad9d640 1 --2- 172.21.15.106:0/574106675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c0cbf80 0x7f4a4c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 -- 172.21.15.106:0/574106675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c0cbf80 msgr2=0x7f4a4c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 --2- 172.21.15.106:0/574106675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c0cbf80 0x7f4a4c15d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 -- 172.21.15.106:0/574106675 >> v2:172.21.15.156:3300/0 conn(0x7f4a4c159ce0 msgr2=0x7f4a4c06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:50.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 --2- 172.21.15.106:0/574106675 >> v2:172.21.15.156:3300/0 conn(0x7f4a4c159ce0 0x7f4a4c06de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a49d9b640 1 -- 172.21.15.106:0/574106675 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a4c0cd9e0 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.543+0000 7f4a4a59c640 1 --2- 172.21.15.106:0/574106675 >> v2:172.21.15.156:3300/0 conn(0x7f4a4c159ce0 0x7f4a4c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.545+0000 7f4a49d9b640 1 --2- 172.21.15.106:0/574106675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c0cba40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06e770 tx=0x7f4a3402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b7482afcec4716 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.545+0000 7f4a4959a640 1 -- 172.21.15.106:0/574106675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34031870 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.545+0000 7f4a4959a640 1 -- 172.21.15.106:0/574106675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a340319d0 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.545+0000 7f4a4959a640 1 -- 172.21.15.106:0/574106675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a34031d60 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 -- 172.21.15.106:0/574106675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 msgr2=0x7f4a4c0cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/574106675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c0cba40 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c06e770 tx=0x7f4a3402b910 comp rx=0 tx=0).stop 2023-11-24T08:09:50.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 -- 172.21.15.106:0/574106675 shutdown_connections 2023-11-24T08:09:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/574106675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c0cbf80 0x7f4a4c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/574106675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c0cba40 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/574106675 >> v2:172.21.15.156:3300/0 conn(0x7f4a4c159ce0 0x7f4a4c06de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 -- 172.21.15.106:0/574106675 >> 172.21.15.106:0/574106675 conn(0x7f4a4c06a920 msgr2=0x7f4a4c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 -- 172.21.15.106:0/574106675 shutdown_connections 2023-11-24T08:09:50.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.546+0000 7f4a50cc2640 1 -- 172.21.15.106:0/574106675 wait complete. 2023-11-24T08:09:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.548+0000 7f4a50cc2640 1 Processor -- start 2023-11-24T08:09:50.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.548+0000 7f4a50cc2640 1 -- start start 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.549+0000 7f4a50cc2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c152020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a50cc2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a4c0cbf80 0x7f4a4c0c4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a50cc2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c159ce0 0x7f4a4c0c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a50cc2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c0caf20 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a50cc2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c152b40 con 0x7f4a4c0cbf80 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a50cc2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a4c1f3e70 con 0x7f4a4c159ce0 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c152020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c152020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32966/0 (socket says 172.21.15.106:32966) 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 -- 172.21.15.106:0/49462255 learned_addr learned my addr 172.21.15.106:0/49462255 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:50.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a49d9b640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a4c0cbf80 0x7f4a4c0c4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4ad9d640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c159ce0 0x7f4a4c0c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 -- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c159ce0 msgr2=0x7f4a4c0c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c159ce0 0x7f4a4c0c50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 -- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a4c0cbf80 msgr2=0x7f4a4c0c4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a4c0cbf80 0x7f4a4c0c4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4a59c640 1 -- 172.21.15.106:0/49462255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a340094d0 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.550+0000 7f4a4ad9d640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c159ce0 0x7f4a4c0c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:50.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.551+0000 7f4a4a59c640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c152020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00cf60 tx=0x7f4a3c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:50.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.551+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00d840 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.551+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00d9a0 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.551+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a4c1f3b90 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.551+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00dd30 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.551+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4c0994c0 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.552+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4c067470 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.553+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4a3c015440 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.554+0000 7f4a42ffd640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4a2c06ae40 0x7f4a2c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.554+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4a3c089c70 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.554+0000 7f4a49d9b640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4a2c06ae40 0x7f4a2c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:50.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.555+0000 7f4a49d9b640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4a2c06ae40 0x7f4a2c06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a34036000 tx=0x7f4a3403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:50.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.559+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4a3c055080 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.796+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f4a4c073b10 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.801+0000 7f4a42ffd640 1 -- 172.21.15.106:0/49462255 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f4a3c01a020 con 0x7f4a4c0b21d0 2023-11-24T08:09:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4a2c06ae40 msgr2=0x7f4a2c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4a2c06ae40 0x7f4a2c06d300 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4a34036000 tx=0x7f4a3403c040 comp rx=0 tx=0).stop 2023-11-24T08:09:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 msgr2=0x7f4a4c152020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c152020 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00cf60 tx=0x7f4a3c00cf90 comp rx=0 tx=0).stop 2023-11-24T08:09:50.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 shutdown_connections 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4a2c06ae40 0x7f4a2c06d300 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4a4c159ce0 0x7f4a4c0c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a4c0cbf80 0x7f4a4c0c4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 --2- 172.21.15.106:0/49462255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4a4c0b21d0 0x7f4a4c152020 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 >> 172.21.15.106:0/49462255 conn(0x7f4a4c06a920 msgr2=0x7f4a4c0ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 shutdown_connections 2023-11-24T08:09:50.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:50.804+0000 7f4a50cc2640 1 -- 172.21.15.106:0/49462255 wait complete. 2023-11-24T08:09:50.823 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-24T08:09:51.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.027+0000 7fbf98dac640 1 Processor -- start 2023-11-24T08:09:51.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.027+0000 7fbf98dac640 1 -- start start 2023-11-24T08:09:51.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf98dac640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbf9415d5c0 0x7fbf940b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf98dac640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbf940d1610 0x7fbf940d01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf98dac640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf940d0af0 con 0x7fbf940d06e0 2023-11-24T08:09:51.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf98dac640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf940d1d80 con 0x7fbf940d1610 2023-11-24T08:09:51.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf98dac640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf940d2060 con 0x7fbf9415d5c0 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf8affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbf940d1610 0x7fbf940d01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf8affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbf940d1610 0x7fbf940d01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32978/0 (socket says 172.21.15.106:32978) 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf8affd640 1 -- 172.21.15.106:0/1196924469 learned_addr learned my addr 172.21.15.106:0/1196924469 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.029+0000 7fbf937fe640 1 --2- 172.21.15.106:0/1196924469 >> v2:172.21.15.156:3300/0 conn(0x7fbf9415d5c0 0x7fbf940b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.030+0000 7fbf8affd640 1 -- 172.21.15.106:0/1196924469 >> v2:172.21.15.156:3300/0 conn(0x7fbf9415d5c0 msgr2=0x7fbf940b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.030+0000 7fbf8affd640 1 --2- 172.21.15.106:0/1196924469 >> v2:172.21.15.156:3300/0 conn(0x7fbf9415d5c0 0x7fbf940b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.030+0000 7fbf8affd640 1 -- 172.21.15.106:0/1196924469 >> v1:172.21.15.156:6789/0 conn(0x7fbf940d06e0 legacy=0x7fbf940b4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.030+0000 7fbf8affd640 1 -- 172.21.15.106:0/1196924469 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf940d2340 con 0x7fbf940d1610 2023-11-24T08:09:51.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.030+0000 7fbf937fe640 1 --2- 172.21.15.106:0/1196924469 >> v2:172.21.15.156:3300/0 conn(0x7fbf9415d5c0 0x7fbf940b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:09:51.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.031+0000 7fbf8affd640 1 --2- 172.21.15.106:0/1196924469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940d01a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf940d4340 tx=0x7fbf7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2177def983c0c640 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:51.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.031+0000 7fbf92ffd640 1 -- 172.21.15.106:0/1196924469 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c031870 con 0x7fbf940d1610 2023-11-24T08:09:51.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.031+0000 7fbf92ffd640 1 -- 172.21.15.106:0/1196924469 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0319d0 con 0x7fbf940d1610 2023-11-24T08:09:51.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.031+0000 7fbf92ffd640 1 -- 172.21.15.106:0/1196924469 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c031d60 con 0x7fbf940d1610 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 -- 172.21.15.106:0/1196924469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 msgr2=0x7fbf940d01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 --2- 172.21.15.106:0/1196924469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940d01a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fbf940d4340 tx=0x7fbf7c02b910 comp rx=0 tx=0).stop 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 -- 172.21.15.106:0/1196924469 shutdown_connections 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 --2- 172.21.15.106:0/1196924469 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940d01a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 --2- 172.21.15.106:0/1196924469 >> v2:172.21.15.156:3300/0 conn(0x7fbf9415d5c0 0x7fbf940b8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 -- 172.21.15.106:0/1196924469 >> 172.21.15.106:0/1196924469 conn(0x7fbf94061310 msgr2=0x7fbf94061720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 -- 172.21.15.106:0/1196924469 shutdown_connections 2023-11-24T08:09:51.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.032+0000 7fbf98dac640 1 -- 172.21.15.106:0/1196924469 wait complete. 2023-11-24T08:09:51.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.034+0000 7fbf98dac640 1 Processor -- start 2023-11-24T08:09:51.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.034+0000 7fbf98dac640 1 -- start start 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf98dac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf940d06e0 0x7fbf94148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf98dac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940b3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf98dac640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf9415d5c0 0x7fbf940b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf98dac640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf941487b0 con 0x7fbf940d1610 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf98dac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf9415c3c0 con 0x7fbf940d06e0 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf98dac640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf940ca810 con 0x7fbf9415d5c0 2023-11-24T08:09:51.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf8affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf8affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:32994/0 (socket says 172.21.15.106:32994) 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf8affd640 1 -- 172.21.15.106:0/604336819 learned_addr learned my addr 172.21.15.106:0/604336819 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.035+0000 7fbf93fff640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf9415d5c0 0x7fbf940b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf8affd640 1 -- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf9415d5c0 msgr2=0x7fbf940b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf8affd640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf9415d5c0 0x7fbf940b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf937fe640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf940d06e0 0x7fbf94148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf8affd640 1 -- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf940d06e0 msgr2=0x7fbf94148270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf8affd640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf940d06e0 0x7fbf94148270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf8affd640 1 -- 172.21.15.106:0/604336819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c0094d0 con 0x7fbf940d1610 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf93fff640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf9415d5c0 0x7fbf940b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:51.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf937fe640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf940d06e0 0x7fbf94148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:51.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf8affd640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940b3350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c02b950 tx=0x7fbf7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:51.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c03ad30 con 0x7fbf940d1610 2023-11-24T08:09:51.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf7c0443f0 con 0x7fbf940d1610 2023-11-24T08:09:51.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c0447a0 con 0x7fbf940d1610 2023-11-24T08:09:51.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf940caaf0 con 0x7fbf940d1610 2023-11-24T08:09:51.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.036+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf941e5e90 con 0x7fbf940d1610 2023-11-24T08:09:51.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.037+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf9404e840 con 0x7fbf940d1610 2023-11-24T08:09:51.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.038+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbf7c04d450 con 0x7fbf940d1610 2023-11-24T08:09:51.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.040+0000 7fbf90ff9640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf7406ae70 0x7fbf7406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.040+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf7c02d040 con 0x7fbf940d1610 2023-11-24T08:09:51.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.040+0000 7fbf937fe640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf7406ae70 0x7fbf7406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.041+0000 7fbf937fe640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf7406ae70 0x7fbf7406d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009540 tx=0x7fbf8400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:51.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.044+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf7c076bf0 con 0x7fbf940d1610 2023-11-24T08:09:51.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.298+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fbf94148a00 con 0x7fbf940d1610 2023-11-24T08:09:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.302+0000 7fbf90ff9640 1 -- 172.21.15.106:0/604336819 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7fbf7c035020 con 0x7fbf940d1610 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf7406ae70 msgr2=0x7fbf7406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf7406ae70 0x7fbf7406d330 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009540 tx=0x7fbf8400b040 comp rx=0 tx=0).stop 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 msgr2=0x7fbf940b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940b3350 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c02b950 tx=0x7fbf7c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 shutdown_connections 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf7406ae70 0x7fbf7406d330 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf9415d5c0 0x7fbf940b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf940d1610 0x7fbf940b3350 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 --2- 172.21.15.106:0/604336819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf940d06e0 0x7fbf94148270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 >> 172.21.15.106:0/604336819 conn(0x7fbf94061310 msgr2=0x7fbf940d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 shutdown_connections 2023-11-24T08:09:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.305+0000 7fbf98dac640 1 -- 172.21.15.106:0/604336819 wait complete. 2023-11-24T08:09:51.325 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-24T08:09:51.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.532+0000 7f977f543640 1 Processor -- start 2023-11-24T08:09:51.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.532+0000 7f977f543640 1 -- start start 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.533+0000 7f977f543640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f9778161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.533+0000 7f977f543640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f97780cf450 0x7f97780cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.533+0000 7f977f543640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97780ce0b0 con 0x7f977815dc80 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.533+0000 7f977f543640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97780cfe10 con 0x7f97780cf450 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.533+0000 7f977f543640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9778161960 con 0x7f9778157c70 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977cade640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f97780cf450 0x7f97780cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977cade640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f97780cf450 0x7f97780cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33006/0 (socket says 172.21.15.106:33006) 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977dae0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f977815dc80 0x7f977815e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35446/0 (socket says 172.21.15.106:35446) 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977cade640 1 -- 172.21.15.106:0/1474828840 learned_addr learned my addr 172.21.15.106:0/1474828840 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:51.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977d2df640 1 --2- 172.21.15.106:0/1474828840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f9778161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977cade640 1 -- 172.21.15.106:0/1474828840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 msgr2=0x7f9778161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977cade640 1 --2- 172.21.15.106:0/1474828840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f9778161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.534+0000 7f977cade640 1 -- 172.21.15.106:0/1474828840 >> v1:172.21.15.106:6789/0 conn(0x7f977815dc80 legacy=0x7f977815e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.535+0000 7f977cade640 1 -- 172.21.15.106:0/1474828840 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977815e8d0 con 0x7f97780cf450 2023-11-24T08:09:51.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.535+0000 7f976ffff640 1 -- 172.21.15.106:0/1474828840 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 590211893 0 0) 0x7f97780ce0b0 con 0x7f977815dc80 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.536+0000 7f977cade640 1 --2- 172.21.15.106:0/1474828840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97780cf450 0x7f97780cf8d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9768009870 tx=0x7f976802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d06e0f02919df83 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.536+0000 7f976ffff640 1 -- 172.21.15.106:0/1474828840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768031640 con 0x7f97780cf450 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.536+0000 7f976ffff640 1 -- 172.21.15.106:0/1474828840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97680317a0 con 0x7f97780cf450 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.536+0000 7f976ffff640 1 -- 172.21.15.106:0/1474828840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768031b30 con 0x7f97780cf450 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.536+0000 7f977f543640 1 -- 172.21.15.106:0/1474828840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97780cf450 msgr2=0x7f97780cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.536+0000 7f977f543640 1 --2- 172.21.15.106:0/1474828840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97780cf450 0x7f97780cf8d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9768009870 tx=0x7f976802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.537+0000 7f977f543640 1 -- 172.21.15.106:0/1474828840 shutdown_connections 2023-11-24T08:09:51.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.537+0000 7f977f543640 1 --2- 172.21.15.106:0/1474828840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97780cf450 0x7f97780cf8d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.537+0000 7f977f543640 1 --2- 172.21.15.106:0/1474828840 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f9778161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.537+0000 7f977f543640 1 -- 172.21.15.106:0/1474828840 >> 172.21.15.106:0/1474828840 conn(0x7f977806a920 msgr2=0x7f977806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.537+0000 7f977f543640 1 -- 172.21.15.106:0/1474828840 shutdown_connections 2023-11-24T08:09:51.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.537+0000 7f977f543640 1 -- 172.21.15.106:0/1474828840 wait complete. 2023-11-24T08:09:51.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.539+0000 7f977f543640 1 Processor -- start 2023-11-24T08:09:51.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.539+0000 7f977f543640 1 -- start start 2023-11-24T08:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977f543640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97780cf450 0x7f977814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977f543640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f97781f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977f543640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 0x7f97780ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977f543640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977809a230 con 0x7f977815dc80 2023-11-24T08:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977f543640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977809a510 con 0x7f97780cf450 2023-11-24T08:09:51.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977f543640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977809a7f0 con 0x7f9778157c70 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977dae0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 0x7f97780ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977dae0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 0x7f97780ca090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33022/0 (socket says 172.21.15.106:33022) 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977dae0640 1 -- 172.21.15.106:0/686156382 learned_addr learned my addr 172.21.15.106:0/686156382 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.540+0000 7f977cade640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f97781f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977d2df640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97780cf450 0x7f977814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977dae0640 1 -- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 msgr2=0x7f97781f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977dae0640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f97781f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977dae0640 1 -- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97780cf450 msgr2=0x7f977814a350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977dae0640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97780cf450 0x7f977814a350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977dae0640 1 -- 172.21.15.106:0/686156382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97680094d0 con 0x7f977815dc80 2023-11-24T08:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.541+0000 7f977dae0640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 0x7f97780ca090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f977000c800 tx=0x7f977000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.542+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9770004d60 con 0x7f977815dc80 2023-11-24T08:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.542+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f977000a640 con 0x7f977815dc80 2023-11-24T08:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.542+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f977000aa30 con 0x7f977815dc80 2023-11-24T08:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.542+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977809aad0 con 0x7f977815dc80 2023-11-24T08:09:51.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.542+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780cc330 con 0x7f977815dc80 2023-11-24T08:09:51.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.544+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977809aff0 con 0x7f977815dc80 2023-11-24T08:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.544+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f977000ab90 con 0x7f977815dc80 2023-11-24T08:09:51.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.545+0000 7f976dffb640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f975c06ae40 0x7f975c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.545+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9770088750 con 0x7f977815dc80 2023-11-24T08:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.545+0000 7f977d2df640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f975c06ae40 0x7f975c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:51.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.546+0000 7f977d2df640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f975c06ae40 0x7f975c06d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9774006fd0 tx=0x7f9774008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:51.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.550+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9770053c00 con 0x7f977815dc80 2023-11-24T08:09:51.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.809+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9778067470 con 0x7f977815dc80 2023-11-24T08:09:51.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.813+0000 7f976dffb640 1 -- 172.21.15.106:0/686156382 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9770018020 con 0x7f977815dc80 2023-11-24T08:09:51.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f975c06ae40 msgr2=0x7f975c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f975c06ae40 0x7f975c06d300 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9774006fd0 tx=0x7f9774008040 comp rx=0 tx=0).stop 2023-11-24T08:09:51.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 msgr2=0x7f97780ca090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:51.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 0x7f97780ca090 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f977000c800 tx=0x7f977000ccc0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 shutdown_connections 2023-11-24T08:09:51.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f977815dc80 0x7f97780ca090 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f975c06ae40 0x7f975c06d300 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9778157c70 0x7f97781f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 --2- 172.21.15.106:0/686156382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97780cf450 0x7f977814a350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:51.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 >> 172.21.15.106:0/686156382 conn(0x7f977806a920 msgr2=0x7f97780b3bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:51.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 shutdown_connections 2023-11-24T08:09:51.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:51.816+0000 7f977f543640 1 -- 172.21.15.106:0/686156382 wait complete. 2023-11-24T08:09:51.834 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-24T08:09:52.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.039+0000 7f2826b3e640 1 Processor -- start 2023-11-24T08:09:52.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.039+0000 7f2826b3e640 1 -- start start 2023-11-24T08:09:52.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.040+0000 7f2826b3e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2820072160 0x7f282015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.040+0000 7f2826b3e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28200c49f0 0x7f282006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.040+0000 7f2826b3e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f282006dc40 0x7f28200cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.040+0000 7f2826b3e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282006e0d0 con 0x7f28200c49f0 2023-11-24T08:09:52.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.040+0000 7f2826b3e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282015ec90 con 0x7f2820072160 2023-11-24T08:09:52.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.040+0000 7f2826b3e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28200cd530 con 0x7f282006dc40 2023-11-24T08:09:52.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28200c49f0 0x7f282006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f28200c49f0 0x7f282006d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33034/0 (socket says 172.21.15.106:33034) 2023-11-24T08:09:52.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 -- 172.21.15.106:0/1683800881 learned_addr learned my addr 172.21.15.106:0/1683800881 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:52.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 -- 172.21.15.106:0/1683800881 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f282006dc40 msgr2=0x7f28200cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:52.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f28248da640 1 --2- 172.21.15.106:0/1683800881 >> v2:172.21.15.156:3300/0 conn(0x7f2820072160 0x7f282015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f28250db640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f282006dc40 0x7f28200cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f282006dc40 0x7f28200cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 -- 172.21.15.106:0/1683800881 >> v2:172.21.15.156:3300/0 conn(0x7f2820072160 msgr2=0x7f282015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:52.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 --2- 172.21.15.106:0/1683800881 >> v2:172.21.15.156:3300/0 conn(0x7f2820072160 0x7f282015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f281ffff640 1 -- 172.21.15.106:0/1683800881 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28200c4ed0 con 0x7f28200c49f0 2023-11-24T08:09:52.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.041+0000 7f28250db640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f282006dc40 0x7f28200cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:52.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.042+0000 7f281ffff640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28200c49f0 0x7f282006d700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28200ccae0 tx=0x7f280c02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78f73a837aa15cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.042+0000 7f281f7fe640 1 -- 172.21.15.106:0/1683800881 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c02ee80 con 0x7f28200c49f0 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f281f7fe640 1 -- 172.21.15.106:0/1683800881 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c036440 con 0x7f28200c49f0 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f281f7fe640 1 -- 172.21.15.106:0/1683800881 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c0367b0 con 0x7f28200c49f0 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 -- 172.21.15.106:0/1683800881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28200c49f0 msgr2=0x7f282006d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28200c49f0 0x7f282006d700 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f28200ccae0 tx=0x7f280c02e0d0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 -- 172.21.15.106:0/1683800881 shutdown_connections 2023-11-24T08:09:52.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f282006dc40 0x7f28200cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 --2- 172.21.15.106:0/1683800881 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f28200c49f0 0x7f282006d700 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 --2- 172.21.15.106:0/1683800881 >> v2:172.21.15.156:3300/0 conn(0x7f2820072160 0x7f282015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 -- 172.21.15.106:0/1683800881 >> 172.21.15.106:0/1683800881 conn(0x7f28200b0c50 msgr2=0x7f28200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:52.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 -- 172.21.15.106:0/1683800881 shutdown_connections 2023-11-24T08:09:52.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.043+0000 7f2826b3e640 1 -- 172.21.15.106:0/1683800881 wait complete. 2023-11-24T08:09:52.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.045+0000 7f2826b3e640 1 Processor -- start 2023-11-24T08:09:52.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.045+0000 7f2826b3e640 1 -- start start 2023-11-24T08:09:52.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f2826b3e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f282006dc40 0x7f28200c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f2826b3e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 0x7f28201f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f2826b3e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28200c49f0 0x7f282015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f2826b3e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200c6940 con 0x7f2820072160 2023-11-24T08:09:52.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f2826b3e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28200c7770 con 0x7f282006dc40 2023-11-24T08:09:52.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f2826b3e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28201f4120 con 0x7f28200c49f0 2023-11-24T08:09:52.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f281ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 0x7f28201f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f281ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 0x7f28201f31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33044/0 (socket says 172.21.15.106:33044) 2023-11-24T08:09:52.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.046+0000 7f281ffff640 1 -- 172.21.15.106:0/2004611216 learned_addr learned my addr 172.21.15.106:0/2004611216 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:52.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281ffff640 1 -- 172.21.15.106:0/2004611216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28200c49f0 msgr2=0x7f282015a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281ffff640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28200c49f0 0x7f282015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281ffff640 1 -- 172.21.15.106:0/2004611216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f282006dc40 msgr2=0x7f28200c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281ffff640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f282006dc40 0x7f28200c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281ffff640 1 -- 172.21.15.106:0/2004611216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280c00b770 con 0x7f2820072160 2023-11-24T08:09:52.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281ffff640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 0x7f28201f31a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:52.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c034b80 con 0x7f2820072160 2023-11-24T08:09:52.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280c036e90 con 0x7f2820072160 2023-11-24T08:09:52.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280c041460 con 0x7f2820072160 2023-11-24T08:09:52.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28201f3b60 con 0x7f2820072160 2023-11-24T08:09:52.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.047+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28200c7390 con 0x7f2820072160 2023-11-24T08:09:52.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.049+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28200c78b0 con 0x7f2820072160 2023-11-24T08:09:52.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.049+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f280c04a3b0 con 0x7f2820072160 2023-11-24T08:09:52.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.050+0000 7f281d7fa640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f280406ae40 0x7f280406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.050+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f280c0abad0 con 0x7f2820072160 2023-11-24T08:09:52.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.051+0000 7f28248da640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f280406ae40 0x7f280406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.051+0000 7f28248da640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f280406ae40 0x7f280406d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2814006fd0 tx=0x7f2814008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:52.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.054+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28200c78b0 con 0x7f2820072160 2023-11-24T08:09:52.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.313+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f28200c78b0 con 0x7f2820072160 2023-11-24T08:09:52.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.318+0000 7f281d7fa640 1 -- 172.21.15.106:0/2004611216 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f28200c78b0 con 0x7f2820072160 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.320+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f280406ae40 msgr2=0x7f280406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.320+0000 7f2826b3e640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f280406ae40 0x7f280406d300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2814006fd0 tx=0x7f2814008040 comp rx=0 tx=0).stop 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 msgr2=0x7f28201f31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 0x7f28201f31a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f280c000c00 tx=0x7f280c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 shutdown_connections 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f280406ae40 0x7f280406d300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f28200c49f0 0x7f282015a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2820072160 0x7f28201f31a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 --2- 172.21.15.106:0/2004611216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f282006dc40 0x7f28200c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 >> 172.21.15.106:0/2004611216 conn(0x7f28200b0c50 msgr2=0x7f28200ad870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 shutdown_connections 2023-11-24T08:09:52.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.321+0000 7f2826b3e640 1 -- 172.21.15.106:0/2004611216 wait complete. 2023-11-24T08:09:52.340 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-24T08:09:52.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.548+0000 7f174a9e2640 1 Processor -- start 2023-11-24T08:09:52.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.548+0000 7f174a9e2640 1 -- start start 2023-11-24T08:09:52.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f174a9e2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1744141b90 0x7f17440b0b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f174a9e2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440b1090 0x7f17440718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f174a9e2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1744071e10 0x7f174415f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f174a9e2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17440b14f0 con 0x7f1744141b90 2023-11-24T08:09:52.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f174a9e2640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17441450d0 con 0x7f1744071e10 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f174a9e2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17441422b0 con 0x7f17440b1090 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1744141b90 0x7f17440b0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1744141b90 0x7f17440b0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33046/0 (socket says 172.21.15.106:33046) 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 -- 172.21.15.106:0/823233304 learned_addr learned my addr 172.21.15.106:0/823233304 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 -- 172.21.15.106:0/823233304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440b1090 msgr2=0x7f17440718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 --2- 172.21.15.106:0/823233304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440b1090 0x7f17440718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 -- 172.21.15.106:0/823233304 >> v2:172.21.15.156:3300/0 conn(0x7f1744071e10 msgr2=0x7f174415f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 --2- 172.21.15.106:0/823233304 >> v2:172.21.15.156:3300/0 conn(0x7f1744071e10 0x7f174415f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.550+0000 7f17499e0640 1 -- 172.21.15.106:0/823233304 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17440725e0 con 0x7f1744141b90 2023-11-24T08:09:52.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.551+0000 7f17499e0640 1 --2- 172.21.15.106:0/823233304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1744141b90 0x7f17440b0b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1740009e50 tx=0x7f174002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f1406b1fb73064 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.552+0000 7f17489de640 1 -- 172.21.15.106:0/823233304 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f174002c9f0 con 0x7f1744141b90 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.552+0000 7f17489de640 1 -- 172.21.15.106:0/823233304 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174002cb50 con 0x7f1744141b90 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.552+0000 7f17489de640 1 -- 172.21.15.106:0/823233304 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17400343f0 con 0x7f1744141b90 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.552+0000 7f174a9e2640 1 -- 172.21.15.106:0/823233304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1744141b90 msgr2=0x7f17440b0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.552+0000 7f174a9e2640 1 --2- 172.21.15.106:0/823233304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1744141b90 0x7f17440b0b50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f1740009e50 tx=0x7f174002be50 comp rx=0 tx=0).stop 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 -- 172.21.15.106:0/823233304 shutdown_connections 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 --2- 172.21.15.106:0/823233304 >> v2:172.21.15.156:3300/0 conn(0x7f1744071e10 0x7f174415f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 --2- 172.21.15.106:0/823233304 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f17440b1090 0x7f17440718d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 --2- 172.21.15.106:0/823233304 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1744141b90 0x7f17440b0b50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 -- 172.21.15.106:0/823233304 >> 172.21.15.106:0/823233304 conn(0x7f174406a500 msgr2=0x7f174406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:52.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 -- 172.21.15.106:0/823233304 shutdown_connections 2023-11-24T08:09:52.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.553+0000 7f174a9e2640 1 -- 172.21.15.106:0/823233304 wait complete. 2023-11-24T08:09:52.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.554+0000 7f174a9e2640 1 Processor -- start 2023-11-24T08:09:52.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.554+0000 7f174a9e2640 1 -- start start 2023-11-24T08:09:52.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.555+0000 7f174a9e2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1744071e10 0x7f17440cf840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.555+0000 7f174a9e2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 0x7f17440ada60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.555+0000 7f174a9e2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1744141b90 0x7f17440adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.555+0000 7f174a9e2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174413fb50 con 0x7f17440b1090 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.555+0000 7f174a9e2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f174413fe30 con 0x7f1744071e10 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.555+0000 7f174a9e2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1744140110 con 0x7f1744141b90 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 0x7f17440ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 0x7f17440ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33054/0 (socket says 172.21.15.106:33054) 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 -- 172.21.15.106:0/4125679131 learned_addr learned my addr 172.21.15.106:0/4125679131 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17499e0640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1744071e10 0x7f17440cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f174a1e1640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1744141b90 0x7f17440adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 -- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1744141b90 msgr2=0x7f17440adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1744141b90 0x7f17440adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 -- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1744071e10 msgr2=0x7f17440cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1744071e10 0x7f17440cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 -- 172.21.15.106:0/4125679131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740009b30 con 0x7f17440b1090 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.556+0000 7f17491df640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 0x7f17440ada60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f173400b410 tx=0x7f173400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.557+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734011830 con 0x7f17440b1090 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.557+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17340119b0 con 0x7f17440b1090 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.557+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1734011d60 con 0x7f17440b1090 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.557+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17441403f0 con 0x7f17440b1090 2023-11-24T08:09:52.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.557+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f174413ed50 con 0x7f17440b1090 2023-11-24T08:09:52.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.558+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174404e840 con 0x7f17440b1090 2023-11-24T08:09:52.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.559+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f173401a400 con 0x7f17440b1090 2023-11-24T08:09:52.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.560+0000 7f173a7fc640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:52.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.560+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1734089450 con 0x7f17440b1090 2023-11-24T08:09:52.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.560+0000 7f17499e0640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:52.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.561+0000 7f17499e0640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f172806ae40 0x7f172806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1740035000 tx=0x7f1740030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:52.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.564+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1734054840 con 0x7f17440b1090 2023-11-24T08:09:52.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.804+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f1744142070 con 0x7f17440b1090 2023-11-24T08:09:52.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.815+0000 7f173a7fc640 1 -- 172.21.15.106:0/4125679131 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f1744142070 con 0x7f17440b1090 2023-11-24T08:09:52.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.817+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f172806ae40 msgr2=0x7f172806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f172806ae40 0x7f172806d300 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f1740035000 tx=0x7f1740030000 comp rx=0 tx=0).stop 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 msgr2=0x7f17440ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 0x7f17440ada60 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f173400b410 tx=0x7f173400b8d0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 shutdown_connections 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f172806ae40 0x7f172806d300 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1744141b90 0x7f17440adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f17440b1090 0x7f17440ada60 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 --2- 172.21.15.106:0/4125679131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1744071e10 0x7f17440cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 >> 172.21.15.106:0/4125679131 conn(0x7f174406a500 msgr2=0x7f174406ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 shutdown_connections 2023-11-24T08:09:52.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:52.818+0000 7f174a9e2640 1 -- 172.21.15.106:0/4125679131 wait complete. 2023-11-24T08:09:52.838 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.035+0000 7fce43bec640 1 Processor -- start 2023-11-24T08:09:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.035+0000 7fce43bec640 1 -- start start 2023-11-24T08:09:53.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.036+0000 7fce43bec640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fce3c0ce640 0x7fce3c0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.036+0000 7fce43bec640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c0ce270 con 0x7fce3c0cfb60 2023-11-24T08:09:53.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.036+0000 7fce43bec640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce3c06d0a0 con 0x7fce3c0cc470 2023-11-24T08:09:53.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.036+0000 7fce43bec640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce3c06c560 con 0x7fce3c0ce640 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.037+0000 7fce41187640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fce3c0cfb60 0x7fce3c06d530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35454/0 (socket says 172.21.15.106:35454) 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.037+0000 7fce41187640 1 -- 172.21.15.106:0/1597445664 learned_addr learned my addr 172.21.15.106:0/1597445664 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.037+0000 7fce42189640 1 --2- 172.21.15.106:0/1597445664 >> v2:172.21.15.156:3300/0 conn(0x7fce3c0ce640 0x7fce3c0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.037+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1650002699 0 0) 0x7fce3c0ce270 con 0x7fce3c0cfb60 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.037+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce28003520 con 0x7fce3c0cfb60 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 824021552 0 0) 0x7fce28003520 con 0x7fce3c0cfb60 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 >> v2:172.21.15.156:3300/0 conn(0x7fce3c0ce640 msgr2=0x7fce3c0abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 --2- 172.21.15.106:0/1597445664 >> v2:172.21.15.156:3300/0 conn(0x7fce3c0ce640 0x7fce3c0abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 >> v1:172.21.15.156:6789/0 conn(0x7fce3c0cc470 legacy=0x7fce3c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c06c770 con 0x7fce3c0cfb60 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fce2c001670 con 0x7fce3c0cfb60 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 >> v1:172.21.15.106:6789/0 conn(0x7fce3c0cfb60 legacy=0x7fce3c06d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce280039c0 0x7fce28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280047c0 0x7fce28004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce28005180 0x7fce2800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c06c560 con 0x7fce280039c0 2023-11-24T08:09:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2800c050 con 0x7fce28005180 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2800c210 con 0x7fce280047c0 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce41988640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce280039c0 0x7fce28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce41988640 1 -- 172.21.15.106:0/1597445664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280047c0 msgr2=0x7fce28004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce41988640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280047c0 0x7fce28004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce41988640 1 -- 172.21.15.106:0/1597445664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce28005180 msgr2=0x7fce2800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce41988640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce28005180 0x7fce2800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce41988640 1 -- 172.21.15.106:0/1597445664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce28003520 con 0x7fce280039c0 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.038+0000 7fce42189640 1 -- 172.21.15.106:0/1597445664 reap_dead start 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.039+0000 7fce41988640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce280039c0 0x7fce28003dc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce38003db0 tx=0x7fce280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfe09ca7672c895 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38032600 con 0x7fce280039c0 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3802e050 con 0x7fce280039c0 2023-11-24T08:09:53.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce40986640 1 -- 172.21.15.106:0/1597445664 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce380329e0 con 0x7fce280039c0 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 -- 172.21.15.106:0/1597445664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce280039c0 msgr2=0x7fce28003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce280039c0 0x7fce28003dc0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce38003db0 tx=0x7fce280057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 -- 172.21.15.106:0/1597445664 shutdown_connections 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 --2- 172.21.15.106:0/1597445664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce280039c0 0x7fce28003dc0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 -- 172.21.15.106:0/1597445664 >> 172.21.15.106:0/1597445664 conn(0x7fce3c06a920 msgr2=0x7fce3c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 -- 172.21.15.106:0/1597445664 shutdown_connections 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.040+0000 7fce43bec640 1 -- 172.21.15.106:0/1597445664 wait complete. 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.042+0000 7fce43bec640 1 Processor -- start 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.042+0000 7fce43bec640 1 -- start start 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce43bec640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280039c0 0x7fce3c0af360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce43bec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce3c14d230 0x7fce3c0af8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce43bec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 0x7fce3c1f7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce43bec640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c09be20 con 0x7fce3c099ca0 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce43bec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c1f81b0 con 0x7fce3c14d230 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce43bec640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce3c09b760 con 0x7fce280039c0 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce42189640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 0x7fce3c1f7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce42189640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 0x7fce3c1f7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33072/0 (socket says 172.21.15.106:33072) 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.043+0000 7fce42189640 1 -- 172.21.15.106:0/744192288 learned_addr learned my addr 172.21.15.106:0/744192288 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce42189640 1 -- 172.21.15.106:0/744192288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280039c0 msgr2=0x7fce3c0af360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce42189640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280039c0 0x7fce3c0af360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce42189640 1 -- 172.21.15.106:0/744192288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce3c14d230 msgr2=0x7fce3c0af8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce42189640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce3c14d230 0x7fce3c0af8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce42189640 1 -- 172.21.15.106:0/744192288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c09ba40 con 0x7fce3c099ca0 2023-11-24T08:09:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce42189640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 0x7fce3c1f7440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0cead0 tx=0x7fce3400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34010040 con 0x7fce3c099ca0 2023-11-24T08:09:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3400ba50 con 0x7fce3c099ca0 2023-11-24T08:09:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3400be00 con 0x7fce3c099ca0 2023-11-24T08:09:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce3c09b410 con 0x7fce3c099ca0 2023-11-24T08:09:53.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.044+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce3c1f7b60 con 0x7fce3c099ca0 2023-11-24T08:09:53.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.045+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3c09bf60 con 0x7fce3c099ca0 2023-11-24T08:09:53.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.046+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fce3401d400 con 0x7fce3c099ca0 2023-11-24T08:09:53.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.047+0000 7fce327fc640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fce2006ae40 0x7fce2006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.047+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fce34089990 con 0x7fce3c099ca0 2023-11-24T08:09:53.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.047+0000 7fce41988640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fce2006ae40 0x7fce2006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.048+0000 7fce41988640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fce2006ae40 0x7fce2006d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.051+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fce3c09bf60 con 0x7fce3c099ca0 2023-11-24T08:09:53.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.227+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce3c09bf60 con 0x7fce3c099ca0 2023-11-24T08:09:53.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.229+0000 7fce327fc640 1 -- 172.21.15.106:0/744192288 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) v1 ==== 74+0+12905 (secure 0 0 0) 0x7fce3c09bf60 con 0x7fce3c099ca0 2023-11-24T08:09:53.231 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:53.231 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":32,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:49.302516+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fce2006ae40 msgr2=0x7fce2006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fce2006ae40 0x7fce2006d300 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fce38034000 tx=0x7fce3802f000 comp rx=0 tx=0).stop 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 msgr2=0x7fce3c1f7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 0x7fce3c1f7440 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fce3c0cead0 tx=0x7fce3400eea0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 shutdown_connections 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fce3c099ca0 0x7fce3c1f7440 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce3c14d230 0x7fce3c0af8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fce2006ae40 0x7fce2006d300 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 --2- 172.21.15.106:0/744192288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fce280039c0 0x7fce3c0af360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 >> 172.21.15.106:0/744192288 conn(0x7fce3c06a920 msgr2=0x7fce3c0ad120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:53.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.231+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 shutdown_connections 2023-11-24T08:09:53.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.232+0000 7fce43bec640 1 -- 172.21.15.106:0/744192288 wait complete. 2023-11-24T08:09:53.250 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:09:53.251 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:09:53.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.448+0000 7f6dd2c51640 1 Processor -- start 2023-11-24T08:09:53.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.448+0000 7f6dd2c51640 1 -- start start 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.449+0000 7f6dd2c51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc15b440 0x7f6dcc0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.449+0000 7f6dd2c51640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6dcc0cbf20 0x7f6dcc0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.449+0000 7f6dd2c51640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dcc0cc3a0 con 0x7f6dcc0c57e0 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.449+0000 7f6dd2c51640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dcc15b820 con 0x7f6dcc0cbf20 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.450+0000 7f6dd2c51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc15eea0 con 0x7f6dcc15b440 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.450+0000 7f6dcbfff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6dcc0cbf20 0x7f6dcc0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.450+0000 7f6dcbfff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6dcc0cbf20 0x7f6dcc0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:42510/0 (socket says 172.21.15.106:42510) 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.450+0000 7f6dcbfff640 1 -- 172.21.15.106:0/1770321197 learned_addr learned my addr 172.21.15.106:0/1770321197 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:53.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.450+0000 7f6dd09ed640 1 --2- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc15b440 0x7f6dcc0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.451+0000 7f6dcbfff640 1 -- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc15b440 msgr2=0x7f6dcc0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.451+0000 7f6dcbfff640 1 --2- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc15b440 0x7f6dcc0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.451+0000 7f6dcbfff640 1 -- 172.21.15.106:0/1770321197 >> v1:172.21.15.156:6789/0 conn(0x7f6dcc0c57e0 legacy=0x7f6dcc0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.451+0000 7f6dcbfff640 1 -- 172.21.15.106:0/1770321197 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dcc06d600 con 0x7f6dcc0cbf20 2023-11-24T08:09:53.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.452+0000 7f6dcbfff640 1 --2- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0cbf20 0x7f6dcc0c52a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009870 tx=0x7f6db802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da4ea133d73ba573 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.453+0000 7f6dcb7fe640 1 -- 172.21.15.106:0/1770321197 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8031640 con 0x7f6dcc0cbf20 2023-11-24T08:09:53.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.453+0000 7f6dcb7fe640 1 -- 172.21.15.106:0/1770321197 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db80317a0 con 0x7f6dcc0cbf20 2023-11-24T08:09:53.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.453+0000 7f6dcb7fe640 1 -- 172.21.15.106:0/1770321197 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db8031b30 con 0x7f6dcc0cbf20 2023-11-24T08:09:53.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0cbf20 msgr2=0x7f6dcc0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0cbf20 0x7f6dcc0c52a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f6db8009870 tx=0x7f6db802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1770321197 shutdown_connections 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0cbf20 0x7f6dcc0c52a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1770321197 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc15b440 0x7f6dcc0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1770321197 >> 172.21.15.106:0/1770321197 conn(0x7f6dcc0b0c50 msgr2=0x7f6dcc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1770321197 shutdown_connections 2023-11-24T08:09:53.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.454+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1770321197 wait complete. 2023-11-24T08:09:53.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.456+0000 7f6dd2c51640 1 Processor -- start 2023-11-24T08:09:53.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.456+0000 7f6dd2c51640 1 -- start start 2023-11-24T08:09:53.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.457+0000 7f6dd2c51640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0c57e0 0x7f6dcc158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.457+0000 7f6dd2c51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc0cbf20 0x7f6dcc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd2c51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 0x7f6dcc0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd2c51640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc158f70 con 0x7f6dcc15b440 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd2c51640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1f1160 con 0x7f6dcc0c57e0 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd2c51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dcc1f1440 con 0x7f6dcc0cbf20 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 0x7f6dcc0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 0x7f6dcc0c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33078/0 (socket says 172.21.15.106:33078) 2023-11-24T08:09:53.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 -- 172.21.15.106:0/1963336846 learned_addr learned my addr 172.21.15.106:0/1963336846 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:53.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd09ed640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0c57e0 0x7f6dcc158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dcbfff640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc0cbf20 0x7f6dcc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 -- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc0cbf20 msgr2=0x7f6dcc154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc0cbf20 0x7f6dcc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 -- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0c57e0 msgr2=0x7f6dcc158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0c57e0 0x7f6dcc158a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.458+0000 7f6dd11ee640 1 -- 172.21.15.106:0/1963336846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db80094d0 con 0x7f6dcc15b440 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.459+0000 7f6dd11ee640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 0x7f6dcc0c7cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c7a0 tx=0x7f6dc400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.459+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4003b70 con 0x7f6dcc15b440 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.459+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc4003cf0 con 0x7f6dcc15b440 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.459+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4003ea0 con 0x7f6dcc15b440 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.459+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dcc1f1720 con 0x7f6dcc15b440 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.459+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dcc1f1ce0 con 0x7f6dcc15b440 2023-11-24T08:09:53.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.460+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dcc1f43e0 con 0x7f6dcc15b440 2023-11-24T08:09:53.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.461+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6dc400aa90 con 0x7f6dcc15b440 2023-11-24T08:09:53.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.462+0000 7f6dc97fa640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6db006ae40 0x7f6db006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.462+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6dc4088f80 con 0x7f6dcc15b440 2023-11-24T08:09:53.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.462+0000 7f6dd09ed640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6db006ae40 0x7f6db006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.463+0000 7f6dd09ed640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6db006ae40 0x7f6db006d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc15e960 tx=0x7f6dc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.467+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6dcc1f43e0 con 0x7f6dcc15b440 2023-11-24T08:09:53.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.650+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6dcc0cc3a0 con 0x7f6dcc15b440 2023-11-24T08:09:53.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.651+0000 7f6dc97fa640 1 -- 172.21.15.106:0/1963336846 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v32) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6dcc0cc3a0 con 0x7f6dcc15b440 2023-11-24T08:09:53.653 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6db006ae40 msgr2=0x7f6db006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6db006ae40 0x7f6db006d300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc15e960 tx=0x7f6dc0008040 comp rx=0 tx=0).stop 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 msgr2=0x7f6dcc0c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 0x7f6dcc0c7cc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6dc400c7a0 tx=0x7f6dc400cc60 comp rx=0 tx=0).stop 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 shutdown_connections 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dcc15b440 0x7f6dcc0c7cc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6db006ae40 0x7f6db006d300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dcc0cbf20 0x7f6dcc154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 --2- 172.21.15.106:0/1963336846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dcc0c57e0 0x7f6dcc158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 >> 172.21.15.106:0/1963336846 conn(0x7f6dcc0b0c50 msgr2=0x7f6dcc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:53.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 shutdown_connections 2023-11-24T08:09:53.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.653+0000 7f6dd2c51640 1 -- 172.21.15.106:0/1963336846 wait complete. 2023-11-24T08:09:53.671 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:09:53.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.875+0000 7f330b11d640 1 Processor -- start 2023-11-24T08:09:53.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.876+0000 7f330b11d640 1 -- start start 2023-11-24T08:09:53.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f330b11d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f33040cc470 0x7f33040abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f330b11d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040ce640 0x7f33040b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f330b11d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f33040cf450 0x7f33040cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f330b11d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33040cfe20 con 0x7f33040cc470 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f330b11d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f330406c560 con 0x7f33040cf450 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f330b11d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33040ce270 con 0x7f33040ce640 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f3308eb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f33040cc470 0x7f33040abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f3308eb9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f33040cc470 0x7f33040abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33094/0 (socket says 172.21.15.106:33094) 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.877+0000 7f3308eb9640 1 -- 172.21.15.106:0/2989039724 learned_addr learned my addr 172.21.15.106:0/2989039724 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f3308eb9640 1 -- 172.21.15.106:0/2989039724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040ce640 msgr2=0x7f33040b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f3303fff640 1 --2- 172.21.15.106:0/2989039724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040ce640 0x7f33040b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f33096ba640 1 --2- 172.21.15.106:0/2989039724 >> v2:172.21.15.156:3300/0 conn(0x7f33040cf450 0x7f33040cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2989039724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040ce640 0x7f33040b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f3308eb9640 1 -- 172.21.15.106:0/2989039724 >> v2:172.21.15.156:3300/0 conn(0x7f33040cf450 msgr2=0x7f33040cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:53.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2989039724 >> v2:172.21.15.156:3300/0 conn(0x7f33040cf450 0x7f33040cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f3308eb9640 1 -- 172.21.15.106:0/2989039724 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33040b4200 con 0x7f33040cc470 2023-11-24T08:09:53.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.878+0000 7f33096ba640 1 --2- 172.21.15.106:0/2989039724 >> v2:172.21.15.156:3300/0 conn(0x7f33040cf450 0x7f33040cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.879+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2989039724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040cc470 0x7f33040abfc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f33040cd610 tx=0x7f32f402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1db85a196d657802 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.879+0000 7f33037fe640 1 -- 172.21.15.106:0/2989039724 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f402ca70 con 0x7f33040cc470 2023-11-24T08:09:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.879+0000 7f33037fe640 1 -- 172.21.15.106:0/2989039724 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f402cbd0 con 0x7f33040cc470 2023-11-24T08:09:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.879+0000 7f33037fe640 1 -- 172.21.15.106:0/2989039724 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f40343f0 con 0x7f33040cc470 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 -- 172.21.15.106:0/2989039724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040cc470 msgr2=0x7f33040abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 --2- 172.21.15.106:0/2989039724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040cc470 0x7f33040abfc0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f33040cd610 tx=0x7f32f402bed0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 -- 172.21.15.106:0/2989039724 shutdown_connections 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 --2- 172.21.15.106:0/2989039724 >> v2:172.21.15.156:3300/0 conn(0x7f33040cf450 0x7f33040cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 --2- 172.21.15.106:0/2989039724 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040ce640 0x7f33040b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 --2- 172.21.15.106:0/2989039724 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040cc470 0x7f33040abfc0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 -- 172.21.15.106:0/2989039724 >> 172.21.15.106:0/2989039724 conn(0x7f330406a920 msgr2=0x7f330406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 -- 172.21.15.106:0/2989039724 shutdown_connections 2023-11-24T08:09:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.880+0000 7f330b11d640 1 -- 172.21.15.106:0/2989039724 wait complete. 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.882+0000 7f330b11d640 1 Processor -- start 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.882+0000 7f330b11d640 1 -- start start 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f330b11d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040cc470 0x7f3304155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f330b11d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 0x7f3304071740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f330b11d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040cf450 0x7f3304071c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f330b11d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041591b0 con 0x7f33040ce640 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f330b11d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3304159490 con 0x7f33040cc470 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f330b11d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f330415a5c0 con 0x7f33040cf450 2023-11-24T08:09:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 0x7f3304071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 0x7f3304071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33102/0 (socket says 172.21.15.106:33102) 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 -- 172.21.15.106:0/2947836543 learned_addr learned my addr 172.21.15.106:0/2947836543 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040cc470 0x7f3304155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f33096ba640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040cf450 0x7f3304071c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 -- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040cf450 msgr2=0x7f3304071c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040cf450 0x7f3304071c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 -- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040cc470 msgr2=0x7f3304155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040cc470 0x7f3304155af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3303fff640 1 -- 172.21.15.106:0/2947836543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f4009b30 con 0x7f33040ce640 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.883+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040cc470 0x7f3304155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.884+0000 7f3303fff640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 0x7f3304071740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c7f0 tx=0x7f32ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.884+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec004220 con 0x7f33040ce640 2023-11-24T08:09:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.884+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec004380 con 0x7f33040ce640 2023-11-24T08:09:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.884+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330415a8a0 con 0x7f33040ce640 2023-11-24T08:09:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.884+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec007860 con 0x7f33040ce640 2023-11-24T08:09:53.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.884+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3304158dd0 con 0x7f33040ce640 2023-11-24T08:09:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.885+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3304067470 con 0x7f33040ce640 2023-11-24T08:09:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.886+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f32ec007bd0 con 0x7f33040ce640 2023-11-24T08:09:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.887+0000 7f33017fa640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32e406ae40 0x7f32e406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.887+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f32ec089290 con 0x7f33040ce640 2023-11-24T08:09:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.887+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32e406ae40 0x7f32e406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.888+0000 7f3308eb9640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32e406ae40 0x7f32e406d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32f4035000 tx=0x7f32f4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:53.891+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32ec0545f0 con 0x7f33040ce640 2023-11-24T08:09:54.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.087+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f33040cd910 con 0x7f33040ce640 2023-11-24T08:09:54.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.088+0000 7f33017fa640 1 -- 172.21.15.106:0/2947836543 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v32) v1 ==== 84+0+65 (secure 0 0 0) 0x7f32ec057c10 con 0x7f33040ce640 2023-11-24T08:09:54.089 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:54.090 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:09:54.090 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:09:54.090 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2023-11-24T08:09:54.090 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2023-11-24T08:09:54.090 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:09:54.090 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:09:54.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.090+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32e406ae40 msgr2=0x7f32e406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32e406ae40 0x7f32e406d300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f32f4035000 tx=0x7f32f4030000 comp rx=0 tx=0).stop 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 msgr2=0x7f3304071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 0x7f3304071740 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c7f0 tx=0x7f32ec00ccb0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 shutdown_connections 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32e406ae40 0x7f32e406d300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33040cf450 0x7f3304071c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33040ce640 0x7f3304071740 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 --2- 172.21.15.106:0/2947836543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040cc470 0x7f3304155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 >> 172.21.15.106:0/2947836543 conn(0x7f330406a920 msgr2=0x7f33040b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 shutdown_connections 2023-11-24T08:09:54.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.091+0000 7f330b11d640 1 -- 172.21.15.106:0/2947836543 wait complete. 2023-11-24T08:09:54.109 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:09:54.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:09:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.326+0000 7f49c73fc640 1 Processor -- start 2023-11-24T08:09:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.326+0000 7f49c73fc640 1 -- start start 2023-11-24T08:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.327+0000 7f49c73fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f49c015c7f0 0x7f49c006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.327+0000 7f49c73fc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00cece0 con 0x7f49c00cf450 2023-11-24T08:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.327+0000 7f49c73fc640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c006d040 con 0x7f49c00ce6c0 2023-11-24T08:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.327+0000 7f49c73fc640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00ceaf0 con 0x7f49c015c7f0 2023-11-24T08:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49c4997640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f49c00cf450 0x7f49c00cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39316/0 (socket says 172.21.15.106:39316) 2023-11-24T08:09:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49c4997640 1 -- 172.21.15.106:0/2593112600 learned_addr learned my addr 172.21.15.106:0/2593112600 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:54.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49c5198640 1 --2- 172.21.15.106:0/2593112600 >> v2:172.21.15.156:3300/0 conn(0x7f49c015c7f0 0x7f49c006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18066392 0 0) 0x7f49c00cece0 con 0x7f49c00cf450 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ac003520 con 0x7f49c00cf450 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3380103904 0 0) 0x7f49ac003520 con 0x7f49c00cf450 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 >> v2:172.21.15.156:3300/0 conn(0x7f49c015c7f0 msgr2=0x7f49c006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.328+0000 7f49b7fff640 1 --2- 172.21.15.106:0/2593112600 >> v2:172.21.15.156:3300/0 conn(0x7f49c015c7f0 0x7f49c006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 >> v1:172.21.15.156:6789/0 conn(0x7f49c00ce6c0 legacy=0x7f49c00714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c006d320 con 0x7f49c00cf450 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f49b0001670 con 0x7f49c00cf450 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 >> v1:172.21.15.106:6789/0 conn(0x7f49c00cf450 legacy=0x7f49c00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac0039c0 0x7f49ac003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac0047c0 0x7f49ac004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00c050 con 0x7f49ac005180 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00c1c0 con 0x7f49ac0047c0 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ac00c380 con 0x7f49ac0039c0 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5999640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5198640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac0039c0 0x7f49ac003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5999640 1 -- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac0039c0 msgr2=0x7f49ac003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5999640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac0039c0 0x7f49ac003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c4997640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac0047c0 0x7f49ac004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5999640 1 -- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac0047c0 msgr2=0x7f49ac004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5999640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49ac0047c0 0x7f49ac004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.329+0000 7f49c5999640 1 -- 172.21.15.106:0/2593112600 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac003520 con 0x7f49ac005180 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.330+0000 7f49c5999640 1 -- 172.21.15.106:0/2593112600 reap_dead start 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.330+0000 7f49c5999640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f49b8001e50 tx=0x7f49ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864d429c5b6d6a65 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8003b70 con 0x7f49ac005180 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8003d10 con 0x7f49ac005180 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49b7fff640 1 -- 172.21.15.106:0/2593112600 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8005310 con 0x7f49ac005180 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 -- 172.21.15.106:0/2593112600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac005180 msgr2=0x7f49ac00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f49b8001e50 tx=0x7f49ac0057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 -- 172.21.15.106:0/2593112600 shutdown_connections 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2593112600 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49ac005180 0x7f49ac00b980 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 -- 172.21.15.106:0/2593112600 >> 172.21.15.106:0/2593112600 conn(0x7f49c006a920 msgr2=0x7f49c006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:54.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 -- 172.21.15.106:0/2593112600 shutdown_connections 2023-11-24T08:09:54.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.331+0000 7f49c73fc640 1 -- 172.21.15.106:0/2593112600 wait complete. 2023-11-24T08:09:54.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.333+0000 7f49c73fc640 1 Processor -- start 2023-11-24T08:09:54.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.333+0000 7f49c73fc640 1 -- start start 2023-11-24T08:09:54.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c73fc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac005180 0x7f49c01589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c73fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c0155de0 0x7f49c01483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c73fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 0x7f49c0156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c73fc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0150920 con 0x7f49c01f33f0 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c73fc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c0148910 con 0x7f49c0155de0 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c73fc640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b2780 con 0x7f49ac005180 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c5999640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 0x7f49c0156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c5999640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 0x7f49c0156880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49188/0 (socket says 172.21.15.106:49188) 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.334+0000 7f49c5999640 1 -- 172.21.15.106:0/2313631001 learned_addr learned my addr 172.21.15.106:0/2313631001 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c4997640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c0155de0 0x7f49c01483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5198640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac005180 0x7f49c01589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5999640 1 -- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac005180 msgr2=0x7f49c01589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5999640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac005180 0x7f49c01589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5999640 1 -- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c0155de0 msgr2=0x7f49c01483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5999640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c0155de0 0x7f49c01483d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5999640 1 -- 172.21.15.106:0/2313631001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00b2a60 con 0x7f49c01f33f0 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c5999640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 0x7f49c0156880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49b8001f70 tx=0x7f49b80014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8006d50 con 0x7f49c01f33f0 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b8007030 con 0x7f49c01f33f0 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b8007420 con 0x7f49c01f33f0 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.335+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00b2d40 con 0x7f49c01f33f0 2023-11-24T08:09:54.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.336+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c0150610 con 0x7f49c01f33f0 2023-11-24T08:09:54.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.337+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00b3260 con 0x7f49c01f33f0 2023-11-24T08:09:54.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.337+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f49b8007580 con 0x7f49c01f33f0 2023-11-24T08:09:54.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.338+0000 7f49b5ffb640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49a406ae70 0x7f49a406d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:54.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.338+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f49b80ab370 con 0x7f49c01f33f0 2023-11-24T08:09:54.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.339+0000 7f49c5198640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49a406ae70 0x7f49a406d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:54.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.339+0000 7f49c5198640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49a406ae70 0x7f49a406d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0098d0 tx=0x7f49bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:54.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.343+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49c00b3260 con 0x7f49c01f33f0 2023-11-24T08:09:54.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:54.537+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f49c0150920 con 0x7f49c01f33f0 2023-11-24T08:09:55.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.283+0000 7f49b5ffb640 1 -- 172.21.15.106:0/2313631001 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v33) v1 ==== 138+0+0 (secure 0 0 0) 0x7f49b8076760 con 0x7f49c01f33f0 2023-11-24T08:09:55.285 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49a406ae70 msgr2=0x7f49a406d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49a406ae70 0x7f49a406d330 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0098d0 tx=0x7f49bc008040 comp rx=0 tx=0).stop 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 msgr2=0x7f49c0156880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 0x7f49c0156880 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f49b8001f70 tx=0x7f49b80014a0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 shutdown_connections 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49c01f33f0 0x7f49c0156880 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49c0155de0 0x7f49c01483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49a406ae70 0x7f49a406d330 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 --2- 172.21.15.106:0/2313631001 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49ac005180 0x7f49c01589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 >> 172.21.15.106:0/2313631001 conn(0x7f49c006a920 msgr2=0x7f49c015cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 shutdown_connections 2023-11-24T08:09:55.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.287+0000 7f49c73fc640 1 -- 172.21.15.106:0/2313631001 wait complete. 2023-11-24T08:09:55.316 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:09:55.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.530+0000 7f825c89b640 1 Processor -- start 2023-11-24T08:09:55.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.531+0000 7f825c89b640 1 -- start start 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.532+0000 7f825c89b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8258146eb0 0x7f8258161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.532+0000 7f825c89b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f825806e160 0x7f82580ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.532+0000 7f825c89b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580cd110 0x7f825806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.532+0000 7f825c89b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82580cd580 con 0x7f825806e160 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.532+0000 7f825c89b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8258161a10 con 0x7f8258146eb0 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.532+0000 7f825c89b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8258161cf0 con 0x7f82580cd110 2023-11-24T08:09:55.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f825806e160 0x7f82580ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f825806e160 0x7f82580ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49198/0 (socket says 172.21.15.106:49198) 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 -- 172.21.15.106:0/2674329790 learned_addr learned my addr 172.21.15.106:0/2674329790 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f825659c640 1 --2- 172.21.15.106:0/2674329790 >> v2:172.21.15.156:3300/0 conn(0x7f8258146eb0 0x7f8258161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 -- 172.21.15.106:0/2674329790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580cd110 msgr2=0x7f825806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 --2- 172.21.15.106:0/2674329790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580cd110 0x7f825806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 -- 172.21.15.106:0/2674329790 >> v2:172.21.15.156:3300/0 conn(0x7f8258146eb0 msgr2=0x7f8258161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:55.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 --2- 172.21.15.106:0/2674329790 >> v2:172.21.15.156:3300/0 conn(0x7f8258146eb0 0x7f8258161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.533+0000 7f824ffff640 1 -- 172.21.15.106:0/2674329790 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258161fd0 con 0x7f825806e160 2023-11-24T08:09:55.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.534+0000 7f824ffff640 1 --2- 172.21.15.106:0/2674329790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825806e160 0x7f82580ccbd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f825806ea50 tx=0x7f824002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0151d04f3f75860 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:55.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f8255d9b640 1 -- 172.21.15.106:0/2674329790 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240031870 con 0x7f825806e160 2023-11-24T08:09:55.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f8255d9b640 1 -- 172.21.15.106:0/2674329790 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82400319d0 con 0x7f825806e160 2023-11-24T08:09:55.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f8255d9b640 1 -- 172.21.15.106:0/2674329790 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8240031d60 con 0x7f825806e160 2023-11-24T08:09:55.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 -- 172.21.15.106:0/2674329790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825806e160 msgr2=0x7f82580ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 --2- 172.21.15.106:0/2674329790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825806e160 0x7f82580ccbd0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f825806ea50 tx=0x7f824002b910 comp rx=0 tx=0).stop 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 -- 172.21.15.106:0/2674329790 shutdown_connections 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 --2- 172.21.15.106:0/2674329790 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580cd110 0x7f825806c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 --2- 172.21.15.106:0/2674329790 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825806e160 0x7f82580ccbd0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 --2- 172.21.15.106:0/2674329790 >> v2:172.21.15.156:3300/0 conn(0x7f8258146eb0 0x7f8258161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.535+0000 7f825c89b640 1 -- 172.21.15.106:0/2674329790 >> 172.21.15.106:0/2674329790 conn(0x7f82580b0c50 msgr2=0x7f82580b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.536+0000 7f825c89b640 1 -- 172.21.15.106:0/2674329790 shutdown_connections 2023-11-24T08:09:55.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.536+0000 7f825c89b640 1 -- 172.21.15.106:0/2674329790 wait complete. 2023-11-24T08:09:55.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.537+0000 7f825c89b640 1 Processor -- start 2023-11-24T08:09:55.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.537+0000 7f825c89b640 1 -- start start 2023-11-24T08:09:55.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.538+0000 7f825c89b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f825806e160 0x7f82580b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.538+0000 7f825c89b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82580cd110 0x7f825815f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.538+0000 7f825c89b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 0x7f825815f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.538+0000 7f825c89b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82580b1d10 con 0x7f8258146eb0 2023-11-24T08:09:55.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.538+0000 7f825c89b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f825815ff00 con 0x7f82580cd110 2023-11-24T08:09:55.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.538+0000 7f825c89b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82581601e0 con 0x7f825806e160 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 0x7f825815f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 0x7f825815f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49206/0 (socket says 172.21.15.106:49206) 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 -- 172.21.15.106:0/175733282 learned_addr learned my addr 172.21.15.106:0/175733282 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f825659c640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f825806e160 0x7f82580b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f824ffff640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82580cd110 0x7f825815f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 -- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f825806e160 msgr2=0x7f82580b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:55.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f825806e160 0x7f82580b3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 -- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82580cd110 msgr2=0x7f825815f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82580cd110 0x7f825815f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 -- 172.21.15.106:0/175733282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82400094d0 con 0x7f8258146eb0 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.539+0000 7f8256d9d640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 0x7f825815f5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f825000b970 tx=0x7f825000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.540+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825000cb40 con 0x7f8258146eb0 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.540+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825000ccc0 con 0x7f8258146eb0 2023-11-24T08:09:55.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.540+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250014680 con 0x7f8258146eb0 2023-11-24T08:09:55.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.540+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82581604c0 con 0x7f8258146eb0 2023-11-24T08:09:55.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.540+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f825815fb00 con 0x7f8258146eb0 2023-11-24T08:09:55.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.541+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82580b1e80 con 0x7f8258146eb0 2023-11-24T08:09:55.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.542+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8250007370 con 0x7f8258146eb0 2023-11-24T08:09:55.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.543+0000 7f824effd640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806ae40 0x7f823806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:55.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.543+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f8250089440 con 0x7f8258146eb0 2023-11-24T08:09:55.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.543+0000 7f825659c640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806ae40 0x7f823806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:55.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.544+0000 7f825659c640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806ae40 0x7f823806d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8248006fd0 tx=0x7f8248008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:55.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.548+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8250054630 con 0x7f8258146eb0 2023-11-24T08:09:55.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:55.738+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8258160650 con 0x7f8258146eb0 2023-11-24T08:09:56.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.297+0000 7f824effd640 1 -- 172.21.15.106:0/175733282 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v34) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8250057c50 con 0x7f8258146eb0 2023-11-24T08:09:56.299 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2023-11-24T08:09:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806ae40 msgr2=0x7f823806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806ae40 0x7f823806d300 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8248006fd0 tx=0x7f8248008040 comp rx=0 tx=0).stop 2023-11-24T08:09:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 msgr2=0x7f825815f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 0x7f825815f5c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f825000b970 tx=0x7f825000be30 comp rx=0 tx=0).stop 2023-11-24T08:09:56.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 shutdown_connections 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8258146eb0 0x7f825815f5c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82580cd110 0x7f825815f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806ae40 0x7f823806d300 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 --2- 172.21.15.106:0/175733282 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f825806e160 0x7f82580b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.301+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 >> 172.21.15.106:0/175733282 conn(0x7f82580b0c50 msgr2=0x7f82580b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.302+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 shutdown_connections 2023-11-24T08:09:56.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.302+0000 7f825c89b640 1 -- 172.21.15.106:0/175733282 wait complete. 2023-11-24T08:09:56.336 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:09:56.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.520+0000 7fbcc5d16640 1 Processor -- start 2023-11-24T08:09:56.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.520+0000 7fbcc5d16640 1 -- start start 2023-11-24T08:09:56.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.521+0000 7fbcc5d16640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbcc015b260 0x7fbcc006e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:56.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.521+0000 7fbcc5d16640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcc00b1250 0x7fbcc006c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:56.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.521+0000 7fbcc5d16640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc015ed20 con 0x7fbcc006e580 2023-11-24T08:09:56.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5d16640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc015b640 con 0x7fbcc00b1250 2023-11-24T08:09:56.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5d16640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc015f8a0 con 0x7fbcc015b260 2023-11-24T08:09:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcc00b1250 0x7fbcc006c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcc00b1250 0x7fbcc006c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49212/0 (socket says 172.21.15.106:49212) 2023-11-24T08:09:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 -- 172.21.15.106:0/962311737 learned_addr learned my addr 172.21.15.106:0/962311737 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc4d14640 1 --2- 172.21.15.106:0/962311737 >> v2:172.21.15.156:3300/0 conn(0x7fbcc015b260 0x7fbcc006e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:56.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 -- 172.21.15.106:0/962311737 >> v2:172.21.15.156:3300/0 conn(0x7fbcc015b260 msgr2=0x7fbcc006e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 --2- 172.21.15.106:0/962311737 >> v2:172.21.15.156:3300/0 conn(0x7fbcc015b260 0x7fbcc006e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 -- 172.21.15.106:0/962311737 >> v1:172.21.15.156:6789/0 conn(0x7fbcc006e580 legacy=0x7fbcc00b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:56.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.522+0000 7fbcc5515640 1 -- 172.21.15.106:0/962311737 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc00abe50 con 0x7fbcc00b1250 2023-11-24T08:09:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.524+0000 7fbcc5515640 1 --2- 172.21.15.106:0/962311737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc00b1250 0x7fbcc006c5b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00d0160 tx=0x7fbcb802bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d590d38ed9d7b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.524+0000 7fbcbf7fe640 1 -- 172.21.15.106:0/962311737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb802c960 con 0x7fbcc00b1250 2023-11-24T08:09:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.524+0000 7fbcbf7fe640 1 -- 172.21.15.106:0/962311737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb802cac0 con 0x7fbcc00b1250 2023-11-24T08:09:56.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.524+0000 7fbcbf7fe640 1 -- 172.21.15.106:0/962311737 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb802ce50 con 0x7fbcc00b1250 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.524+0000 7fbcc5d16640 1 -- 172.21.15.106:0/962311737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc00b1250 msgr2=0x7fbcc006c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.524+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/962311737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc00b1250 0x7fbcc006c5b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00d0160 tx=0x7fbcb802bdc0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.525+0000 7fbcc5d16640 1 -- 172.21.15.106:0/962311737 shutdown_connections 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.525+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/962311737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc00b1250 0x7fbcc006c5b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.525+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/962311737 >> v2:172.21.15.156:3300/0 conn(0x7fbcc015b260 0x7fbcc006e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.525+0000 7fbcc5d16640 1 -- 172.21.15.106:0/962311737 >> 172.21.15.106:0/962311737 conn(0x7fbcc00cbb40 msgr2=0x7fbcc00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.525+0000 7fbcc5d16640 1 -- 172.21.15.106:0/962311737 shutdown_connections 2023-11-24T08:09:56.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.525+0000 7fbcc5d16640 1 -- 172.21.15.106:0/962311737 wait complete. 2023-11-24T08:09:56.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.526+0000 7fbcc5d16640 1 Processor -- start 2023-11-24T08:09:56.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.527+0000 7fbcc5d16640 1 -- start start 2023-11-24T08:09:56.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.527+0000 7fbcc5d16640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc006e580 0x7fbcc015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:56.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5d16640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcc00b1250 0x7fbcc01f66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:56.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5d16640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 0x7fbcc01f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:56.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5d16640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015df10 con 0x7fbcc015b260 2023-11-24T08:09:56.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5d16640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015d290 con 0x7fbcc006e580 2023-11-24T08:09:56.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5d16640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc015d570 con 0x7fbcc00b1250 2023-11-24T08:09:56.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 0x7fbcc01f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:56.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 0x7fbcc01f6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49214/0 (socket says 172.21.15.106:49214) 2023-11-24T08:09:56.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 -- 172.21.15.106:0/1190382711 learned_addr learned my addr 172.21.15.106:0/1190382711 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc4d14640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc006e580 0x7fbcc015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcbffff640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcc00b1250 0x7fbcc01f66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 -- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcc00b1250 msgr2=0x7fbcc01f66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcc00b1250 0x7fbcc01f66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 -- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc006e580 msgr2=0x7fbcc015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc006e580 0x7fbcc015c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:56.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.528+0000 7fbcc5515640 1 -- 172.21.15.106:0/1190382711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8009b30 con 0x7fbcc015b260 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.529+0000 7fbcc5515640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 0x7fbcc01f6c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8036040 tx=0x7fbcb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.529+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8009510 con 0x7fbcc015b260 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.529+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb803e450 con 0x7fbcc015b260 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.529+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb803e800 con 0x7fbcc015b260 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.529+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc015d850 con 0x7fbcc015b260 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.529+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc015a6b0 con 0x7fbcc015b260 2023-11-24T08:09:56.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.530+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc015e050 con 0x7fbcc015b260 2023-11-24T08:09:56.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.531+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fbcb803e960 con 0x7fbcc015b260 2023-11-24T08:09:56.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.532+0000 7fbcbd7fa640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbca406ae40 0x7fbca406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:56.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.532+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbcb80ac7f0 con 0x7fbcc015b260 2023-11-24T08:09:56.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.533+0000 7fbcc4d14640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbca406ae40 0x7fbca406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:56.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.533+0000 7fbcc4d14640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbca406ae40 0x7fbca406d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4006fd0 tx=0x7fbcb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:56.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.536+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbcb8077a90 con 0x7fbcc015b260 2023-11-24T08:09:56.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:56.758+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbcc015e050 con 0x7fbcc015b260 2023-11-24T08:09:57.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.324+0000 7fbcbd7fa640 1 -- 172.21.15.106:0/1190382711 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v11) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbcc015e050 con 0x7fbcc015b260 2023-11-24T08:09:57.327 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 4 and data pool 5 2023-11-24T08:09:57.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbca406ae40 msgr2=0x7fbca406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbca406ae40 0x7fbca406d300 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4006fd0 tx=0x7fbcb4008040 comp rx=0 tx=0).stop 2023-11-24T08:09:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 msgr2=0x7fbcc01f6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 0x7fbcc01f6c10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fbcb8036040 tx=0x7fbcb8004a40 comp rx=0 tx=0).stop 2023-11-24T08:09:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 shutdown_connections 2023-11-24T08:09:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcc015b260 0x7fbcc01f6c10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbca406ae40 0x7fbca406d300 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcc00b1250 0x7fbcc01f66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 --2- 172.21.15.106:0/1190382711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc006e580 0x7fbcc015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.332+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 >> 172.21.15.106:0/1190382711 conn(0x7fbcc00cbb40 msgr2=0x7fbcc00b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.333+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 shutdown_connections 2023-11-24T08:09:57.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.333+0000 7fbcc5d16640 1 -- 172.21.15.106:0/1190382711 wait complete. 2023-11-24T08:09:57.367 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:57.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.537+0000 7f9e8a61c640 1 Processor -- start 2023-11-24T08:09:57.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.537+0000 7f9e8a61c640 1 -- start start 2023-11-24T08:09:57.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e8a61c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e840cfb60 0x7f9e84161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e8a61c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e840ce0b0 con 0x7f9e84157c70 2023-11-24T08:09:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e8a61c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e84161960 con 0x7f9e8415dc80 2023-11-24T08:09:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e8a61c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e8415e8d0 con 0x7f9e840cfb60 2023-11-24T08:09:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e837fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e840cfb60 0x7f9e84161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e837fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e840cfb60 0x7f9e84161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49224/0 (socket says 172.21.15.106:49224) 2023-11-24T08:09:57.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e837fe640 1 -- 172.21.15.106:0/1832089853 learned_addr learned my addr 172.21.15.106:0/1832089853 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e837fe640 1 -- 172.21.15.106:0/1832089853 >> v1:172.21.15.156:6789/0 conn(0x7f9e8415dc80 legacy=0x7f9e840b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e837fe640 1 -- 172.21.15.106:0/1832089853 >> v1:172.21.15.106:6789/0 conn(0x7f9e84157c70 legacy=0x7f9e840cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.539+0000 7f9e837fe640 1 -- 172.21.15.106:0/1832089853 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8415ebb0 con 0x7f9e840cfb60 2023-11-24T08:09:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.540+0000 7f9e837fe640 1 --2- 172.21.15.106:0/1832089853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e840cfb60 0x7f9e84161df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009870 tx=0x7f9e7802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f2fc2077454501 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e82ffd640 1 -- 172.21.15.106:0/1832089853 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78031860 con 0x7f9e840cfb60 2023-11-24T08:09:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e82ffd640 1 -- 172.21.15.106:0/1832089853 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e780319c0 con 0x7f9e840cfb60 2023-11-24T08:09:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e82ffd640 1 -- 172.21.15.106:0/1832089853 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78031d50 con 0x7f9e840cfb60 2023-11-24T08:09:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 -- 172.21.15.106:0/1832089853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e840cfb60 msgr2=0x7f9e84161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/1832089853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e840cfb60 0x7f9e84161df0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e78009870 tx=0x7f9e7802b900 comp rx=0 tx=0).stop 2023-11-24T08:09:57.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 -- 172.21.15.106:0/1832089853 shutdown_connections 2023-11-24T08:09:57.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/1832089853 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e840cfb60 0x7f9e84161df0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 -- 172.21.15.106:0/1832089853 >> 172.21.15.106:0/1832089853 conn(0x7f9e8406a920 msgr2=0x7f9e8406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:57.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 -- 172.21.15.106:0/1832089853 shutdown_connections 2023-11-24T08:09:57.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.541+0000 7f9e8a61c640 1 -- 172.21.15.106:0/1832089853 wait complete. 2023-11-24T08:09:57.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.543+0000 7f9e8a61c640 1 Processor -- start 2023-11-24T08:09:57.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.543+0000 7f9e8a61c640 1 -- start start 2023-11-24T08:09:57.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e8a61c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e840cfb60 0x7f9e8414d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e8a61c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e84157c70 0x7f9e8414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e8a61c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 0x7f9e841e9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e8a61c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8414f930 con 0x7f9e8415dc80 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e8a61c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841e9ae0 con 0x7f9e84157c70 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e8a61c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e841e9dc0 con 0x7f9e840cfb60 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e88bb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 0x7f9e841e9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e88bb9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 0x7f9e841e9020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49232/0 (socket says 172.21.15.106:49232) 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.544+0000 7f9e88bb9640 1 -- 172.21.15.106:0/236787652 learned_addr learned my addr 172.21.15.106:0/236787652 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e88bb9640 1 -- 172.21.15.106:0/236787652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e840cfb60 msgr2=0x7f9e8414d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e88bb9640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e840cfb60 0x7f9e8414d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e88bb9640 1 -- 172.21.15.106:0/236787652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e84157c70 msgr2=0x7f9e8414fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e88bb9640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e84157c70 0x7f9e8414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e88bb9640 1 -- 172.21.15.106:0/236787652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780094d0 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e88bb9640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 0x7f9e841e9020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c780 tx=0x7f9e7c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c003d80 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e7c00ce30 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c005290 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.545+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8414f270 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.546+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8414d6c0 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.547+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8414f0d0 con 0x7f9e8415dc80 2023-11-24T08:09:57.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.547+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9e7c0053f0 con 0x7f9e8415dc80 2023-11-24T08:09:57.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.548+0000 7f9e80ff9640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e7006ae40 0x7f9e7006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.548+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e7c088a50 con 0x7f9e8415dc80 2023-11-24T08:09:57.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.549+0000 7f9e83fff640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e7006ae40 0x7f9e7006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.549+0000 7f9e83fff640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e7006ae40 0x7f9e7006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000f60 tx=0x7f9e740079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:57.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.553+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e8414f0d0 con 0x7f9e8415dc80 2023-11-24T08:09:57.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.732+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e8414fa70 con 0x7f9e8415dc80 2023-11-24T08:09:57.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.734+0000 7f9e80ff9640 1 -- 172.21.15.106:0/236787652 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16630 (secure 0 0 0) 0x7f9e8414fa70 con 0x7f9e8415dc80 2023-11-24T08:09:57.736 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:57.736 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":35,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:56.760159+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e7006ae40 msgr2=0x7f9e7006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e7006ae40 0x7f9e7006d300 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f9e74000f60 tx=0x7f9e740079b0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 msgr2=0x7f9e841e9020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 0x7f9e841e9020 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c00c780 tx=0x7f9e7c00cc40 comp rx=0 tx=0).stop 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 shutdown_connections 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e8415dc80 0x7f9e841e9020 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e84157c70 0x7f9e8414fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e7006ae40 0x7f9e7006d300 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 --2- 172.21.15.106:0/236787652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e840cfb60 0x7f9e8414d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 >> 172.21.15.106:0/236787652 conn(0x7f9e8406a920 msgr2=0x7f9e840b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:57.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 shutdown_connections 2023-11-24T08:09:57.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.737+0000 7f9e8a61c640 1 -- 172.21.15.106:0/236787652 wait complete. 2023-11-24T08:09:57.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:09:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.957+0000 7fc16da56640 1 Processor -- start 2023-11-24T08:09:57.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.957+0000 7fc16da56640 1 -- start start 2023-11-24T08:09:57.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.958+0000 7fc16da56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc16806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.958+0000 7fc16da56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1680cf450 0x7fc1680cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.958+0000 7fc16da56640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc1680cdfb0 0x7fc1680ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.958+0000 7fc16da56640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc168156010 con 0x7fc1680cf450 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.958+0000 7fc16da56640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc16806ce30 con 0x7fc1680cdfb0 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.958+0000 7fc16da56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1680cfe10 con 0x7fc16815c7f0 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1680cf450 0x7fc1680cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc1680cf450 0x7fc1680cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49240/0 (socket says 172.21.15.106:49240) 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3419037029 learned_addr learned my addr 172.21.15.106:0/3419037029 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc166ffd640 1 --2- 172.21.15.106:0/3419037029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc16806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc1677fe640 1 --2- 172.21.15.106:0/3419037029 >> v2:172.21.15.156:3300/0 conn(0x7fc1680cdfb0 0x7fc1680ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3419037029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 msgr2=0x7fc16806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:57.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 --2- 172.21.15.106:0/3419037029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc16806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3419037029 >> v2:172.21.15.156:3300/0 conn(0x7fc1680cdfb0 msgr2=0x7fc1680ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:57.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 --2- 172.21.15.106:0/3419037029 >> v2:172.21.15.156:3300/0 conn(0x7fc1680cdfb0 0x7fc1680ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3419037029 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1680cece0 con 0x7fc1680cf450 2023-11-24T08:09:57.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.959+0000 7fc1677fe640 1 --2- 172.21.15.106:0/3419037029 >> v2:172.21.15.156:3300/0 conn(0x7fc1680cdfb0 0x7fc1680ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.960+0000 7fc15e7fc640 1 --2- 172.21.15.106:0/3419037029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc1680cf8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc1680aebc0 tx=0x7fc15002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90bba25bda546299 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.960+0000 7fc1667fc640 1 -- 172.21.15.106:0/3419037029 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150031870 con 0x7fc1680cf450 2023-11-24T08:09:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc1667fc640 1 -- 172.21.15.106:0/3419037029 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1500319d0 con 0x7fc1680cf450 2023-11-24T08:09:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc1667fc640 1 -- 172.21.15.106:0/3419037029 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150031d60 con 0x7fc1680cf450 2023-11-24T08:09:57.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 -- 172.21.15.106:0/3419037029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 msgr2=0x7fc1680cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 --2- 172.21.15.106:0/3419037029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc1680cf8d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc1680aebc0 tx=0x7fc15002b910 comp rx=0 tx=0).stop 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 -- 172.21.15.106:0/3419037029 shutdown_connections 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 --2- 172.21.15.106:0/3419037029 >> v2:172.21.15.156:3300/0 conn(0x7fc1680cdfb0 0x7fc1680ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 --2- 172.21.15.106:0/3419037029 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc1680cf8d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 --2- 172.21.15.106:0/3419037029 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc16806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 -- 172.21.15.106:0/3419037029 >> 172.21.15.106:0/3419037029 conn(0x7fc16806a920 msgr2=0x7fc16806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 -- 172.21.15.106:0/3419037029 shutdown_connections 2023-11-24T08:09:57.964 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.961+0000 7fc16da56640 1 -- 172.21.15.106:0/3419037029 wait complete. 2023-11-24T08:09:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.963+0000 7fc16da56640 1 Processor -- start 2023-11-24T08:09:57.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.963+0000 7fc16da56640 1 -- start start 2023-11-24T08:09:57.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc16da56640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1680cdfb0 0x7fc168099540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc16da56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc168146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.967 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc16da56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc1680ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc16da56640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168147b80 con 0x7fc1680cf450 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc16da56640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc168146cf0 con 0x7fc1680cdfb0 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc16da56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc168099a80 con 0x7fc16815c7f0 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc168146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc15e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc168146440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49252/0 (socket says 172.21.15.106:49252) 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3398684126 learned_addr learned my addr 172.21.15.106:0/3398684126 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3398684126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 msgr2=0x7fc1680ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc15e7fc640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc1680ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.964+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3398684126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1680cdfb0 msgr2=0x7fc168099540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc15e7fc640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1680cdfb0 0x7fc168099540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc15e7fc640 1 -- 172.21.15.106:0/3398684126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1500094d0 con 0x7fc1680cf450 2023-11-24T08:09:57.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc15e7fc640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc168146440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc150000c00 tx=0x7fc1500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:57.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc15003abe0 con 0x7fc1680cf450 2023-11-24T08:09:57.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc15003ad40 con 0x7fc1680cf450 2023-11-24T08:09:57.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc168146f00 con 0x7fc1680cf450 2023-11-24T08:09:57.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc150044610 con 0x7fc1680cf450 2023-11-24T08:09:57.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.965+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1681474c0 con 0x7fc1680cf450 2023-11-24T08:09:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.966+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1680b51b0 con 0x7fc1680cf450 2023-11-24T08:09:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.967+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fc150044770 con 0x7fc1680cf450 2023-11-24T08:09:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.968+0000 7fc164ff9640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc14806ae40 0x7fc14806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.968+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc15002d040 con 0x7fc1680cf450 2023-11-24T08:09:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.968+0000 7fc166ffd640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc14806ae40 0x7fc14806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:57.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.969+0000 7fc166ffd640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc14806ae40 0x7fc14806d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc158006fd0 tx=0x7fc158008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:57.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:57.972+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc150076e80 con 0x7fc1680cf450 2023-11-24T08:09:58.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.161+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc168067470 con 0x7fc1680cf450 2023-11-24T08:09:58.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.164+0000 7fc164ff9640 1 -- 172.21.15.106:0/3398684126 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16630 (secure 0 0 0) 0x7fc15007a4a0 con 0x7fc1680cf450 2023-11-24T08:09:58.166 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:09:58.166 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":35,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:56.760159+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:09:58.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.166+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc14806ae40 msgr2=0x7fc14806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.166+0000 7fc16da56640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc14806ae40 0x7fc14806d300 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fc158006fd0 tx=0x7fc158008040 comp rx=0 tx=0).stop 2023-11-24T08:09:58.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.166+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 msgr2=0x7fc168146440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.166+0000 7fc16da56640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc168146440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc150000c00 tx=0x7fc1500046c0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 shutdown_connections 2023-11-24T08:09:58.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc14806ae40 0x7fc14806d300 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc16815c7f0 0x7fc1680ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc1680cf450 0x7fc168146440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 --2- 172.21.15.106:0/3398684126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1680cdfb0 0x7fc168099540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 >> 172.21.15.106:0/3398684126 conn(0x7fc16806a920 msgr2=0x7fc168071de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:58.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 shutdown_connections 2023-11-24T08:09:58.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.167+0000 7fc16da56640 1 -- 172.21.15.106:0/3398684126 wait complete. 2023-11-24T08:09:58.188 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:09:58.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.399+0000 7fa75e231640 1 Processor -- start 2023-11-24T08:09:58.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.400+0000 7fa75e231640 1 -- start start 2023-11-24T08:09:58.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.401+0000 7fa75e231640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 0x7fa7580b3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.401+0000 7fa75e231640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7580b37b0 con 0x7fa75815d620 2023-11-24T08:09:58.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.401+0000 7fa75e231640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa75806f9b0 con 0x7fa7580b5430 2023-11-24T08:09:58.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.401+0000 7fa75e231640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa75806d900 con 0x7fa7580999d0 2023-11-24T08:09:58.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.401+0000 7fa7577fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa75815d620 0x7fa7580b4d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39326/0 (socket says 172.21.15.106:39326) 2023-11-24T08:09:58.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.401+0000 7fa7577fe640 1 -- 172.21.15.106:0/3129151617 learned_addr learned my addr 172.21.15.106:0/3129151617 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:09:58.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564883748 0 0) 0x7fa748001110 con 0x7fa75815d620 2023-11-24T08:09:58.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa738003520 con 0x7fa75815d620 2023-11-24T08:09:58.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa756ffd640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 0x7fa7580b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:58.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1499716126 0 0) 0x7fa738003520 con 0x7fa75815d620 2023-11-24T08:09:58.407 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 msgr2=0x7fa7580b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:58.407 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 0x7fa7580b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.407 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 >> v1:172.21.15.156:6789/0 conn(0x7fa7580b5430 legacy=0x7fa758167280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.407 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa75806dbe0 con 0x7fa75815d620 2023-11-24T08:09:58.407 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fa748001800 con 0x7fa75815d620 2023-11-24T08:09:58.408 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 >> v1:172.21.15.106:6789/0 conn(0x7fa75815d620 legacy=0x7fa7580b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.408 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa738003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7380047c0 0x7fa738004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 0x7fa738005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.402+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa75806d900 con 0x7fa7380039a0 2023-11-24T08:09:58.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73800ba90 con 0x7fa7380047c0 2023-11-24T08:09:58.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73800bc50 con 0x7fa7580999d0 2023-11-24T08:09:58.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7577fe640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa738003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:58.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa756ffd640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7380047c0 0x7fa738004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:58.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7577fe640 1 -- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 msgr2=0x7fa738005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:58.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7577fe640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7580999d0 0x7fa738005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7577fe640 1 -- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7380047c0 msgr2=0x7fa738004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:58.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7577fe640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7380047c0 0x7fa738004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.403+0000 7fa7577fe640 1 -- 172.21.15.106:0/3129151617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738003520 con 0x7fa7380039a0 2023-11-24T08:09:58.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.404+0000 7fa7577fe640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa738003dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa748001eb0 tx=0x7fa7380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d290dacf00b800 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:58.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.404+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748005ad0 con 0x7fa7380039a0 2023-11-24T08:09:58.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa748005c50 con 0x7fa7380039a0 2023-11-24T08:09:58.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa7567fc640 1 -- 172.21.15.106:0/3129151617 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74802c790 con 0x7fa7380039a0 2023-11-24T08:09:58.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa75e231640 1 -- 172.21.15.106:0/3129151617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 msgr2=0x7fa738003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa75e231640 1 --2- 172.21.15.106:0/3129151617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa738003dc0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa748001eb0 tx=0x7fa7380057f0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa757fff640 1 -- 172.21.15.106:0/3129151617 reap_dead start 2023-11-24T08:09:58.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa75e231640 1 -- 172.21.15.106:0/3129151617 shutdown_connections 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa75e231640 1 -- 172.21.15.106:0/3129151617 >> 172.21.15.106:0/3129151617 conn(0x7fa75806c090 msgr2=0x7fa7580b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa75e231640 1 -- 172.21.15.106:0/3129151617 shutdown_connections 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.405+0000 7fa75e231640 1 -- 172.21.15.106:0/3129151617 wait complete. 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.407+0000 7fa75e231640 1 Processor -- start 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.407+0000 7fa75e231640 1 -- start start 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa75e231640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa75815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa75e231640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa758160040 0x7fa758144920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa75e231640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7581604c0 0x7fa758144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa75e231640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7581453a0 con 0x7fa7380039a0 2023-11-24T08:09:58.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa75e231640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa758159a40 con 0x7fa758160040 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa75e231640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa758159c90 con 0x7fa7581604c0 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa7577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa75815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa7577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa75815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49274/0 (socket says 172.21.15.106:49274) 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.408+0000 7fa7577fe640 1 -- 172.21.15.106:0/2538153237 learned_addr learned my addr 172.21.15.106:0/2538153237 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa7577fe640 1 -- 172.21.15.106:0/2538153237 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7581604c0 msgr2=0x7fa758144e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa7577fe640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7581604c0 0x7fa758144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa7577fe640 1 -- 172.21.15.106:0/2538153237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa758160040 msgr2=0x7fa758144920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa7577fe640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa758160040 0x7fa758144920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa7577fe640 1 -- 172.21.15.106:0/2538153237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa758159f70 con 0x7fa7380039a0 2023-11-24T08:09:58.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa7577fe640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa75815aa50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa74802c330 tx=0x7fa748001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa748039070 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa748000f10 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa74803c040 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.409+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa75815a250 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.410+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa75815af90 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.410+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa75804e8b0 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.412+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fa748032070 con 0x7fa7380039a0 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.412+0000 7fa74ffff640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73c06ae70 0x7fa73c06d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:58.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.412+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7480aa7e0 con 0x7fa7380039a0 2023-11-24T08:09:58.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.413+0000 7fa756ffd640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73c06ae70 0x7fa73c06d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:58.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.413+0000 7fa756ffd640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73c06ae70 0x7fa73c06d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa758160d10 tx=0x7fa740009750 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:58.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.417+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa748075890 con 0x7fa7380039a0 2023-11-24T08:09:58.537 INFO:tasks.ceph.mds.c.smithi106.stderr:2023-11-24T08:09:58.535+0000 7f420ede4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:09:58.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:58.646+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa7580ca1a0 con 0x7fa7380039a0 2023-11-24T08:09:59.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.335+0000 7fa74ffff640 1 -- 172.21.15.106:0/2538153237 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v13) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa7480493a0 con 0x7fa7380039a0 2023-11-24T08:09:59.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73c06ae70 msgr2=0x7fa73c06d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:59.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73c06ae70 0x7fa73c06d330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa758160d10 tx=0x7fa740009750 comp rx=0 tx=0).stop 2023-11-24T08:09:59.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 msgr2=0x7fa75815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:59.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa75815aa50 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa74802c330 tx=0x7fa748001ac0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 shutdown_connections 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73c06ae70 0x7fa73c06d330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa7581604c0 0x7fa758144e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa758160040 0x7fa758144920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 --2- 172.21.15.106:0/2538153237 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa7380039a0 0x7fa75815aa50 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.338+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 >> 172.21.15.106:0/2538153237 conn(0x7fa75806c090 msgr2=0x7fa7580c95f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.339+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 shutdown_connections 2023-11-24T08:09:59.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.339+0000 7fa75e231640 1 -- 172.21.15.106:0/2538153237 wait complete. 2023-11-24T08:09:59.360 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:09:59.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:09:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.575+0000 7f2a98e56640 1 Processor -- start 2023-11-24T08:09:59.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.575+0000 7f2a98e56640 1 -- start start 2023-11-24T08:09:59.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.577+0000 7f2a98e56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a94162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.577+0000 7f2a98e56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a94162540 0x7f2a940c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.577+0000 7f2a98e56640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a94166510 con 0x7f2a940ccbd0 2023-11-24T08:09:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.577+0000 7f2a98e56640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a941655f0 con 0x7f2a94162540 2023-11-24T08:09:59.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.577+0000 7f2a98e56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940cc380 con 0x7f2a941660b0 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.577+0000 7f2a92d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a94162540 0x7f2a940c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a92d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2a94162540 0x7f2a940c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49282/0 (socket says 172.21.15.106:49282) 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a92d9d640 1 -- 172.21.15.106:0/402184206 learned_addr learned my addr 172.21.15.106:0/402184206 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a89d9b640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a94162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a92d9d640 1 -- 172.21.15.106:0/402184206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 msgr2=0x7f2a94162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a92d9d640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a94162000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a92d9d640 1 -- 172.21.15.106:0/402184206 >> v1:172.21.15.106:6789/0 conn(0x7f2a940ccbd0 legacy=0x7f2a940ccfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a92d9d640 1 -- 172.21.15.106:0/402184206 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94162b30 con 0x7f2a94162540 2023-11-24T08:09:59.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.578+0000 7f2a89d9b640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a94162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.579+0000 7f2a92d9d640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a940c9650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57748aa3eadac3d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.579+0000 7f2a91d9b640 1 -- 172.21.15.106:0/402184206 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c030840 con 0x7f2a94162540 2023-11-24T08:09:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.579+0000 7f2a91d9b640 1 -- 172.21.15.106:0/402184206 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c0309a0 con 0x7f2a94162540 2023-11-24T08:09:59.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.579+0000 7f2a91d9b640 1 -- 172.21.15.106:0/402184206 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c030d30 con 0x7f2a94162540 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 -- 172.21.15.106:0/402184206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 msgr2=0x7f2a940c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a940c9650 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f2a8c0098a0 tx=0x7f2a8c02b980 comp rx=0 tx=0).stop 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 -- 172.21.15.106:0/402184206 shutdown_connections 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a940c9650 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 --2- 172.21.15.106:0/402184206 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a94162000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 -- 172.21.15.106:0/402184206 >> 172.21.15.106:0/402184206 conn(0x7f2a940b0c50 msgr2=0x7f2a940b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:09:59.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 -- 172.21.15.106:0/402184206 shutdown_connections 2023-11-24T08:09:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.580+0000 7f2a98e56640 1 -- 172.21.15.106:0/402184206 wait complete. 2023-11-24T08:09:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.582+0000 7f2a98e56640 1 Processor -- start 2023-11-24T08:09:59.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.582+0000 7f2a98e56640 1 -- start start 2023-11-24T08:09:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a98e56640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a940ccbd0 0x7f2a9415f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:59.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a98e56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a941615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a98e56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a941636a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a98e56640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94154740 con 0x7f2a94162540 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a98e56640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94142770 con 0x7f2a940ccbd0 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a98e56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94160010 con 0x7f2a941660b0 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a89d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a941615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a89d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a941615e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49298/0 (socket says 172.21.15.106:49298) 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.583+0000 7f2a89d9b640 1 -- 172.21.15.106:0/3722693899 learned_addr learned my addr 172.21.15.106:0/3722693899 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a92d9d640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a941636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a89d9b640 1 -- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 msgr2=0x7f2a941636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a89d9b640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a941636a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a9259c640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a940ccbd0 0x7f2a9415f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a89d9b640 1 -- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a940ccbd0 msgr2=0x7f2a9415f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:09:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a89d9b640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a940ccbd0 0x7f2a9415f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:09:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a89d9b640 1 -- 172.21.15.106:0/3722693899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a8c0094d0 con 0x7f2a94162540 2023-11-24T08:09:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a92d9d640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a941636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:59.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.584+0000 7f2a9259c640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a940ccbd0 0x7f2a9415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.585+0000 7f2a89d9b640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a941615e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c7f0 tx=0x7f2a7c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.585+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007820 con 0x7f2a94162540 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.585+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c007980 con 0x7f2a94162540 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.585+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007cf0 con 0x7f2a94162540 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.585+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a941602f0 con 0x7f2a94162540 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.585+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a941608b0 con 0x7f2a94162540 2023-11-24T08:09:59.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.586+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9404e8b0 con 0x7f2a94162540 2023-11-24T08:09:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.587+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2a7c0113f0 con 0x7f2a94162540 2023-11-24T08:09:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.588+0000 7f2a8b7fe640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a7406ae40 0x7f2a7406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:09:59.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.588+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a7c089a50 con 0x7f2a94162540 2023-11-24T08:09:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.588+0000 7f2a9259c640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a7406ae40 0x7f2a7406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:09:59.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.589+0000 7f2a9259c640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a7406ae40 0x7f2a7406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a840012e0 tx=0x7f2a84009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:09:59.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.593+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a7c0549d0 con 0x7f2a94162540 2023-11-24T08:09:59.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:09:59.821+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2a9412c1b0 con 0x7f2a94162540 2023-11-24T08:10:00.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.338+0000 7f2a8b7fe640 1 -- 172.21.15.106:0/3722693899 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2a7c01f020 con 0x7f2a94162540 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.342+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a7406ae40 msgr2=0x7f2a7406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a7406ae40 0x7f2a7406d300 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a840012e0 tx=0x7f2a84009040 comp rx=0 tx=0).stop 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 msgr2=0x7f2a941615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a941615e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c7f0 tx=0x7f2a7c00ccb0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 shutdown_connections 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a7406ae40 0x7f2a7406d300 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a941660b0 0x7f2a941636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a94162540 0x7f2a941615e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 --2- 172.21.15.106:0/3722693899 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a940ccbd0 0x7f2a9415f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 >> 172.21.15.106:0/3722693899 conn(0x7f2a940b0c50 msgr2=0x7f2a94157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:00.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 shutdown_connections 2023-11-24T08:10:00.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.343+0000 7f2a98e56640 1 -- 172.21.15.106:0/3722693899 wait complete. 2023-11-24T08:10:00.365 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:10:00.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:00.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.572+0000 7f0fe637c640 1 Processor -- start 2023-11-24T08:10:00.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.572+0000 7f0fe637c640 1 -- start start 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.573+0000 7f0fe637c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0fe00b2250 0x7f0fe015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.573+0000 7f0fe637c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0fe00b2aa0 0x7f0fe015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.573+0000 7f0fe637c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fe015ab80 con 0x7f0fe00b2630 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.573+0000 7f0fe637c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe015a990 con 0x7f0fe00b2250 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.573+0000 7f0fe637c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fe015e680 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0fe00b2250 0x7f0fe015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0fe00b2250 0x7f0fe015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49300/0 (socket says 172.21.15.106:49300) 2023-11-24T08:10:00.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 -- 172.21.15.106:0/3884271255 learned_addr learned my addr 172.21.15.106:0/3884271255 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:00.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdf7fe640 1 --1- 172.21.15.106:0/3884271255 >> v1:172.21.15.106:6789/0 conn(0x7f0fe00b2630 0x7f0fe00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39344/0 (socket says 172.21.15.106:39344) 2023-11-24T08:10:00.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fe4919640 1 --2- 172.21.15.106:0/3884271255 >> v2:172.21.15.156:3300/0 conn(0x7f0fe00b2aa0 0x7f0fe015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:00.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 -- 172.21.15.106:0/3884271255 >> v2:172.21.15.156:3300/0 conn(0x7f0fe00b2aa0 msgr2=0x7f0fe015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:00.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 --2- 172.21.15.106:0/3884271255 >> v2:172.21.15.156:3300/0 conn(0x7f0fe00b2aa0 0x7f0fe015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 -- 172.21.15.106:0/3884271255 >> v1:172.21.15.106:6789/0 conn(0x7f0fe00b2630 legacy=0x7f0fe00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fdffff640 1 -- 172.21.15.106:0/3884271255 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe015e960 con 0x7f0fe00b2250 2023-11-24T08:10:00.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.574+0000 7f0fe4919640 1 --2- 172.21.15.106:0/3884271255 >> v2:172.21.15.156:3300/0 conn(0x7f0fe00b2aa0 0x7f0fe015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.575+0000 7f0fdffff640 1 --2- 172.21.15.106:0/3884271255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2250 0x7f0fe015df80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00b2fa0 tx=0x7f0fd002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f4d9d99c415be47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fdeffd640 1 -- 172.21.15.106:0/3884271255 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0030860 con 0x7f0fe00b2250 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fdeffd640 1 -- 172.21.15.106:0/3884271255 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd00309c0 con 0x7f0fe00b2250 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fdeffd640 1 -- 172.21.15.106:0/3884271255 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd0030d50 con 0x7f0fe00b2250 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 -- 172.21.15.106:0/3884271255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2250 msgr2=0x7f0fe015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 --2- 172.21.15.106:0/3884271255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2250 0x7f0fe015df80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f0fe00b2fa0 tx=0x7f0fd002b9a0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 -- 172.21.15.106:0/3884271255 shutdown_connections 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 --2- 172.21.15.106:0/3884271255 >> v2:172.21.15.156:3300/0 conn(0x7f0fe00b2aa0 0x7f0fe015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 --2- 172.21.15.106:0/3884271255 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2250 0x7f0fe015df80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 -- 172.21.15.106:0/3884271255 >> 172.21.15.106:0/3884271255 conn(0x7f0fe006a920 msgr2=0x7f0fe006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:00.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 -- 172.21.15.106:0/3884271255 shutdown_connections 2023-11-24T08:10:00.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.576+0000 7f0fe637c640 1 -- 172.21.15.106:0/3884271255 wait complete. 2023-11-24T08:10:00.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.578+0000 7f0fe637c640 1 Processor -- start 2023-11-24T08:10:00.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.578+0000 7f0fe637c640 1 -- start start 2023-11-24T08:10:00.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.579+0000 7f0fe637c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe00b2250 0x7f0fe00afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:00.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.579+0000 7f0fe637c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fe00b2630 0x7f0fe014ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.579+0000 7f0fe637c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 0x7f0fe0154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.579+0000 7f0fe637c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe015ccc0 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.579+0000 7f0fe637c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe0168d10 con 0x7f0fe00b2250 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.579+0000 7f0fe637c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fe014b350 con 0x7f0fe00b2630 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 0x7f0fe0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 0x7f0fe0154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49302/0 (socket says 172.21.15.106:49302) 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 -- 172.21.15.106:0/1226487720 learned_addr learned my addr 172.21.15.106:0/1226487720 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fdffff640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe00b2250 0x7f0fe00afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fdf7fe640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fe00b2630 0x7f0fe014ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 -- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fe00b2630 msgr2=0x7f0fe014ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fe00b2630 0x7f0fe014ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 -- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe00b2250 msgr2=0x7f0fe00afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe00b2250 0x7f0fe00afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 -- 172.21.15.106:0/1226487720 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fd00094d0 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.580+0000 7f0fe4919640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 0x7f0fe0154070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800cca0 tx=0x7f0fd800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.581+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800f680 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.581+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fd800f800 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.581+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fd800fbb0 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.581+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fe00cd7c0 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.581+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fe01683e0 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.582+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fe006c8c0 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.583+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f0fd800b040 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.584+0000 7f0fdcff9640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0fcc06ae40 0x7f0fcc06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:00.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.584+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fd808a040 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.585+0000 7f0fdffff640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0fcc06ae40 0x7f0fcc06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:00.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.585+0000 7f0fdffff640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0fcc06ae40 0x7f0fcc06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0036000 tx=0x7f0fd0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:00.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.589+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0fd80bce90 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.819+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0fe00cd600 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.820+0000 7f0fdcff9640 1 -- 172.21.15.106:0/1226487720 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f0fd801d090 con 0x7f0fe00b2aa0 2023-11-24T08:10:00.822 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:00.822 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:09:56.761029+0000","modified":"2023-11-24T08:09:59.823336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6836/3854927347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3854927347},{"type":"v1","addr":"172.21.15.106:6836","nonce":3854927347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:10:00.822 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 14 2023-11-24T08:10:00.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0fcc06ae40 msgr2=0x7f0fcc06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0fcc06ae40 0x7f0fcc06d300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0fd0036000 tx=0x7f0fd0008030 comp rx=0 tx=0).stop 2023-11-24T08:10:00.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 msgr2=0x7f0fe0154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:00.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 0x7f0fe0154070 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0fd800cca0 tx=0x7f0fd800a650 comp rx=0 tx=0).stop 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 shutdown_connections 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0fe00b2aa0 0x7f0fe0154070 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0fcc06ae40 0x7f0fcc06d300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0fe00b2630 0x7f0fe014ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 --2- 172.21.15.106:0/1226487720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0fe00b2250 0x7f0fe00afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 >> 172.21.15.106:0/1226487720 conn(0x7f0fe006a920 msgr2=0x7f0fe015b980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.823+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 shutdown_connections 2023-11-24T08:10:00.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:00.824+0000 7f0fe637c640 1 -- 172.21.15.106:0/1226487720 wait complete. 2023-11-24T08:10:00.846 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:01.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.054+0000 7f6bf0ba7640 1 Processor -- start 2023-11-24T08:10:01.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.054+0000 7f6bf0ba7640 1 -- start start 2023-11-24T08:10:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bf0ba7640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6bec144b30 0x7f6bec15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bf0ba7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bec0b1de0 0x7f6bec0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bf0ba7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec15bf80 con 0x7f6bec15c410 2023-11-24T08:10:01.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bf0ba7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec0b27b0 con 0x7f6bec0b1de0 2023-11-24T08:10:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bf0ba7640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec0b1560 con 0x7f6bec144b30 2023-11-24T08:10:01.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bebfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bec0b1de0 0x7f6bec0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bebfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6bec0b1de0 0x7f6bec0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49306/0 (socket says 172.21.15.106:49306) 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6bebfff640 1 -- 172.21.15.106:0/2503021054 learned_addr learned my addr 172.21.15.106:0/2503021054 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.056+0000 7f6beb7fe640 1 --2- 172.21.15.106:0/2503021054 >> v2:172.21.15.156:3300/0 conn(0x7f6bec144b30 0x7f6bec15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.057+0000 7f6bebfff640 1 -- 172.21.15.106:0/2503021054 >> v2:172.21.15.156:3300/0 conn(0x7f6bec144b30 msgr2=0x7f6bec15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.057+0000 7f6bebfff640 1 --2- 172.21.15.106:0/2503021054 >> v2:172.21.15.156:3300/0 conn(0x7f6bec144b30 0x7f6bec15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.057+0000 7f6bebfff640 1 -- 172.21.15.106:0/2503021054 >> v1:172.21.15.106:6789/0 conn(0x7f6bec15c410 legacy=0x7f6bec0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.057+0000 7f6bebfff640 1 -- 172.21.15.106:0/2503021054 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0b1280 con 0x7f6bec0b1de0 2023-11-24T08:10:01.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.057+0000 7f6bea7fc640 1 -- 172.21.15.106:0/2503021054 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1279402750 0 0) 0x7f6bec15bf80 con 0x7f6bec15c410 2023-11-24T08:10:01.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.058+0000 7f6bebfff640 1 --2- 172.21.15.106:0/2503021054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec0b1de0 0x7f6bec0b2270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bec06f690 tx=0x7f6be402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8639dbb268282e53 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.058+0000 7f6bea7fc640 1 -- 172.21.15.106:0/2503021054 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be402cb20 con 0x7f6bec0b1de0 2023-11-24T08:10:01.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.058+0000 7f6bea7fc640 1 -- 172.21.15.106:0/2503021054 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be402cc80 con 0x7f6bec0b1de0 2023-11-24T08:10:01.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.058+0000 7f6bea7fc640 1 -- 172.21.15.106:0/2503021054 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be40343f0 con 0x7f6bec0b1de0 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/2503021054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec0b1de0 msgr2=0x7f6bec0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/2503021054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec0b1de0 0x7f6bec0b2270 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f6bec06f690 tx=0x7f6be402bf80 comp rx=0 tx=0).stop 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/2503021054 shutdown_connections 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/2503021054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec0b1de0 0x7f6bec0b2270 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/2503021054 >> v2:172.21.15.156:3300/0 conn(0x7f6bec144b30 0x7f6bec15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/2503021054 >> 172.21.15.106:0/2503021054 conn(0x7f6bec0cb830 msgr2=0x7f6bec0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/2503021054 shutdown_connections 2023-11-24T08:10:01.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.059+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/2503021054 wait complete. 2023-11-24T08:10:01.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.060+0000 7f6bf0ba7640 1 Processor -- start 2023-11-24T08:10:01.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.061+0000 7f6bf0ba7640 1 -- start start 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.061+0000 7f6bf0ba7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bec0b1de0 0x7f6bec15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.061+0000 7f6bf0ba7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 0x7f6bec15a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6bf0ba7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bec15c410 0x7f6bec0c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6bf0ba7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0ca920 con 0x7f6bec144b30 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6bf0ba7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec133c80 con 0x7f6bec0b1de0 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6bf0ba7640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec15af00 con 0x7f6bec15c410 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 0x7f6bec15a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 0x7f6bec15a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49320/0 (socket says 172.21.15.106:49320) 2023-11-24T08:10:01.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 -- 172.21.15.106:0/3081616991 learned_addr learned my addr 172.21.15.106:0/3081616991 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beb7fe640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bec0b1de0 0x7f6bec15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 -- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bec15c410 msgr2=0x7f6bec0c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6bebfff640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bec15c410 0x7f6bec0c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bec15c410 0x7f6bec0c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 -- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bec0b1de0 msgr2=0x7f6bec15e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bec0b1de0 0x7f6bec15e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beaffd640 1 -- 172.21.15.106:0/3081616991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6be4009b30 con 0x7f6bec144b30 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.062+0000 7f6beb7fe640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bec0b1de0 0x7f6bec15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:01.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.063+0000 7f6beaffd640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 0x7f6bec15a9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6be000c410 tx=0x7f6be000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.063+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be00035f0 con 0x7f6bec144b30 2023-11-24T08:10:01.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.063+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be0003770 con 0x7f6bec144b30 2023-11-24T08:10:01.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.063+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec0cad40 con 0x7f6bec144b30 2023-11-24T08:10:01.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.063+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0003b20 con 0x7f6bec144b30 2023-11-24T08:10:01.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.063+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec0c9f80 con 0x7f6bec144b30 2023-11-24T08:10:01.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.064+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec04e840 con 0x7f6bec144b30 2023-11-24T08:10:01.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.065+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f6be0003c80 con 0x7f6bec144b30 2023-11-24T08:10:01.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.066+0000 7f6bd3fff640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bd806ae40 0x7f6bd806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.066+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6be0088ba0 con 0x7f6bec144b30 2023-11-24T08:10:01.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.066+0000 7f6beb7fe640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bd806ae40 0x7f6bd806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.067+0000 7f6beb7fe640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bd806ae40 0x7f6bd806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006fd0 tx=0x7f6bdc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.071+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6be0053ce0 con 0x7f6bec144b30 2023-11-24T08:10:01.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.258+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bec1442e0 con 0x7f6bec144b30 2023-11-24T08:10:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.261+0000 7f6bd3fff640 1 -- 172.21.15.106:0/3081616991 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f6be0017020 con 0x7f6bec144b30 2023-11-24T08:10:01.264 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:01.264 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:58.320664+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bd806ae40 msgr2=0x7f6bd806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bd806ae40 0x7f6bd806d300 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6bdc006fd0 tx=0x7f6bdc008040 comp rx=0 tx=0).stop 2023-11-24T08:10:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 msgr2=0x7f6bec15a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 0x7f6bec15a9c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6be000c410 tx=0x7f6be000c8d0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 shutdown_connections 2023-11-24T08:10:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bd806ae40 0x7f6bd806d300 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6bec15c410 0x7f6bec0c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6bec144b30 0x7f6bec15a9c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 --2- 172.21.15.106:0/3081616991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bec0b1de0 0x7f6bec15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 >> 172.21.15.106:0/3081616991 conn(0x7f6bec0cb830 msgr2=0x7f6bec15dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:01.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 shutdown_connections 2023-11-24T08:10:01.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.264+0000 7f6bf0ba7640 1 -- 172.21.15.106:0/3081616991 wait complete. 2023-11-24T08:10:01.283 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:10:01.283 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.480+0000 7f4202941640 1 Processor -- start 2023-11-24T08:10:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.480+0000 7f4202941640 1 -- start start 2023-11-24T08:10:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.481+0000 7f4202941640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f41fc157c70 0x7f41fc161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f4202941640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc0ce0b0 con 0x7f41fc0cf450 2023-11-24T08:10:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f4202941640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41fc161960 con 0x7f41fc15e390 2023-11-24T08:10:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f4202941640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41fc15e8d0 con 0x7f41fc157c70 2023-11-24T08:10:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f41fc157c70 0x7f41fc161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f41fc157c70 0x7f41fc161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49334/0 (socket says 172.21.15.106:49334) 2023-11-24T08:10:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fbfff640 1 -- 172.21.15.106:0/1855566344 learned_addr learned my addr 172.21.15.106:0/1855566344 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fb7fe640 1 --1- 172.21.15.106:0/1855566344 >> v1:172.21.15.106:6789/0 conn(0x7f41fc0cf450 0x7f41fc15dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39364/0 (socket says 172.21.15.106:39364) 2023-11-24T08:10:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fbfff640 1 -- 172.21.15.106:0/1855566344 >> v1:172.21.15.156:6789/0 conn(0x7f41fc15e390 legacy=0x7f41fc0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fbfff640 1 -- 172.21.15.106:0/1855566344 >> v1:172.21.15.106:6789/0 conn(0x7f41fc0cf450 legacy=0x7f41fc15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41fbfff640 1 -- 172.21.15.106:0/1855566344 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc15ebb0 con 0x7f41fc157c70 2023-11-24T08:10:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.482+0000 7f41faffd640 1 -- 172.21.15.106:0/1855566344 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1901090202 0 0) 0x7f41fc0ce0b0 con 0x7f41fc0cf450 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.483+0000 7f41fbfff640 1 --2- 172.21.15.106:0/1855566344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc157c70 0x7f41fc161df0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009870 tx=0x7f41ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced7c8123601783 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f41faffd640 1 -- 172.21.15.106:0/1855566344 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec031870 con 0x7f41fc157c70 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f41faffd640 1 -- 172.21.15.106:0/1855566344 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec0319d0 con 0x7f41fc157c70 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f41faffd640 1 -- 172.21.15.106:0/1855566344 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec031d60 con 0x7f41fc157c70 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f4202941640 1 -- 172.21.15.106:0/1855566344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc157c70 msgr2=0x7f41fc161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f4202941640 1 --2- 172.21.15.106:0/1855566344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc157c70 0x7f41fc161df0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009870 tx=0x7f41ec02b910 comp rx=0 tx=0).stop 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f4202941640 1 -- 172.21.15.106:0/1855566344 shutdown_connections 2023-11-24T08:10:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f4202941640 1 --2- 172.21.15.106:0/1855566344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc157c70 0x7f41fc161df0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f4202941640 1 -- 172.21.15.106:0/1855566344 >> 172.21.15.106:0/1855566344 conn(0x7f41fc06a920 msgr2=0x7f41fc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:01.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.484+0000 7f4202941640 1 -- 172.21.15.106:0/1855566344 shutdown_connections 2023-11-24T08:10:01.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.485+0000 7f4202941640 1 -- 172.21.15.106:0/1855566344 wait complete. 2023-11-24T08:10:01.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.486+0000 7f4202941640 1 Processor -- start 2023-11-24T08:10:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.486+0000 7f4202941640 1 -- start start 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.487+0000 7f4202941640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 0x7f41fc09a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.487+0000 7f4202941640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41fc157c70 0x7f41fc14cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.487+0000 7f4202941640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc15e390 0x7f41fc14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.487+0000 7f4202941640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc14d4a0 con 0x7f41fc0cf450 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.487+0000 7f4202941640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc14d700 con 0x7f41fc15e390 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.487+0000 7f4202941640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41fc14d9e0 con 0x7f41fc157c70 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 0x7f41fc09a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 0x7f41fc09a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49346/0 (socket says 172.21.15.106:49346) 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 -- 172.21.15.106:0/3508785026 learned_addr learned my addr 172.21.15.106:0/3508785026 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fb7fe640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41fc157c70 0x7f41fc14cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f4200ede640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc15e390 0x7f41fc14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 -- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41fc157c70 msgr2=0x7f41fc14cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41fc157c70 0x7f41fc14cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 -- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc15e390 msgr2=0x7f41fc14c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc15e390 0x7f41fc14c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f41fbfff640 1 -- 172.21.15.106:0/3508785026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ec0094d0 con 0x7f41fc0cf450 2023-11-24T08:10:01.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.488+0000 7f4200ede640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc15e390 0x7f41fc14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:10:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.489+0000 7f41fbfff640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 0x7f41fc09a380 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009870 tx=0x7f41ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.489+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec031820 con 0x7f41fc0cf450 2023-11-24T08:10:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.489+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ec0443f0 con 0x7f41fc0cf450 2023-11-24T08:10:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.489+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ec0447a0 con 0x7f41fc0cf450 2023-11-24T08:10:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.489+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41fc14dcc0 con 0x7f41fc0cf450 2023-11-24T08:10:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.489+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41fc14e280 con 0x7f41fc0cf450 2023-11-24T08:10:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.490+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41fc067470 con 0x7f41fc0cf450 2023-11-24T08:10:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.491+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f41ec04d450 con 0x7f41fc0cf450 2023-11-24T08:10:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.492+0000 7f41f8ff9640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f41e806ae40 0x7f41e806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.492+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41ec02d040 con 0x7f41fc0cf450 2023-11-24T08:10:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.492+0000 7f41fb7fe640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f41e806ae40 0x7f41e806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.493+0000 7f41fb7fe640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f41e806ae40 0x7f41e806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0ae4e0 tx=0x7f41f0007c00 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.496+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f41ec076e70 con 0x7f41fc0cf450 2023-11-24T08:10:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.670+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41fc14db50 con 0x7f41e806ae40 2023-11-24T08:10:01.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.691+0000 7f41f8ff9640 1 -- 172.21.15.106:0/3508785026 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286816 (secure 0 0 0) 0x7f41fc14db50 con 0x7f41e806ae40 2023-11-24T08:10:01.697 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:01.698 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.697+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f41e806ae40 msgr2=0x7f41e806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.697+0000 7f4202941640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f41e806ae40 0x7f41e806d300 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f41fc0ae4e0 tx=0x7f41f0007c00 comp rx=0 tx=0).stop 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.697+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 msgr2=0x7f41fc09a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 0x7f41fc09a380 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009870 tx=0x7f41ec0046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 shutdown_connections 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41fc15e390 0x7f41fc14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f41e806ae40 0x7f41e806d300 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41fc157c70 0x7f41fc14cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 --2- 172.21.15.106:0/3508785026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f41fc0cf450 0x7f41fc09a380 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 >> 172.21.15.106:0/3508785026 conn(0x7f41fc06a920 msgr2=0x7f41fc0b0150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:01.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 shutdown_connections 2023-11-24T08:10:01.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.698+0000 7f4202941640 1 -- 172.21.15.106:0/3508785026 wait complete. 2023-11-24T08:10:01.718 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-11-24T08:10:01.295452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237240,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737480,"statfs":{"total":773094113280,"available":772851179520,"internally_reserved":0,"allocated":4497408,"data_stored":2465772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001900"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333991+0000","last_change":"2023-11-24T08:09:57.339594+0000","last_active":"2023-11-24T08:09:58.333991+0000","last_peered":"2023-11-24T08:09:58.333991+0000","last_clean":"2023-11-24T08:09:58.333991+0000","last_became_active":"2023-11-24T08:09:57.339301+0000","last_became_peered":"2023-11-24T08:09:57.339301+0000","last_unstale":"2023-11-24T08:09:58.333991+0000","last_undegraded":"2023-11-24T08:09:58.333991+0000","last_fullsized":"2023-11-24T08:09:58.333991+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:05.820823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349514+0000","last_change":"2023-11-24T08:09:57.348745+0000","last_active":"2023-11-24T08:09:58.349514+0000","last_peered":"2023-11-24T08:09:58.349514+0000","last_clean":"2023-11-24T08:09:58.349514+0000","last_became_active":"2023-11-24T08:09:57.344690+0000","last_became_peered":"2023-11-24T08:09:57.344690+0000","last_unstale":"2023-11-24T08:09:58.349514+0000","last_undegraded":"2023-11-24T08:09:58.349514+0000","last_fullsized":"2023-11-24T08:09:58.349514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:32.378462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332830+0000","last_change":"2023-11-24T08:09:57.345052+0000","last_active":"2023-11-24T08:09:58.332830+0000","last_peered":"2023-11-24T08:09:58.332830+0000","last_clean":"2023-11-24T08:09:58.332830+0000","last_became_active":"2023-11-24T08:09:57.344742+0000","last_became_peered":"2023-11-24T08:09:57.344742+0000","last_unstale":"2023-11-24T08:09:58.332830+0000","last_undegraded":"2023-11-24T08:09:58.332830+0000","last_fullsized":"2023-11-24T08:09:58.332830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:48.577263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330861+0000","last_change":"2023-11-24T08:09:57.345287+0000","last_active":"2023-11-24T08:09:58.330861+0000","last_peered":"2023-11-24T08:09:58.330861+0000","last_clean":"2023-11-24T08:09:58.330861+0000","last_became_active":"2023-11-24T08:09:57.344969+0000","last_became_peered":"2023-11-24T08:09:57.344969+0000","last_unstale":"2023-11-24T08:09:58.330861+0000","last_undegraded":"2023-11-24T08:09:58.330861+0000","last_fullsized":"2023-11-24T08:09:58.330861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:06.577748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349687+0000","last_change":"2023-11-24T08:09:57.340128+0000","last_active":"2023-11-24T08:09:58.349687+0000","last_peered":"2023-11-24T08:09:58.349687+0000","last_clean":"2023-11-24T08:09:58.349687+0000","last_became_active":"2023-11-24T08:09:57.339007+0000","last_became_peered":"2023-11-24T08:09:57.339007+0000","last_unstale":"2023-11-24T08:09:58.349687+0000","last_undegraded":"2023-11-24T08:09:58.349687+0000","last_fullsized":"2023-11-24T08:09:58.349687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:48:18.665823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349392+0000","last_change":"2023-11-24T08:09:57.349184+0000","last_active":"2023-11-24T08:09:58.349392+0000","last_peered":"2023-11-24T08:09:58.349392+0000","last_clean":"2023-11-24T08:09:58.349392+0000","last_became_active":"2023-11-24T08:09:57.345791+0000","last_became_peered":"2023-11-24T08:09:57.345791+0000","last_unstale":"2023-11-24T08:09:58.349392+0000","last_undegraded":"2023-11-24T08:09:58.349392+0000","last_fullsized":"2023-11-24T08:09:58.349392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:23.964175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351812+0000","last_change":"2023-11-24T08:09:57.334549+0000","last_active":"2023-11-24T08:09:58.351812+0000","last_peered":"2023-11-24T08:09:58.351812+0000","last_clean":"2023-11-24T08:09:58.351812+0000","last_became_active":"2023-11-24T08:09:57.332014+0000","last_became_peered":"2023-11-24T08:09:57.332014+0000","last_unstale":"2023-11-24T08:09:58.351812+0000","last_undegraded":"2023-11-24T08:09:58.351812+0000","last_fullsized":"2023-11-24T08:09:58.351812+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:01.583026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349688+0000","last_change":"2023-11-24T08:09:57.334754+0000","last_active":"2023-11-24T08:09:58.349688+0000","last_peered":"2023-11-24T08:09:58.349688+0000","last_clean":"2023-11-24T08:09:58.349688+0000","last_became_active":"2023-11-24T08:09:57.334508+0000","last_became_peered":"2023-11-24T08:09:57.334508+0000","last_unstale":"2023-11-24T08:09:58.349688+0000","last_undegraded":"2023-11-24T08:09:58.349688+0000","last_fullsized":"2023-11-24T08:09:58.349688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:41.487984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352676+0000","last_change":"2023-11-24T08:09:57.344389+0000","last_active":"2023-11-24T08:09:58.352676+0000","last_peered":"2023-11-24T08:09:58.352676+0000","last_clean":"2023-11-24T08:09:58.352676+0000","last_became_active":"2023-11-24T08:09:57.344085+0000","last_became_peered":"2023-11-24T08:09:57.344085+0000","last_unstale":"2023-11-24T08:09:58.352676+0000","last_undegraded":"2023-11-24T08:09:58.352676+0000","last_fullsized":"2023-11-24T08:09:58.352676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:21.341963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331334+0000","last_change":"2023-11-24T08:09:57.347866+0000","last_active":"2023-11-24T08:09:58.331334+0000","last_peered":"2023-11-24T08:09:58.331334+0000","last_clean":"2023-11-24T08:09:58.331334+0000","last_became_active":"2023-11-24T08:09:57.347598+0000","last_became_peered":"2023-11-24T08:09:57.347598+0000","last_unstale":"2023-11-24T08:09:58.331334+0000","last_undegraded":"2023-11-24T08:09:58.331334+0000","last_fullsized":"2023-11-24T08:09:58.331334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:36.347389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352472+0000","last_change":"2023-11-24T08:09:57.348482+0000","last_active":"2023-11-24T08:09:58.352472+0000","last_peered":"2023-11-24T08:09:58.352472+0000","last_clean":"2023-11-24T08:09:58.352472+0000","last_became_active":"2023-11-24T08:09:57.348212+0000","last_became_peered":"2023-11-24T08:09:57.348212+0000","last_unstale":"2023-11-24T08:09:58.352472+0000","last_undegraded":"2023-11-24T08:09:58.352472+0000","last_fullsized":"2023-11-24T08:09:58.352472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:32.378462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332427+0000","last_change":"2023-11-24T08:09:57.350385+0000","last_active":"2023-11-24T08:09:58.332427+0000","last_peered":"2023-11-24T08:09:58.332427+0000","last_clean":"2023-11-24T08:09:58.332427+0000","last_became_active":"2023-11-24T08:09:57.350147+0000","last_became_peered":"2023-11-24T08:09:57.350147+0000","last_unstale":"2023-11-24T08:09:58.332427+0000","last_undegraded":"2023-11-24T08:09:58.332427+0000","last_fullsized":"2023-11-24T08:09:58.332427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:34.608636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349829+0000","last_change":"2023-11-24T08:09:57.333929+0000","last_active":"2023-11-24T08:09:58.349829+0000","last_peered":"2023-11-24T08:09:58.349829+0000","last_clean":"2023-11-24T08:09:58.349829+0000","last_became_active":"2023-11-24T08:09:57.333536+0000","last_became_peered":"2023-11-24T08:09:57.333536+0000","last_unstale":"2023-11-24T08:09:58.349829+0000","last_undegraded":"2023-11-24T08:09:58.349829+0000","last_fullsized":"2023-11-24T08:09:58.349829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:51:18.149278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349719+0000","last_change":"2023-11-24T08:09:57.334678+0000","last_active":"2023-11-24T08:09:58.349719+0000","last_peered":"2023-11-24T08:09:58.349719+0000","last_clean":"2023-11-24T08:09:58.349719+0000","last_became_active":"2023-11-24T08:09:57.334450+0000","last_became_peered":"2023-11-24T08:09:57.334450+0000","last_unstale":"2023-11-24T08:09:58.349719+0000","last_undegraded":"2023-11-24T08:09:58.349719+0000","last_fullsized":"2023-11-24T08:09:58.349719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:39.725826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331838+0000","last_change":"2023-11-24T08:09:57.342912+0000","last_active":"2023-11-24T08:09:58.331838+0000","last_peered":"2023-11-24T08:09:58.331838+0000","last_clean":"2023-11-24T08:09:58.331838+0000","last_became_active":"2023-11-24T08:09:57.342606+0000","last_became_peered":"2023-11-24T08:09:57.342606+0000","last_unstale":"2023-11-24T08:09:58.331838+0000","last_undegraded":"2023-11-24T08:09:58.331838+0000","last_fullsized":"2023-11-24T08:09:58.331838+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:18.248284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350352+0000","last_change":"2023-11-24T08:09:57.338833+0000","last_active":"2023-11-24T08:09:58.350352+0000","last_peered":"2023-11-24T08:09:58.350352+0000","last_clean":"2023-11-24T08:09:58.350352+0000","last_became_active":"2023-11-24T08:09:57.338332+0000","last_became_peered":"2023-11-24T08:09:57.338332+0000","last_unstale":"2023-11-24T08:09:58.350352+0000","last_undegraded":"2023-11-24T08:09:58.350352+0000","last_fullsized":"2023-11-24T08:09:58.350352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:19.185246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352778+0000","last_change":"2023-11-24T08:09:57.328071+0000","last_active":"2023-11-24T08:09:58.352778+0000","last_peered":"2023-11-24T08:09:58.352778+0000","last_clean":"2023-11-24T08:09:58.352778+0000","last_became_active":"2023-11-24T08:09:57.327695+0000","last_became_peered":"2023-11-24T08:09:57.327695+0000","last_unstale":"2023-11-24T08:09:58.352778+0000","last_undegraded":"2023-11-24T08:09:58.352778+0000","last_fullsized":"2023-11-24T08:09:58.352778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:29.840340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349200+0000","last_change":"2023-11-24T08:09:57.336519+0000","last_active":"2023-11-24T08:09:58.349200+0000","last_peered":"2023-11-24T08:09:58.349200+0000","last_clean":"2023-11-24T08:09:58.349200+0000","last_became_active":"2023-11-24T08:09:57.336296+0000","last_became_peered":"2023-11-24T08:09:57.336296+0000","last_unstale":"2023-11-24T08:09:58.349200+0000","last_undegraded":"2023-11-24T08:09:58.349200+0000","last_fullsized":"2023-11-24T08:09:58.349200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:10.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350048+0000","last_change":"2023-11-24T08:09:57.348089+0000","last_active":"2023-11-24T08:09:58.350048+0000","last_peered":"2023-11-24T08:09:58.350048+0000","last_clean":"2023-11-24T08:09:58.350048+0000","last_became_active":"2023-11-24T08:09:57.347837+0000","last_became_peered":"2023-11-24T08:09:57.347837+0000","last_unstale":"2023-11-24T08:09:58.350048+0000","last_undegraded":"2023-11-24T08:09:58.350048+0000","last_fullsized":"2023-11-24T08:09:58.350048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:27:04.437872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350628+0000","last_change":"2023-11-24T08:09:57.340443+0000","last_active":"2023-11-24T08:09:58.350628+0000","last_peered":"2023-11-24T08:09:58.350628+0000","last_clean":"2023-11-24T08:09:58.350628+0000","last_became_active":"2023-11-24T08:09:57.340222+0000","last_became_peered":"2023-11-24T08:09:57.340222+0000","last_unstale":"2023-11-24T08:09:58.350628+0000","last_undegraded":"2023-11-24T08:09:58.350628+0000","last_fullsized":"2023-11-24T08:09:58.350628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:28.443968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350506+0000","last_change":"2023-11-24T08:09:57.347587+0000","last_active":"2023-11-24T08:09:58.350506+0000","last_peered":"2023-11-24T08:09:58.350506+0000","last_clean":"2023-11-24T08:09:58.350506+0000","last_became_active":"2023-11-24T08:09:57.345081+0000","last_became_peered":"2023-11-24T08:09:57.345081+0000","last_unstale":"2023-11-24T08:09:58.350506+0000","last_undegraded":"2023-11-24T08:09:58.350506+0000","last_fullsized":"2023-11-24T08:09:58.350506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:54.576837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350353+0000","last_change":"2023-11-24T08:09:57.346560+0000","last_active":"2023-11-24T08:09:58.350353+0000","last_peered":"2023-11-24T08:09:58.350353+0000","last_clean":"2023-11-24T08:09:58.350353+0000","last_became_active":"2023-11-24T08:09:57.346267+0000","last_became_peered":"2023-11-24T08:09:57.346267+0000","last_unstale":"2023-11-24T08:09:58.350353+0000","last_undegraded":"2023-11-24T08:09:58.350353+0000","last_fullsized":"2023-11-24T08:09:58.350353+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:31.108048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351797+0000","last_change":"2023-11-24T08:09:57.344265+0000","last_active":"2023-11-24T08:09:58.351797+0000","last_peered":"2023-11-24T08:09:58.351797+0000","last_clean":"2023-11-24T08:09:58.351797+0000","last_became_active":"2023-11-24T08:09:57.344027+0000","last_became_peered":"2023-11-24T08:09:57.344027+0000","last_unstale":"2023-11-24T08:09:58.351797+0000","last_undegraded":"2023-11-24T08:09:58.351797+0000","last_fullsized":"2023-11-24T08:09:58.351797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:55.117423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331659+0000","last_change":"2023-11-24T08:09:57.337893+0000","last_active":"2023-11-24T08:09:58.331659+0000","last_peered":"2023-11-24T08:09:58.331659+0000","last_clean":"2023-11-24T08:09:58.331659+0000","last_became_active":"2023-11-24T08:09:57.336317+0000","last_became_peered":"2023-11-24T08:09:57.336317+0000","last_unstale":"2023-11-24T08:09:58.331659+0000","last_undegraded":"2023-11-24T08:09:58.331659+0000","last_fullsized":"2023-11-24T08:09:58.331659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:08:48.148109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331469+0000","last_change":"2023-11-24T08:09:57.336201+0000","last_active":"2023-11-24T08:09:58.331469+0000","last_peered":"2023-11-24T08:09:58.331469+0000","last_clean":"2023-11-24T08:09:58.331469+0000","last_became_active":"2023-11-24T08:09:57.335896+0000","last_became_peered":"2023-11-24T08:09:57.335896+0000","last_unstale":"2023-11-24T08:09:58.331469+0000","last_undegraded":"2023-11-24T08:09:58.331469+0000","last_fullsized":"2023-11-24T08:09:58.331469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:41.487984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349333+0000","last_change":"2023-11-24T08:09:57.327968+0000","last_active":"2023-11-24T08:09:58.349333+0000","last_peered":"2023-11-24T08:09:58.349333+0000","last_clean":"2023-11-24T08:09:58.349333+0000","last_became_active":"2023-11-24T08:09:57.327666+0000","last_became_peered":"2023-11-24T08:09:57.327666+0000","last_unstale":"2023-11-24T08:09:58.349333+0000","last_undegraded":"2023-11-24T08:09:58.349333+0000","last_fullsized":"2023-11-24T08:09:58.349333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:41.039861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331861+0000","last_change":"2023-11-24T08:09:57.342058+0000","last_active":"2023-11-24T08:09:58.331861+0000","last_peered":"2023-11-24T08:09:58.331861+0000","last_clean":"2023-11-24T08:09:58.331861+0000","last_became_active":"2023-11-24T08:09:57.341681+0000","last_became_peered":"2023-11-24T08:09:57.341681+0000","last_unstale":"2023-11-24T08:09:58.331861+0000","last_undegraded":"2023-11-24T08:09:58.331861+0000","last_fullsized":"2023-11-24T08:09:58.331861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:21.341963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332508+0000","last_change":"2023-11-24T08:09:57.347744+0000","last_active":"2023-11-24T08:09:58.332508+0000","last_peered":"2023-11-24T08:09:58.332508+0000","last_clean":"2023-11-24T08:09:58.332508+0000","last_became_active":"2023-11-24T08:09:57.347529+0000","last_became_peered":"2023-11-24T08:09:57.347529+0000","last_unstale":"2023-11-24T08:09:58.332508+0000","last_undegraded":"2023-11-24T08:09:58.332508+0000","last_fullsized":"2023-11-24T08:09:58.332508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:25:52.692767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350961+0000","last_change":"2023-11-24T08:09:57.341318+0000","last_active":"2023-11-24T08:09:58.350961+0000","last_peered":"2023-11-24T08:09:58.350961+0000","last_clean":"2023-11-24T08:09:58.350961+0000","last_became_active":"2023-11-24T08:09:57.340942+0000","last_became_peered":"2023-11-24T08:09:57.340942+0000","last_unstale":"2023-11-24T08:09:58.350961+0000","last_undegraded":"2023-11-24T08:09:58.350961+0000","last_fullsized":"2023-11-24T08:09:58.350961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:06.577748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351950+0000","last_change":"2023-11-24T08:09:57.329124+0000","last_active":"2023-11-24T08:09:58.351950+0000","last_peered":"2023-11-24T08:09:58.351950+0000","last_clean":"2023-11-24T08:09:58.351950+0000","last_became_active":"2023-11-24T08:09:57.328498+0000","last_became_peered":"2023-11-24T08:09:57.328498+0000","last_unstale":"2023-11-24T08:09:58.351950+0000","last_undegraded":"2023-11-24T08:09:58.351950+0000","last_fullsized":"2023-11-24T08:09:58.351950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:10.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333337+0000","last_change":"2023-11-24T08:09:57.336739+0000","last_active":"2023-11-24T08:09:58.333337+0000","last_peered":"2023-11-24T08:09:58.333337+0000","last_clean":"2023-11-24T08:09:58.333337+0000","last_became_active":"2023-11-24T08:09:57.336445+0000","last_became_peered":"2023-11-24T08:09:57.336445+0000","last_unstale":"2023-11-24T08:09:58.333337+0000","last_undegraded":"2023-11-24T08:09:58.333337+0000","last_fullsized":"2023-11-24T08:09:58.333337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:51.848914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349779+0000","last_change":"2023-11-24T08:09:57.338607+0000","last_active":"2023-11-24T08:09:58.349779+0000","last_peered":"2023-11-24T08:09:58.349779+0000","last_clean":"2023-11-24T08:09:58.349779+0000","last_became_active":"2023-11-24T08:09:57.337741+0000","last_became_peered":"2023-11-24T08:09:57.337741+0000","last_unstale":"2023-11-24T08:09:58.349779+0000","last_undegraded":"2023-11-24T08:09:58.349779+0000","last_fullsized":"2023-11-24T08:09:58.349779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:21.341963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352161+0000","last_change":"2023-11-24T08:09:57.342705+0000","last_active":"2023-11-24T08:09:58.352161+0000","last_peered":"2023-11-24T08:09:58.352161+0000","last_clean":"2023-11-24T08:09:58.352161+0000","last_became_active":"2023-11-24T08:09:57.342412+0000","last_became_peered":"2023-11-24T08:09:57.342412+0000","last_unstale":"2023-11-24T08:09:58.352161+0000","last_undegraded":"2023-11-24T08:09:58.352161+0000","last_fullsized":"2023-11-24T08:09:58.352161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:31.108048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.356102+0000","last_change":"2023-11-24T08:09:56.335055+0000","last_active":"2023-11-24T08:09:57.356102+0000","last_peered":"2023-11-24T08:09:57.356102+0000","last_clean":"2023-11-24T08:09:57.356102+0000","last_became_active":"2023-11-24T08:09:56.333890+0000","last_became_peered":"2023-11-24T08:09:56.333890+0000","last_unstale":"2023-11-24T08:09:57.356102+0000","last_undegraded":"2023-11-24T08:09:57.356102+0000","last_fullsized":"2023-11-24T08:09:57.356102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:27.338331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333830+0000","last_change":"2023-11-24T08:09:57.337726+0000","last_active":"2023-11-24T08:09:58.333830+0000","last_peered":"2023-11-24T08:09:58.333830+0000","last_clean":"2023-11-24T08:09:58.333830+0000","last_became_active":"2023-11-24T08:09:57.337408+0000","last_became_peered":"2023-11-24T08:09:57.337408+0000","last_unstale":"2023-11-24T08:09:58.333830+0000","last_undegraded":"2023-11-24T08:09:58.333830+0000","last_fullsized":"2023-11-24T08:09:58.333830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:22.560748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352258+0000","last_change":"2023-11-24T08:09:56.334036+0000","last_active":"2023-11-24T08:09:58.352258+0000","last_peered":"2023-11-24T08:09:58.352258+0000","last_clean":"2023-11-24T08:09:58.352258+0000","last_became_active":"2023-11-24T08:09:56.333585+0000","last_became_peered":"2023-11-24T08:09:56.333585+0000","last_unstale":"2023-11-24T08:09:58.352258+0000","last_undegraded":"2023-11-24T08:09:58.352258+0000","last_fullsized":"2023-11-24T08:09:58.352258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:04.110358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351362+0000","last_change":"2023-11-24T08:09:57.346217+0000","last_active":"2023-11-24T08:09:58.351362+0000","last_peered":"2023-11-24T08:09:58.351362+0000","last_clean":"2023-11-24T08:09:58.351362+0000","last_became_active":"2023-11-24T08:09:57.345105+0000","last_became_peered":"2023-11-24T08:09:57.345105+0000","last_unstale":"2023-11-24T08:09:58.351362+0000","last_undegraded":"2023-11-24T08:09:58.351362+0000","last_fullsized":"2023-11-24T08:09:58.351362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:36.347389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.358219+0000","last_change":"2023-11-24T08:09:56.326820+0000","last_active":"2023-11-24T08:09:57.358219+0000","last_peered":"2023-11-24T08:09:57.358219+0000","last_clean":"2023-11-24T08:09:57.358219+0000","last_became_active":"2023-11-24T08:09:56.326554+0000","last_became_peered":"2023-11-24T08:09:56.326554+0000","last_unstale":"2023-11-24T08:09:57.358219+0000","last_undegraded":"2023-11-24T08:09:57.358219+0000","last_fullsized":"2023-11-24T08:09:57.358219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:21.555016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332148+0000","last_change":"2023-11-24T08:09:57.342008+0000","last_active":"2023-11-24T08:09:58.332148+0000","last_peered":"2023-11-24T08:09:58.332148+0000","last_clean":"2023-11-24T08:09:58.332148+0000","last_became_active":"2023-11-24T08:09:57.341358+0000","last_became_peered":"2023-11-24T08:09:57.341358+0000","last_unstale":"2023-11-24T08:09:58.332148+0000","last_undegraded":"2023-11-24T08:09:58.332148+0000","last_fullsized":"2023-11-24T08:09:58.332148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:32.521394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.357872+0000","last_change":"2023-11-24T08:09:56.331123+0000","last_active":"2023-11-24T08:09:57.357872+0000","last_peered":"2023-11-24T08:09:57.357872+0000","last_clean":"2023-11-24T08:09:57.357872+0000","last_became_active":"2023-11-24T08:09:56.326470+0000","last_became_peered":"2023-11-24T08:09:56.326470+0000","last_unstale":"2023-11-24T08:09:57.357872+0000","last_undegraded":"2023-11-24T08:09:57.357872+0000","last_fullsized":"2023-11-24T08:09:57.357872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:56:24.340594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352316+0000","last_change":"2023-11-24T08:09:57.338671+0000","last_active":"2023-11-24T08:09:58.352316+0000","last_peered":"2023-11-24T08:09:58.352316+0000","last_clean":"2023-11-24T08:09:58.352316+0000","last_became_active":"2023-11-24T08:09:57.338343+0000","last_became_peered":"2023-11-24T08:09:57.338343+0000","last_unstale":"2023-11-24T08:09:58.352316+0000","last_undegraded":"2023-11-24T08:09:58.352316+0000","last_fullsized":"2023-11-24T08:09:58.352316+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:23.964175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.357995+0000","last_change":"2023-11-24T08:09:56.329938+0000","last_active":"2023-11-24T08:09:57.357995+0000","last_peered":"2023-11-24T08:09:57.357995+0000","last_clean":"2023-11-24T08:09:57.357995+0000","last_became_active":"2023-11-24T08:09:56.327630+0000","last_became_peered":"2023-11-24T08:09:56.327630+0000","last_unstale":"2023-11-24T08:09:57.357995+0000","last_undegraded":"2023-11-24T08:09:57.357995+0000","last_fullsized":"2023-11-24T08:09:57.357995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:21.555016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332299+0000","last_change":"2023-11-24T08:09:57.347128+0000","last_active":"2023-11-24T08:09:58.332299+0000","last_peered":"2023-11-24T08:09:58.332299+0000","last_clean":"2023-11-24T08:09:58.332299+0000","last_became_active":"2023-11-24T08:09:57.346788+0000","last_became_peered":"2023-11-24T08:09:57.346788+0000","last_unstale":"2023-11-24T08:09:58.332299+0000","last_undegraded":"2023-11-24T08:09:58.332299+0000","last_fullsized":"2023-11-24T08:09:58.332299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:42.456620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.358072+0000","last_change":"2023-11-24T08:09:56.326659+0000","last_active":"2023-11-24T08:09:57.358072+0000","last_peered":"2023-11-24T08:09:57.358072+0000","last_clean":"2023-11-24T08:09:57.358072+0000","last_became_active":"2023-11-24T08:09:56.326157+0000","last_became_peered":"2023-11-24T08:09:56.326157+0000","last_unstale":"2023-11-24T08:09:57.358072+0000","last_undegraded":"2023-11-24T08:09:57.358072+0000","last_fullsized":"2023-11-24T08:09:57.358072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:18.292579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350023+0000","last_change":"2023-11-24T08:09:57.333696+0000","last_active":"2023-11-24T08:09:58.350023+0000","last_peered":"2023-11-24T08:09:58.350023+0000","last_clean":"2023-11-24T08:09:58.350023+0000","last_became_active":"2023-11-24T08:09:57.333416+0000","last_became_peered":"2023-11-24T08:09:57.333416+0000","last_unstale":"2023-11-24T08:09:58.350023+0000","last_undegraded":"2023-11-24T08:09:58.350023+0000","last_fullsized":"2023-11-24T08:09:58.350023+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.928845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.308635+0000","last_change":"2023-11-24T08:09:56.328940+0000","last_active":"2023-11-24T08:09:57.308635+0000","last_peered":"2023-11-24T08:09:57.308635+0000","last_clean":"2023-11-24T08:09:57.308635+0000","last_became_active":"2023-11-24T08:09:56.328528+0000","last_became_peered":"2023-11-24T08:09:56.328528+0000","last_unstale":"2023-11-24T08:09:57.308635+0000","last_undegraded":"2023-11-24T08:09:57.308635+0000","last_fullsized":"2023-11-24T08:09:57.308635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:30.102315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330406+0000","last_change":"2023-11-24T08:09:57.339427+0000","last_active":"2023-11-24T08:09:58.330406+0000","last_peered":"2023-11-24T08:09:58.330406+0000","last_clean":"2023-11-24T08:09:58.330406+0000","last_became_active":"2023-11-24T08:09:57.339122+0000","last_became_peered":"2023-11-24T08:09:57.339122+0000","last_unstale":"2023-11-24T08:09:58.330406+0000","last_undegraded":"2023-11-24T08:09:58.330406+0000","last_fullsized":"2023-11-24T08:09:58.330406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:31.108048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.325896+0000","last_change":"2023-11-24T08:09:56.323625+0000","last_active":"2023-11-24T08:09:57.325896+0000","last_peered":"2023-11-24T08:09:57.325896+0000","last_clean":"2023-11-24T08:09:57.325896+0000","last_became_active":"2023-11-24T08:09:56.323161+0000","last_became_peered":"2023-11-24T08:09:56.323161+0000","last_unstale":"2023-11-24T08:09:57.325896+0000","last_undegraded":"2023-11-24T08:09:57.325896+0000","last_fullsized":"2023-11-24T08:09:57.325896+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:57.236491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333032+0000","last_change":"2023-11-24T08:09:57.349505+0000","last_active":"2023-11-24T08:09:58.333032+0000","last_peered":"2023-11-24T08:09:58.333032+0000","last_clean":"2023-11-24T08:09:58.333032+0000","last_became_active":"2023-11-24T08:09:57.349249+0000","last_became_peered":"2023-11-24T08:09:57.349249+0000","last_unstale":"2023-11-24T08:09:58.333032+0000","last_undegraded":"2023-11-24T08:09:58.333032+0000","last_fullsized":"2023-11-24T08:09:58.333032+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:24.386449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.309129+0000","last_change":"2023-11-24T08:09:56.335469+0000","last_active":"2023-11-24T08:09:57.309129+0000","last_peered":"2023-11-24T08:09:57.309129+0000","last_clean":"2023-11-24T08:09:57.309129+0000","last_became_active":"2023-11-24T08:09:56.335167+0000","last_became_peered":"2023-11-24T08:09:56.335167+0000","last_unstale":"2023-11-24T08:09:57.309129+0000","last_undegraded":"2023-11-24T08:09:57.309129+0000","last_fullsized":"2023-11-24T08:09:57.309129+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:53.571105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350342+0000","last_change":"2023-11-24T08:09:57.346589+0000","last_active":"2023-11-24T08:09:58.350342+0000","last_peered":"2023-11-24T08:09:58.350342+0000","last_clean":"2023-11-24T08:09:58.350342+0000","last_became_active":"2023-11-24T08:09:57.346290+0000","last_became_peered":"2023-11-24T08:09:57.346290+0000","last_unstale":"2023-11-24T08:09:58.350342+0000","last_undegraded":"2023-11-24T08:09:58.350342+0000","last_fullsized":"2023-11-24T08:09:58.350342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:02.739444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352443+0000","last_change":"2023-11-24T08:09:56.328690+0000","last_active":"2023-11-24T08:09:58.352443+0000","last_peered":"2023-11-24T08:09:58.352443+0000","last_clean":"2023-11-24T08:09:58.352443+0000","last_became_active":"2023-11-24T08:09:56.327008+0000","last_became_peered":"2023-11-24T08:09:56.327008+0000","last_unstale":"2023-11-24T08:09:58.352443+0000","last_undegraded":"2023-11-24T08:09:58.352443+0000","last_fullsized":"2023-11-24T08:09:58.352443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:04.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349611+0000","last_change":"2023-11-24T08:09:57.326872+0000","last_active":"2023-11-24T08:09:58.349611+0000","last_peered":"2023-11-24T08:09:58.349611+0000","last_clean":"2023-11-24T08:09:58.349611+0000","last_became_active":"2023-11-24T08:09:57.325481+0000","last_became_peered":"2023-11-24T08:09:57.325481+0000","last_unstale":"2023-11-24T08:09:58.349611+0000","last_undegraded":"2023-11-24T08:09:58.349611+0000","last_fullsized":"2023-11-24T08:09:58.349611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:29.840340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.355340+0000","last_change":"2023-11-24T08:09:56.335960+0000","last_active":"2023-11-24T08:09:57.355340+0000","last_peered":"2023-11-24T08:09:57.355340+0000","last_clean":"2023-11-24T08:09:57.355340+0000","last_became_active":"2023-11-24T08:09:56.335665+0000","last_became_peered":"2023-11-24T08:09:56.335665+0000","last_unstale":"2023-11-24T08:09:57.355340+0000","last_undegraded":"2023-11-24T08:09:57.355340+0000","last_fullsized":"2023-11-24T08:09:57.355340+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:04.045363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332684+0000","last_change":"2023-11-24T08:09:57.346488+0000","last_active":"2023-11-24T08:09:58.332684+0000","last_peered":"2023-11-24T08:09:58.332684+0000","last_clean":"2023-11-24T08:09:58.332684+0000","last_became_active":"2023-11-24T08:09:57.346237+0000","last_became_peered":"2023-11-24T08:09:57.346237+0000","last_unstale":"2023-11-24T08:09:58.332684+0000","last_undegraded":"2023-11-24T08:09:58.332684+0000","last_fullsized":"2023-11-24T08:09:58.332684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:12:49.330012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.363084+0000","last_change":"2023-11-24T08:09:56.327612+0000","last_active":"2023-11-24T08:09:57.363084+0000","last_peered":"2023-11-24T08:09:57.363084+0000","last_clean":"2023-11-24T08:09:57.363084+0000","last_became_active":"2023-11-24T08:09:56.327332+0000","last_became_peered":"2023-11-24T08:09:56.327332+0000","last_unstale":"2023-11-24T08:09:57.363084+0000","last_undegraded":"2023-11-24T08:09:57.363084+0000","last_fullsized":"2023-11-24T08:09:57.363084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:20.336231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330532+0000","last_change":"2023-11-24T08:09:57.342440+0000","last_active":"2023-11-24T08:09:58.330532+0000","last_peered":"2023-11-24T08:09:58.330532+0000","last_clean":"2023-11-24T08:09:58.330532+0000","last_became_active":"2023-11-24T08:09:57.342154+0000","last_became_peered":"2023-11-24T08:09:57.342154+0000","last_unstale":"2023-11-24T08:09:58.330532+0000","last_undegraded":"2023-11-24T08:09:58.330532+0000","last_fullsized":"2023-11-24T08:09:58.330532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:54.576837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.353995+0000","last_change":"2023-11-24T08:09:56.325701+0000","last_active":"2023-11-24T08:09:57.353995+0000","last_peered":"2023-11-24T08:09:57.353995+0000","last_clean":"2023-11-24T08:09:57.353995+0000","last_became_active":"2023-11-24T08:09:56.325397+0000","last_became_peered":"2023-11-24T08:09:56.325397+0000","last_unstale":"2023-11-24T08:09:57.353995+0000","last_undegraded":"2023-11-24T08:09:57.353995+0000","last_fullsized":"2023-11-24T08:09:57.353995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:04.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350313+0000","last_change":"2023-11-24T08:09:57.338102+0000","last_active":"2023-11-24T08:09:58.350313+0000","last_peered":"2023-11-24T08:09:58.350313+0000","last_clean":"2023-11-24T08:09:58.350313+0000","last_became_active":"2023-11-24T08:09:57.337854+0000","last_became_peered":"2023-11-24T08:09:57.337854+0000","last_unstale":"2023-11-24T08:09:58.350313+0000","last_undegraded":"2023-11-24T08:09:58.350313+0000","last_fullsized":"2023-11-24T08:09:58.350313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:01.999633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.329607+0000","last_change":"2023-11-24T08:09:56.322061+0000","last_active":"2023-11-24T08:09:57.329607+0000","last_peered":"2023-11-24T08:09:57.329607+0000","last_clean":"2023-11-24T08:09:57.329607+0000","last_became_active":"2023-11-24T08:09:56.321806+0000","last_became_peered":"2023-11-24T08:09:56.321806+0000","last_unstale":"2023-11-24T08:09:57.329607+0000","last_undegraded":"2023-11-24T08:09:57.329607+0000","last_fullsized":"2023-11-24T08:09:57.329607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:04.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350082+0000","last_change":"2023-11-24T08:09:57.338019+0000","last_active":"2023-11-24T08:09:58.350082+0000","last_peered":"2023-11-24T08:09:58.350082+0000","last_clean":"2023-11-24T08:09:58.350082+0000","last_became_active":"2023-11-24T08:09:57.337328+0000","last_became_peered":"2023-11-24T08:09:57.337328+0000","last_unstale":"2023-11-24T08:09:58.350082+0000","last_undegraded":"2023-11-24T08:09:58.350082+0000","last_fullsized":"2023-11-24T08:09:58.350082+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:32.521394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.308006+0000","last_change":"2023-11-24T08:09:56.335379+0000","last_active":"2023-11-24T08:09:57.308006+0000","last_peered":"2023-11-24T08:09:57.308006+0000","last_clean":"2023-11-24T08:09:57.308006+0000","last_became_active":"2023-11-24T08:09:56.335102+0000","last_became_peered":"2023-11-24T08:09:56.335102+0000","last_unstale":"2023-11-24T08:09:57.308006+0000","last_undegraded":"2023-11-24T08:09:57.308006+0000","last_fullsized":"2023-11-24T08:09:57.308006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333412+0000","last_change":"2023-11-24T08:09:57.337960+0000","last_active":"2023-11-24T08:09:58.333412+0000","last_peered":"2023-11-24T08:09:58.333412+0000","last_clean":"2023-11-24T08:09:58.333412+0000","last_became_active":"2023-11-24T08:09:57.337386+0000","last_became_peered":"2023-11-24T08:09:57.337386+0000","last_unstale":"2023-11-24T08:09:58.333412+0000","last_undegraded":"2023-11-24T08:09:58.333412+0000","last_fullsized":"2023-11-24T08:09:58.333412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:28.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.360921+0000","last_change":"2023-11-24T08:09:56.322047+0000","last_active":"2023-11-24T08:09:57.360921+0000","last_peered":"2023-11-24T08:09:57.360921+0000","last_clean":"2023-11-24T08:09:57.360921+0000","last_became_active":"2023-11-24T08:09:56.321761+0000","last_became_peered":"2023-11-24T08:09:56.321761+0000","last_unstale":"2023-11-24T08:09:57.360921+0000","last_undegraded":"2023-11-24T08:09:57.360921+0000","last_fullsized":"2023-11-24T08:09:57.360921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:27.993623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351482+0000","last_change":"2023-11-24T08:09:57.344842+0000","last_active":"2023-11-24T08:09:58.351482+0000","last_peered":"2023-11-24T08:09:58.351482+0000","last_clean":"2023-11-24T08:09:58.351482+0000","last_became_active":"2023-11-24T08:09:57.344499+0000","last_became_peered":"2023-11-24T08:09:57.344499+0000","last_unstale":"2023-11-24T08:09:58.351482+0000","last_undegraded":"2023-11-24T08:09:58.351482+0000","last_fullsized":"2023-11-24T08:09:58.351482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:41.487984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.357012+0000","last_change":"2023-11-24T08:09:56.327820+0000","last_active":"2023-11-24T08:09:57.357012+0000","last_peered":"2023-11-24T08:09:57.357012+0000","last_clean":"2023-11-24T08:09:57.357012+0000","last_became_active":"2023-11-24T08:09:56.327550+0000","last_became_peered":"2023-11-24T08:09:56.327550+0000","last_unstale":"2023-11-24T08:09:57.357012+0000","last_undegraded":"2023-11-24T08:09:57.357012+0000","last_fullsized":"2023-11-24T08:09:57.357012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:00:59.923113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333043+0000","last_change":"2023-11-24T08:09:57.348775+0000","last_active":"2023-11-24T08:09:58.333043+0000","last_peered":"2023-11-24T08:09:58.333043+0000","last_clean":"2023-11-24T08:09:58.333043+0000","last_became_active":"2023-11-24T08:09:57.348186+0000","last_became_peered":"2023-11-24T08:09:57.348186+0000","last_unstale":"2023-11-24T08:09:58.333043+0000","last_undegraded":"2023-11-24T08:09:58.333043+0000","last_fullsized":"2023-11-24T08:09:58.333043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:41:18.445825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.353657+0000","last_change":"2023-11-24T08:09:56.325891+0000","last_active":"2023-11-24T08:09:57.353657+0000","last_peered":"2023-11-24T08:09:57.353657+0000","last_clean":"2023-11-24T08:09:57.353657+0000","last_became_active":"2023-11-24T08:09:56.325495+0000","last_became_peered":"2023-11-24T08:09:56.325495+0000","last_unstale":"2023-11-24T08:09:57.353657+0000","last_undegraded":"2023-11-24T08:09:57.353657+0000","last_fullsized":"2023-11-24T08:09:57.353657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:04.110358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330843+0000","last_change":"2023-11-24T08:09:57.341510+0000","last_active":"2023-11-24T08:09:58.330843+0000","last_peered":"2023-11-24T08:09:58.330843+0000","last_clean":"2023-11-24T08:09:58.330843+0000","last_became_active":"2023-11-24T08:09:57.341193+0000","last_became_peered":"2023-11-24T08:09:57.341193+0000","last_unstale":"2023-11-24T08:09:58.330843+0000","last_undegraded":"2023-11-24T08:09:58.330843+0000","last_fullsized":"2023-11-24T08:09:58.330843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:55:37.663117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.359531+0000","last_change":"2023-11-24T08:09:56.329612+0000","last_active":"2023-11-24T08:09:57.359531+0000","last_peered":"2023-11-24T08:09:57.359531+0000","last_clean":"2023-11-24T08:09:57.359531+0000","last_became_active":"2023-11-24T08:09:56.329330+0000","last_became_peered":"2023-11-24T08:09:56.329330+0000","last_unstale":"2023-11-24T08:09:57.359531+0000","last_undegraded":"2023-11-24T08:09:57.359531+0000","last_fullsized":"2023-11-24T08:09:57.359531+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:48:17.660091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351185+0000","last_change":"2023-11-24T08:09:57.340436+0000","last_active":"2023-11-24T08:09:58.351185+0000","last_peered":"2023-11-24T08:09:58.351185+0000","last_clean":"2023-11-24T08:09:58.351185+0000","last_became_active":"2023-11-24T08:09:57.340164+0000","last_became_peered":"2023-11-24T08:09:57.340164+0000","last_unstale":"2023-11-24T08:09:58.351185+0000","last_undegraded":"2023-11-24T08:09:58.351185+0000","last_fullsized":"2023-11-24T08:09:58.351185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:19.298311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.322327+0000","last_change":"2023-11-24T08:09:56.330279+0000","last_active":"2023-11-24T08:09:57.322327+0000","last_peered":"2023-11-24T08:09:57.322327+0000","last_clean":"2023-11-24T08:09:57.322327+0000","last_became_active":"2023-11-24T08:09:56.329931+0000","last_became_peered":"2023-11-24T08:09:56.329931+0000","last_unstale":"2023-11-24T08:09:57.322327+0000","last_undegraded":"2023-11-24T08:09:57.322327+0000","last_fullsized":"2023-11-24T08:09:57.322327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:27.338331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349852+0000","last_change":"2023-11-24T08:09:57.337668+0000","last_active":"2023-11-24T08:09:58.349852+0000","last_peered":"2023-11-24T08:09:58.349852+0000","last_clean":"2023-11-24T08:09:58.349852+0000","last_became_active":"2023-11-24T08:09:57.337408+0000","last_became_peered":"2023-11-24T08:09:57.337408+0000","last_unstale":"2023-11-24T08:09:58.349852+0000","last_undegraded":"2023-11-24T08:09:58.349852+0000","last_fullsized":"2023-11-24T08:09:58.349852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:12.046326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352273+0000","last_change":"2023-11-24T08:09:56.319325+0000","last_active":"2023-11-24T08:09:58.352273+0000","last_peered":"2023-11-24T08:09:58.352273+0000","last_clean":"2023-11-24T08:09:58.352273+0000","last_became_active":"2023-11-24T08:09:56.318542+0000","last_became_peered":"2023-11-24T08:09:56.318542+0000","last_unstale":"2023-11-24T08:09:58.352273+0000","last_undegraded":"2023-11-24T08:09:58.352273+0000","last_fullsized":"2023-11-24T08:09:58.352273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:50.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349949+0000","last_change":"2023-11-24T08:09:57.338376+0000","last_active":"2023-11-24T08:09:58.349949+0000","last_peered":"2023-11-24T08:09:58.349949+0000","last_clean":"2023-11-24T08:09:58.349949+0000","last_became_active":"2023-11-24T08:09:57.338128+0000","last_became_peered":"2023-11-24T08:09:57.338128+0000","last_unstale":"2023-11-24T08:09:58.349949+0000","last_undegraded":"2023-11-24T08:09:58.349949+0000","last_fullsized":"2023-11-24T08:09:58.349949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:28.344063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.314575+0000","last_change":"2023-11-24T08:09:56.333904+0000","last_active":"2023-11-24T08:09:57.314575+0000","last_peered":"2023-11-24T08:09:57.314575+0000","last_clean":"2023-11-24T08:09:57.314575+0000","last_became_active":"2023-11-24T08:09:56.331340+0000","last_became_peered":"2023-11-24T08:09:56.331340+0000","last_unstale":"2023-11-24T08:09:57.314575+0000","last_undegraded":"2023-11-24T08:09:57.314575+0000","last_fullsized":"2023-11-24T08:09:57.314575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:30.935923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349861+0000","last_change":"2023-11-24T08:09:57.343688+0000","last_active":"2023-11-24T08:09:58.349861+0000","last_peered":"2023-11-24T08:09:58.349861+0000","last_clean":"2023-11-24T08:09:58.349861+0000","last_became_active":"2023-11-24T08:09:57.343488+0000","last_became_peered":"2023-11-24T08:09:57.343488+0000","last_unstale":"2023-11-24T08:09:58.349861+0000","last_undegraded":"2023-11-24T08:09:58.349861+0000","last_fullsized":"2023-11-24T08:09:58.349861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:45.085173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.329834+0000","last_change":"2023-11-24T08:09:56.325039+0000","last_active":"2023-11-24T08:09:57.329834+0000","last_peered":"2023-11-24T08:09:57.329834+0000","last_clean":"2023-11-24T08:09:57.329834+0000","last_became_active":"2023-11-24T08:09:56.324552+0000","last_became_peered":"2023-11-24T08:09:56.324552+0000","last_unstale":"2023-11-24T08:09:57.329834+0000","last_undegraded":"2023-11-24T08:09:57.329834+0000","last_fullsized":"2023-11-24T08:09:57.329834+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:27.993623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351709+0000","last_change":"2023-11-24T08:09:57.328012+0000","last_active":"2023-11-24T08:09:58.351709+0000","last_peered":"2023-11-24T08:09:58.351709+0000","last_clean":"2023-11-24T08:09:58.351709+0000","last_became_active":"2023-11-24T08:09:57.327707+0000","last_became_peered":"2023-11-24T08:09:57.327707+0000","last_unstale":"2023-11-24T08:09:58.351709+0000","last_undegraded":"2023-11-24T08:09:58.351709+0000","last_fullsized":"2023-11-24T08:09:58.351709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:10.423396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.333056+0000","last_change":"2023-11-24T08:09:56.338027+0000","last_active":"2023-11-24T08:09:57.333056+0000","last_peered":"2023-11-24T08:09:57.333056+0000","last_clean":"2023-11-24T08:09:57.333056+0000","last_became_active":"2023-11-24T08:09:56.337414+0000","last_became_peered":"2023-11-24T08:09:56.337414+0000","last_unstale":"2023-11-24T08:09:57.333056+0000","last_undegraded":"2023-11-24T08:09:57.333056+0000","last_fullsized":"2023-11-24T08:09:57.333056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:16.239226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351404+0000","last_change":"2023-11-24T08:09:57.349436+0000","last_active":"2023-11-24T08:09:58.351404+0000","last_peered":"2023-11-24T08:09:58.351404+0000","last_clean":"2023-11-24T08:09:58.351404+0000","last_became_active":"2023-11-24T08:09:57.349144+0000","last_became_peered":"2023-11-24T08:09:57.349144+0000","last_unstale":"2023-11-24T08:09:58.351404+0000","last_undegraded":"2023-11-24T08:09:58.351404+0000","last_fullsized":"2023-11-24T08:09:58.351404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:58.242223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.356911+0000","last_change":"2023-11-24T08:09:56.332402+0000","last_active":"2023-11-24T08:09:57.356911+0000","last_peered":"2023-11-24T08:09:57.356911+0000","last_clean":"2023-11-24T08:09:57.356911+0000","last_became_active":"2023-11-24T08:09:56.332013+0000","last_became_peered":"2023-11-24T08:09:56.332013+0000","last_unstale":"2023-11-24T08:09:57.356911+0000","last_undegraded":"2023-11-24T08:09:57.356911+0000","last_fullsized":"2023-11-24T08:09:57.356911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:03.938076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333768+0000","last_change":"2023-11-24T08:09:57.344032+0000","last_active":"2023-11-24T08:09:58.333768+0000","last_peered":"2023-11-24T08:09:58.333768+0000","last_clean":"2023-11-24T08:09:58.333768+0000","last_became_active":"2023-11-24T08:09:57.343763+0000","last_became_peered":"2023-11-24T08:09:57.343763+0000","last_unstale":"2023-11-24T08:09:58.333768+0000","last_undegraded":"2023-11-24T08:09:58.333768+0000","last_fullsized":"2023-11-24T08:09:58.333768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:40.250323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352338+0000","last_change":"2023-11-24T08:09:56.316753+0000","last_active":"2023-11-24T08:09:58.352338+0000","last_peered":"2023-11-24T08:09:58.352338+0000","last_clean":"2023-11-24T08:09:58.352338+0000","last_became_active":"2023-11-24T08:09:56.316367+0000","last_became_peered":"2023-11-24T08:09:56.316367+0000","last_unstale":"2023-11-24T08:09:58.352338+0000","last_undegraded":"2023-11-24T08:09:58.352338+0000","last_fullsized":"2023-11-24T08:09:58.352338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:21.555016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352091+0000","last_change":"2023-11-24T08:09:57.348112+0000","last_active":"2023-11-24T08:09:58.352091+0000","last_peered":"2023-11-24T08:09:58.352091+0000","last_clean":"2023-11-24T08:09:58.352091+0000","last_became_active":"2023-11-24T08:09:57.347745+0000","last_became_peered":"2023-11-24T08:09:57.347745+0000","last_unstale":"2023-11-24T08:09:58.352091+0000","last_undegraded":"2023-11-24T08:09:58.352091+0000","last_fullsized":"2023-11-24T08:09:58.352091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:04.943808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.330200+0000","last_change":"2023-11-24T08:09:56.330274+0000","last_active":"2023-11-24T08:09:57.330200+0000","last_peered":"2023-11-24T08:09:57.330200+0000","last_clean":"2023-11-24T08:09:57.330200+0000","last_became_active":"2023-11-24T08:09:56.329610+0000","last_became_peered":"2023-11-24T08:09:56.329610+0000","last_unstale":"2023-11-24T08:09:57.330200+0000","last_undegraded":"2023-11-24T08:09:57.330200+0000","last_fullsized":"2023-11-24T08:09:57.330200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:54.111691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333679+0000","last_change":"2023-11-24T08:09:57.335971+0000","last_active":"2023-11-24T08:09:58.333679+0000","last_peered":"2023-11-24T08:09:58.333679+0000","last_clean":"2023-11-24T08:09:58.333679+0000","last_became_active":"2023-11-24T08:09:57.335638+0000","last_became_peered":"2023-11-24T08:09:57.335638+0000","last_unstale":"2023-11-24T08:09:58.333679+0000","last_undegraded":"2023-11-24T08:09:58.333679+0000","last_fullsized":"2023-11-24T08:09:58.333679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:05.116090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352248+0000","last_change":"2023-11-24T08:09:56.329734+0000","last_active":"2023-11-24T08:09:58.352248+0000","last_peered":"2023-11-24T08:09:58.352248+0000","last_clean":"2023-11-24T08:09:58.352248+0000","last_became_active":"2023-11-24T08:09:56.329443+0000","last_became_peered":"2023-11-24T08:09:56.329443+0000","last_unstale":"2023-11-24T08:09:58.352248+0000","last_undegraded":"2023-11-24T08:09:58.352248+0000","last_fullsized":"2023-11-24T08:09:58.352248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:27.993623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352112+0000","last_change":"2023-11-24T08:09:57.341345+0000","last_active":"2023-11-24T08:09:58.352112+0000","last_peered":"2023-11-24T08:09:58.352112+0000","last_clean":"2023-11-24T08:09:58.352112+0000","last_became_active":"2023-11-24T08:09:57.341027+0000","last_became_peered":"2023-11-24T08:09:57.341027+0000","last_unstale":"2023-11-24T08:09:58.352112+0000","last_undegraded":"2023-11-24T08:09:58.352112+0000","last_fullsized":"2023-11-24T08:09:58.352112+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.928845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.329742+0000","last_change":"2023-11-24T08:09:56.335501+0000","last_active":"2023-11-24T08:09:57.329742+0000","last_peered":"2023-11-24T08:09:57.329742+0000","last_clean":"2023-11-24T08:09:57.329742+0000","last_became_active":"2023-11-24T08:09:56.335205+0000","last_became_peered":"2023-11-24T08:09:56.335205+0000","last_unstale":"2023-11-24T08:09:57.329742+0000","last_undegraded":"2023-11-24T08:09:57.329742+0000","last_fullsized":"2023-11-24T08:09:57.329742+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.348908+0000","last_change":"2023-11-24T08:09:57.326634+0000","last_active":"2023-11-24T08:09:58.348908+0000","last_peered":"2023-11-24T08:09:58.348908+0000","last_clean":"2023-11-24T08:09:58.348908+0000","last_became_active":"2023-11-24T08:09:57.325999+0000","last_became_peered":"2023-11-24T08:09:57.325999+0000","last_unstale":"2023-11-24T08:09:58.348908+0000","last_undegraded":"2023-11-24T08:09:58.348908+0000","last_fullsized":"2023-11-24T08:09:58.348908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:01.583026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.331905+0000","last_change":"2023-11-24T08:09:56.335034+0000","last_active":"2023-11-24T08:09:57.331905+0000","last_peered":"2023-11-24T08:09:57.331905+0000","last_clean":"2023-11-24T08:09:57.331905+0000","last_became_active":"2023-11-24T08:09:56.334326+0000","last_became_peered":"2023-11-24T08:09:56.334326+0000","last_unstale":"2023-11-24T08:09:57.331905+0000","last_undegraded":"2023-11-24T08:09:57.331905+0000","last_fullsized":"2023-11-24T08:09:57.331905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350634+0000","last_change":"2023-11-24T08:09:57.351440+0000","last_active":"2023-11-24T08:09:58.350634+0000","last_peered":"2023-11-24T08:09:58.350634+0000","last_clean":"2023-11-24T08:09:58.350634+0000","last_became_active":"2023-11-24T08:09:57.351180+0000","last_became_peered":"2023-11-24T08:09:57.351180+0000","last_unstale":"2023-11-24T08:09:58.350634+0000","last_undegraded":"2023-11-24T08:09:58.350634+0000","last_fullsized":"2023-11-24T08:09:58.350634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:55:37.663117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.307853+0000","last_change":"2023-11-24T08:09:56.337047+0000","last_active":"2023-11-24T08:09:57.307853+0000","last_peered":"2023-11-24T08:09:57.307853+0000","last_clean":"2023-11-24T08:09:57.307853+0000","last_became_active":"2023-11-24T08:09:56.336873+0000","last_became_peered":"2023-11-24T08:09:56.336873+0000","last_unstale":"2023-11-24T08:09:57.307853+0000","last_undegraded":"2023-11-24T08:09:57.307853+0000","last_fullsized":"2023-11-24T08:09:57.307853+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:04.110358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352219+0000","last_change":"2023-11-24T08:09:57.334475+0000","last_active":"2023-11-24T08:09:58.352219+0000","last_peered":"2023-11-24T08:09:58.352219+0000","last_clean":"2023-11-24T08:09:58.352219+0000","last_became_active":"2023-11-24T08:09:57.333265+0000","last_became_peered":"2023-11-24T08:09:57.333265+0000","last_unstale":"2023-11-24T08:09:58.352219+0000","last_undegraded":"2023-11-24T08:09:58.352219+0000","last_fullsized":"2023-11-24T08:09:58.352219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:41.039861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.311778+0000","last_change":"2023-11-24T08:09:56.332890+0000","last_active":"2023-11-24T08:09:57.311778+0000","last_peered":"2023-11-24T08:09:57.311778+0000","last_clean":"2023-11-24T08:09:57.311778+0000","last_became_active":"2023-11-24T08:09:56.332562+0000","last_became_peered":"2023-11-24T08:09:56.332562+0000","last_unstale":"2023-11-24T08:09:57.311778+0000","last_undegraded":"2023-11-24T08:09:57.311778+0000","last_fullsized":"2023-11-24T08:09:57.311778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:27.438236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":177,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.315434+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:09:57.315434+0000","last_peered":"2023-11-24T08:09:57.315434+0000","last_clean":"2023-11-24T08:09:57.315434+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:09:57.315434+0000","last_undegraded":"2023-11-24T08:09:57.315434+0000","last_fullsized":"2023-11-24T08:09:57.315434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1003520,"data_stored":750485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.141},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.0740000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.643},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.246}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.536},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.353}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.7090000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.446},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.76}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":413696,"data_stored":160643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.3640000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.0800000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.85699999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.145},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.512}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.4810000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.25},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.4410000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.536},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.5629999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":160699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.4359999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.999},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5900000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.546},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6870000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.766}]},{"osd":6,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":2.121},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.728}]},{"osd":7,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.752,"15min":1.752},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.9689999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":2.1989999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":160131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.05}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.329},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.097}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.212},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.1830000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4410000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":161661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.88},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.755}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.208}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.407},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.0129999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.494}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.248},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":0.86099999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":160097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.96599999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.327},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.27}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.536}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.51}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":421888,"data_stored":161661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.75}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.518},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.4159999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":750395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2010000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.119},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.345}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.4319999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.159},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.3140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:01.724 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:01.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.973+0000 7f690ffff640 1 Processor -- start 2023-11-24T08:10:01.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.974+0000 7f690ffff640 1 -- start start 2023-11-24T08:10:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.975+0000 7f690ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f691015c410 0x7f691015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f690ffff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691015bf80 con 0x7f69100b1de0 2023-11-24T08:10:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f690ffff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69100b1560 con 0x7f6910144b30 2023-11-24T08:10:01.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f690ffff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69100b1280 con 0x7f691015c410 2023-11-24T08:10:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f6907fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f691015c410 0x7f691015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f6907fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f691015c410 0x7f691015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49358/0 (socket says 172.21.15.106:49358) 2023-11-24T08:10:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f6907fff640 1 -- 172.21.15.106:0/321451887 learned_addr learned my addr 172.21.15.106:0/321451887 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f690f7fe640 1 --1- 172.21.15.106:0/321451887 >> v1:172.21.15.106:6789/0 conn(0x7f69100b1de0 0x7f69100b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39380/0 (socket says 172.21.15.106:39380) 2023-11-24T08:10:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f6907fff640 1 -- 172.21.15.106:0/321451887 >> v1:172.21.15.156:6789/0 conn(0x7f6910144b30 legacy=0x7f69100b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:01.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f6907fff640 1 -- 172.21.15.106:0/321451887 >> v1:172.21.15.106:6789/0 conn(0x7f69100b1de0 legacy=0x7f69100b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.976+0000 7f6907fff640 1 -- 172.21.15.106:0/321451887 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691015b930 con 0x7f691015c410 2023-11-24T08:10:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.977+0000 7f6907fff640 1 --2- 172.21.15.106:0/321451887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f691015c410 0x7f691015f920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009870 tx=0x7f68f802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55da45dd82d9e930 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690e7fc640 1 -- 172.21.15.106:0/321451887 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8030860 con 0x7f691015c410 2023-11-24T08:10:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690e7fc640 1 -- 172.21.15.106:0/321451887 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f80309c0 con 0x7f691015c410 2023-11-24T08:10:01.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690e7fc640 1 -- 172.21.15.106:0/321451887 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f8030d50 con 0x7f691015c410 2023-11-24T08:10:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690ffff640 1 -- 172.21.15.106:0/321451887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f691015c410 msgr2=0x7f691015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690ffff640 1 --2- 172.21.15.106:0/321451887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f691015c410 0x7f691015f920 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009870 tx=0x7f68f802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690ffff640 1 -- 172.21.15.106:0/321451887 shutdown_connections 2023-11-24T08:10:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690ffff640 1 --2- 172.21.15.106:0/321451887 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f691015c410 0x7f691015f920 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.978+0000 7f690ffff640 1 -- 172.21.15.106:0/321451887 >> 172.21.15.106:0/321451887 conn(0x7f69100cb830 msgr2=0x7f69100cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:01.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.979+0000 7f690ffff640 1 -- 172.21.15.106:0/321451887 shutdown_connections 2023-11-24T08:10:01.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.979+0000 7f690ffff640 1 -- 172.21.15.106:0/321451887 wait complete. 2023-11-24T08:10:01.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.980+0000 7f690ffff640 1 Processor -- start 2023-11-24T08:10:01.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.980+0000 7f690ffff640 1 -- start start 2023-11-24T08:10:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.981+0000 7f690ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 0x7f691013c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.981+0000 7f690ffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6910144b30 0x7f69100c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.981+0000 7f690ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f691015c410 0x7f691013f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.981+0000 7f690ffff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c6190 con 0x7f69100b1de0 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.981+0000 7f690ffff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6910141c90 con 0x7f691015c410 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.981+0000 7f690ffff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69101427d0 con 0x7f6910144b30 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 0x7f691013c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 0x7f691013c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49364/0 (socket says 172.21.15.106:49364) 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 -- 172.21.15.106:0/243104778 learned_addr learned my addr 172.21.15.106:0/243104778 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 -- 172.21.15.106:0/243104778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6910144b30 msgr2=0x7f69100c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6910144b30 0x7f69100c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 -- 172.21.15.106:0/243104778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f691015c410 msgr2=0x7f691013f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f691015c410 0x7f691013f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:01.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 -- 172.21.15.106:0/243104778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80094d0 con 0x7f69100b1de0 2023-11-24T08:10:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f690effd640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 0x7f691013c890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f690000b3f0 tx=0x7f690000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.982+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011830 con 0x7f69100b1de0 2023-11-24T08:10:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.983+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900011990 con 0x7f69100b1de0 2023-11-24T08:10:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.983+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900011d40 con 0x7f69100b1de0 2023-11-24T08:10:01.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.983+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6910141ea0 con 0x7f69100b1de0 2023-11-24T08:10:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.983+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6910142460 con 0x7f69100b1de0 2023-11-24T08:10:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.984+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f691004e840 con 0x7f69100b1de0 2023-11-24T08:10:01.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.985+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f690001a400 con 0x7f69100b1de0 2023-11-24T08:10:01.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.985+0000 7f69077fe640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f68f006ae40 0x7f68f006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:01.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.986+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69000894d0 con 0x7f69100b1de0 2023-11-24T08:10:01.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.986+0000 7f6907fff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f68f006ae40 0x7f68f006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:01.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.987+0000 7f6907fff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f68f006ae40 0x7f68f006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68f8036000 tx=0x7f68f803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:01.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:01.990+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f69000544e0 con 0x7f69100b1de0 2023-11-24T08:10:02.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.178+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6910142910 con 0x7f68f006ae40 2023-11-24T08:10:02.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.199+0000 7f69077fe640 1 -- 172.21.15.106:0/243104778 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286816 (secure 0 0 0) 0x7f68f8030b30 con 0x7f68f006ae40 2023-11-24T08:10:02.205 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:02.207 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:02.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f68f006ae40 msgr2=0x7f68f006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f68f006ae40 0x7f68f006d300 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f68f8036000 tx=0x7f68f803b040 comp rx=0 tx=0).stop 2023-11-24T08:10:02.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 msgr2=0x7f691013c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 0x7f691013c890 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f690000b3f0 tx=0x7f690000b8b0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 shutdown_connections 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f691015c410 0x7f691013f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f68f006ae40 0x7f68f006d300 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6910144b30 0x7f69100c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 --2- 172.21.15.106:0/243104778 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f69100b1de0 0x7f691013c890 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 >> 172.21.15.106:0/243104778 conn(0x7f69100cb830 msgr2=0x7f69100cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 shutdown_connections 2023-11-24T08:10:02.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.205+0000 7f690ffff640 1 -- 172.21.15.106:0/243104778 wait complete. 2023-11-24T08:10:02.226 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2023-11-24T08:10:01.295452+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":237240,"kb_used_data":4392,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754737480,"statfs":{"total":773094113280,"available":772851179520,"internally_reserved":0,"allocated":4497408,"data_stored":2465772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":43,"apply_latency_ms":43,"commit_latency_ns":43000000,"apply_latency_ns":43000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001900"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333991+0000","last_change":"2023-11-24T08:09:57.339594+0000","last_active":"2023-11-24T08:09:58.333991+0000","last_peered":"2023-11-24T08:09:58.333991+0000","last_clean":"2023-11-24T08:09:58.333991+0000","last_became_active":"2023-11-24T08:09:57.339301+0000","last_became_peered":"2023-11-24T08:09:57.339301+0000","last_unstale":"2023-11-24T08:09:58.333991+0000","last_undegraded":"2023-11-24T08:09:58.333991+0000","last_fullsized":"2023-11-24T08:09:58.333991+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:05.820823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349514+0000","last_change":"2023-11-24T08:09:57.348745+0000","last_active":"2023-11-24T08:09:58.349514+0000","last_peered":"2023-11-24T08:09:58.349514+0000","last_clean":"2023-11-24T08:09:58.349514+0000","last_became_active":"2023-11-24T08:09:57.344690+0000","last_became_peered":"2023-11-24T08:09:57.344690+0000","last_unstale":"2023-11-24T08:09:58.349514+0000","last_undegraded":"2023-11-24T08:09:58.349514+0000","last_fullsized":"2023-11-24T08:09:58.349514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:32.378462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332830+0000","last_change":"2023-11-24T08:09:57.345052+0000","last_active":"2023-11-24T08:09:58.332830+0000","last_peered":"2023-11-24T08:09:58.332830+0000","last_clean":"2023-11-24T08:09:58.332830+0000","last_became_active":"2023-11-24T08:09:57.344742+0000","last_became_peered":"2023-11-24T08:09:57.344742+0000","last_unstale":"2023-11-24T08:09:58.332830+0000","last_undegraded":"2023-11-24T08:09:58.332830+0000","last_fullsized":"2023-11-24T08:09:58.332830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:48.577263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330861+0000","last_change":"2023-11-24T08:09:57.345287+0000","last_active":"2023-11-24T08:09:58.330861+0000","last_peered":"2023-11-24T08:09:58.330861+0000","last_clean":"2023-11-24T08:09:58.330861+0000","last_became_active":"2023-11-24T08:09:57.344969+0000","last_became_peered":"2023-11-24T08:09:57.344969+0000","last_unstale":"2023-11-24T08:09:58.330861+0000","last_undegraded":"2023-11-24T08:09:58.330861+0000","last_fullsized":"2023-11-24T08:09:58.330861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:06.577748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349687+0000","last_change":"2023-11-24T08:09:57.340128+0000","last_active":"2023-11-24T08:09:58.349687+0000","last_peered":"2023-11-24T08:09:58.349687+0000","last_clean":"2023-11-24T08:09:58.349687+0000","last_became_active":"2023-11-24T08:09:57.339007+0000","last_became_peered":"2023-11-24T08:09:57.339007+0000","last_unstale":"2023-11-24T08:09:58.349687+0000","last_undegraded":"2023-11-24T08:09:58.349687+0000","last_fullsized":"2023-11-24T08:09:58.349687+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:48:18.665823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349392+0000","last_change":"2023-11-24T08:09:57.349184+0000","last_active":"2023-11-24T08:09:58.349392+0000","last_peered":"2023-11-24T08:09:58.349392+0000","last_clean":"2023-11-24T08:09:58.349392+0000","last_became_active":"2023-11-24T08:09:57.345791+0000","last_became_peered":"2023-11-24T08:09:57.345791+0000","last_unstale":"2023-11-24T08:09:58.349392+0000","last_undegraded":"2023-11-24T08:09:58.349392+0000","last_fullsized":"2023-11-24T08:09:58.349392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:23.964175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351812+0000","last_change":"2023-11-24T08:09:57.334549+0000","last_active":"2023-11-24T08:09:58.351812+0000","last_peered":"2023-11-24T08:09:58.351812+0000","last_clean":"2023-11-24T08:09:58.351812+0000","last_became_active":"2023-11-24T08:09:57.332014+0000","last_became_peered":"2023-11-24T08:09:57.332014+0000","last_unstale":"2023-11-24T08:09:58.351812+0000","last_undegraded":"2023-11-24T08:09:58.351812+0000","last_fullsized":"2023-11-24T08:09:58.351812+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:01.583026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349688+0000","last_change":"2023-11-24T08:09:57.334754+0000","last_active":"2023-11-24T08:09:58.349688+0000","last_peered":"2023-11-24T08:09:58.349688+0000","last_clean":"2023-11-24T08:09:58.349688+0000","last_became_active":"2023-11-24T08:09:57.334508+0000","last_became_peered":"2023-11-24T08:09:57.334508+0000","last_unstale":"2023-11-24T08:09:58.349688+0000","last_undegraded":"2023-11-24T08:09:58.349688+0000","last_fullsized":"2023-11-24T08:09:58.349688+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:41.487984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352676+0000","last_change":"2023-11-24T08:09:57.344389+0000","last_active":"2023-11-24T08:09:58.352676+0000","last_peered":"2023-11-24T08:09:58.352676+0000","last_clean":"2023-11-24T08:09:58.352676+0000","last_became_active":"2023-11-24T08:09:57.344085+0000","last_became_peered":"2023-11-24T08:09:57.344085+0000","last_unstale":"2023-11-24T08:09:58.352676+0000","last_undegraded":"2023-11-24T08:09:58.352676+0000","last_fullsized":"2023-11-24T08:09:58.352676+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:21.341963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331334+0000","last_change":"2023-11-24T08:09:57.347866+0000","last_active":"2023-11-24T08:09:58.331334+0000","last_peered":"2023-11-24T08:09:58.331334+0000","last_clean":"2023-11-24T08:09:58.331334+0000","last_became_active":"2023-11-24T08:09:57.347598+0000","last_became_peered":"2023-11-24T08:09:57.347598+0000","last_unstale":"2023-11-24T08:09:58.331334+0000","last_undegraded":"2023-11-24T08:09:58.331334+0000","last_fullsized":"2023-11-24T08:09:58.331334+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:36.347389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352472+0000","last_change":"2023-11-24T08:09:57.348482+0000","last_active":"2023-11-24T08:09:58.352472+0000","last_peered":"2023-11-24T08:09:58.352472+0000","last_clean":"2023-11-24T08:09:58.352472+0000","last_became_active":"2023-11-24T08:09:57.348212+0000","last_became_peered":"2023-11-24T08:09:57.348212+0000","last_unstale":"2023-11-24T08:09:58.352472+0000","last_undegraded":"2023-11-24T08:09:58.352472+0000","last_fullsized":"2023-11-24T08:09:58.352472+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:49:32.378462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332427+0000","last_change":"2023-11-24T08:09:57.350385+0000","last_active":"2023-11-24T08:09:58.332427+0000","last_peered":"2023-11-24T08:09:58.332427+0000","last_clean":"2023-11-24T08:09:58.332427+0000","last_became_active":"2023-11-24T08:09:57.350147+0000","last_became_peered":"2023-11-24T08:09:57.350147+0000","last_unstale":"2023-11-24T08:09:58.332427+0000","last_undegraded":"2023-11-24T08:09:58.332427+0000","last_fullsized":"2023-11-24T08:09:58.332427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:34.608636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349829+0000","last_change":"2023-11-24T08:09:57.333929+0000","last_active":"2023-11-24T08:09:58.349829+0000","last_peered":"2023-11-24T08:09:58.349829+0000","last_clean":"2023-11-24T08:09:58.349829+0000","last_became_active":"2023-11-24T08:09:57.333536+0000","last_became_peered":"2023-11-24T08:09:57.333536+0000","last_unstale":"2023-11-24T08:09:58.349829+0000","last_undegraded":"2023-11-24T08:09:58.349829+0000","last_fullsized":"2023-11-24T08:09:58.349829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:51:18.149278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349719+0000","last_change":"2023-11-24T08:09:57.334678+0000","last_active":"2023-11-24T08:09:58.349719+0000","last_peered":"2023-11-24T08:09:58.349719+0000","last_clean":"2023-11-24T08:09:58.349719+0000","last_became_active":"2023-11-24T08:09:57.334450+0000","last_became_peered":"2023-11-24T08:09:57.334450+0000","last_unstale":"2023-11-24T08:09:58.349719+0000","last_undegraded":"2023-11-24T08:09:58.349719+0000","last_fullsized":"2023-11-24T08:09:58.349719+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:46:39.725826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331838+0000","last_change":"2023-11-24T08:09:57.342912+0000","last_active":"2023-11-24T08:09:58.331838+0000","last_peered":"2023-11-24T08:09:58.331838+0000","last_clean":"2023-11-24T08:09:58.331838+0000","last_became_active":"2023-11-24T08:09:57.342606+0000","last_became_peered":"2023-11-24T08:09:57.342606+0000","last_unstale":"2023-11-24T08:09:58.331838+0000","last_undegraded":"2023-11-24T08:09:58.331838+0000","last_fullsized":"2023-11-24T08:09:58.331838+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:03:18.248284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350352+0000","last_change":"2023-11-24T08:09:57.338833+0000","last_active":"2023-11-24T08:09:58.350352+0000","last_peered":"2023-11-24T08:09:58.350352+0000","last_clean":"2023-11-24T08:09:58.350352+0000","last_became_active":"2023-11-24T08:09:57.338332+0000","last_became_peered":"2023-11-24T08:09:57.338332+0000","last_unstale":"2023-11-24T08:09:58.350352+0000","last_undegraded":"2023-11-24T08:09:58.350352+0000","last_fullsized":"2023-11-24T08:09:58.350352+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:19.185246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352778+0000","last_change":"2023-11-24T08:09:57.328071+0000","last_active":"2023-11-24T08:09:58.352778+0000","last_peered":"2023-11-24T08:09:58.352778+0000","last_clean":"2023-11-24T08:09:58.352778+0000","last_became_active":"2023-11-24T08:09:57.327695+0000","last_became_peered":"2023-11-24T08:09:57.327695+0000","last_unstale":"2023-11-24T08:09:58.352778+0000","last_undegraded":"2023-11-24T08:09:58.352778+0000","last_fullsized":"2023-11-24T08:09:58.352778+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:29.840340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349200+0000","last_change":"2023-11-24T08:09:57.336519+0000","last_active":"2023-11-24T08:09:58.349200+0000","last_peered":"2023-11-24T08:09:58.349200+0000","last_clean":"2023-11-24T08:09:58.349200+0000","last_became_active":"2023-11-24T08:09:57.336296+0000","last_became_peered":"2023-11-24T08:09:57.336296+0000","last_unstale":"2023-11-24T08:09:58.349200+0000","last_undegraded":"2023-11-24T08:09:58.349200+0000","last_fullsized":"2023-11-24T08:09:58.349200+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:10.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350048+0000","last_change":"2023-11-24T08:09:57.348089+0000","last_active":"2023-11-24T08:09:58.350048+0000","last_peered":"2023-11-24T08:09:58.350048+0000","last_clean":"2023-11-24T08:09:58.350048+0000","last_became_active":"2023-11-24T08:09:57.347837+0000","last_became_peered":"2023-11-24T08:09:57.347837+0000","last_unstale":"2023-11-24T08:09:58.350048+0000","last_undegraded":"2023-11-24T08:09:58.350048+0000","last_fullsized":"2023-11-24T08:09:58.350048+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:27:04.437872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350628+0000","last_change":"2023-11-24T08:09:57.340443+0000","last_active":"2023-11-24T08:09:58.350628+0000","last_peered":"2023-11-24T08:09:58.350628+0000","last_clean":"2023-11-24T08:09:58.350628+0000","last_became_active":"2023-11-24T08:09:57.340222+0000","last_became_peered":"2023-11-24T08:09:57.340222+0000","last_unstale":"2023-11-24T08:09:58.350628+0000","last_undegraded":"2023-11-24T08:09:58.350628+0000","last_fullsized":"2023-11-24T08:09:58.350628+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:28.443968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350506+0000","last_change":"2023-11-24T08:09:57.347587+0000","last_active":"2023-11-24T08:09:58.350506+0000","last_peered":"2023-11-24T08:09:58.350506+0000","last_clean":"2023-11-24T08:09:58.350506+0000","last_became_active":"2023-11-24T08:09:57.345081+0000","last_became_peered":"2023-11-24T08:09:57.345081+0000","last_unstale":"2023-11-24T08:09:58.350506+0000","last_undegraded":"2023-11-24T08:09:58.350506+0000","last_fullsized":"2023-11-24T08:09:58.350506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:54.576837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350353+0000","last_change":"2023-11-24T08:09:57.346560+0000","last_active":"2023-11-24T08:09:58.350353+0000","last_peered":"2023-11-24T08:09:58.350353+0000","last_clean":"2023-11-24T08:09:58.350353+0000","last_became_active":"2023-11-24T08:09:57.346267+0000","last_became_peered":"2023-11-24T08:09:57.346267+0000","last_unstale":"2023-11-24T08:09:58.350353+0000","last_undegraded":"2023-11-24T08:09:58.350353+0000","last_fullsized":"2023-11-24T08:09:58.350353+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:31.108048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351797+0000","last_change":"2023-11-24T08:09:57.344265+0000","last_active":"2023-11-24T08:09:58.351797+0000","last_peered":"2023-11-24T08:09:58.351797+0000","last_clean":"2023-11-24T08:09:58.351797+0000","last_became_active":"2023-11-24T08:09:57.344027+0000","last_became_peered":"2023-11-24T08:09:57.344027+0000","last_unstale":"2023-11-24T08:09:58.351797+0000","last_undegraded":"2023-11-24T08:09:58.351797+0000","last_fullsized":"2023-11-24T08:09:58.351797+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:55.117423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331659+0000","last_change":"2023-11-24T08:09:57.337893+0000","last_active":"2023-11-24T08:09:58.331659+0000","last_peered":"2023-11-24T08:09:58.331659+0000","last_clean":"2023-11-24T08:09:58.331659+0000","last_became_active":"2023-11-24T08:09:57.336317+0000","last_became_peered":"2023-11-24T08:09:57.336317+0000","last_unstale":"2023-11-24T08:09:58.331659+0000","last_undegraded":"2023-11-24T08:09:58.331659+0000","last_fullsized":"2023-11-24T08:09:58.331659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:08:48.148109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331469+0000","last_change":"2023-11-24T08:09:57.336201+0000","last_active":"2023-11-24T08:09:58.331469+0000","last_peered":"2023-11-24T08:09:58.331469+0000","last_clean":"2023-11-24T08:09:58.331469+0000","last_became_active":"2023-11-24T08:09:57.335896+0000","last_became_peered":"2023-11-24T08:09:57.335896+0000","last_unstale":"2023-11-24T08:09:58.331469+0000","last_undegraded":"2023-11-24T08:09:58.331469+0000","last_fullsized":"2023-11-24T08:09:58.331469+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:41.487984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349333+0000","last_change":"2023-11-24T08:09:57.327968+0000","last_active":"2023-11-24T08:09:58.349333+0000","last_peered":"2023-11-24T08:09:58.349333+0000","last_clean":"2023-11-24T08:09:58.349333+0000","last_became_active":"2023-11-24T08:09:57.327666+0000","last_became_peered":"2023-11-24T08:09:57.327666+0000","last_unstale":"2023-11-24T08:09:58.349333+0000","last_undegraded":"2023-11-24T08:09:58.349333+0000","last_fullsized":"2023-11-24T08:09:58.349333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:41.039861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.331861+0000","last_change":"2023-11-24T08:09:57.342058+0000","last_active":"2023-11-24T08:09:58.331861+0000","last_peered":"2023-11-24T08:09:58.331861+0000","last_clean":"2023-11-24T08:09:58.331861+0000","last_became_active":"2023-11-24T08:09:57.341681+0000","last_became_peered":"2023-11-24T08:09:57.341681+0000","last_unstale":"2023-11-24T08:09:58.331861+0000","last_undegraded":"2023-11-24T08:09:58.331861+0000","last_fullsized":"2023-11-24T08:09:58.331861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:21.341963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332508+0000","last_change":"2023-11-24T08:09:57.347744+0000","last_active":"2023-11-24T08:09:58.332508+0000","last_peered":"2023-11-24T08:09:58.332508+0000","last_clean":"2023-11-24T08:09:58.332508+0000","last_became_active":"2023-11-24T08:09:57.347529+0000","last_became_peered":"2023-11-24T08:09:57.347529+0000","last_unstale":"2023-11-24T08:09:58.332508+0000","last_undegraded":"2023-11-24T08:09:58.332508+0000","last_fullsized":"2023-11-24T08:09:58.332508+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:25:52.692767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350961+0000","last_change":"2023-11-24T08:09:57.341318+0000","last_active":"2023-11-24T08:09:58.350961+0000","last_peered":"2023-11-24T08:09:58.350961+0000","last_clean":"2023-11-24T08:09:58.350961+0000","last_became_active":"2023-11-24T08:09:57.340942+0000","last_became_peered":"2023-11-24T08:09:57.340942+0000","last_unstale":"2023-11-24T08:09:58.350961+0000","last_undegraded":"2023-11-24T08:09:58.350961+0000","last_fullsized":"2023-11-24T08:09:58.350961+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:06.577748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351950+0000","last_change":"2023-11-24T08:09:57.329124+0000","last_active":"2023-11-24T08:09:58.351950+0000","last_peered":"2023-11-24T08:09:58.351950+0000","last_clean":"2023-11-24T08:09:58.351950+0000","last_became_active":"2023-11-24T08:09:57.328498+0000","last_became_peered":"2023-11-24T08:09:57.328498+0000","last_unstale":"2023-11-24T08:09:58.351950+0000","last_undegraded":"2023-11-24T08:09:58.351950+0000","last_fullsized":"2023-11-24T08:09:58.351950+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:08:10.513879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333337+0000","last_change":"2023-11-24T08:09:57.336739+0000","last_active":"2023-11-24T08:09:58.333337+0000","last_peered":"2023-11-24T08:09:58.333337+0000","last_clean":"2023-11-24T08:09:58.333337+0000","last_became_active":"2023-11-24T08:09:57.336445+0000","last_became_peered":"2023-11-24T08:09:57.336445+0000","last_unstale":"2023-11-24T08:09:58.333337+0000","last_undegraded":"2023-11-24T08:09:58.333337+0000","last_fullsized":"2023-11-24T08:09:58.333337+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:51.848914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349779+0000","last_change":"2023-11-24T08:09:57.338607+0000","last_active":"2023-11-24T08:09:58.349779+0000","last_peered":"2023-11-24T08:09:58.349779+0000","last_clean":"2023-11-24T08:09:58.349779+0000","last_became_active":"2023-11-24T08:09:57.337741+0000","last_became_peered":"2023-11-24T08:09:57.337741+0000","last_unstale":"2023-11-24T08:09:58.349779+0000","last_undegraded":"2023-11-24T08:09:58.349779+0000","last_fullsized":"2023-11-24T08:09:58.349779+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:21.341963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352161+0000","last_change":"2023-11-24T08:09:57.342705+0000","last_active":"2023-11-24T08:09:58.352161+0000","last_peered":"2023-11-24T08:09:58.352161+0000","last_clean":"2023-11-24T08:09:58.352161+0000","last_became_active":"2023-11-24T08:09:57.342412+0000","last_became_peered":"2023-11-24T08:09:57.342412+0000","last_unstale":"2023-11-24T08:09:58.352161+0000","last_undegraded":"2023-11-24T08:09:58.352161+0000","last_fullsized":"2023-11-24T08:09:58.352161+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:31.108048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"35'4","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.356102+0000","last_change":"2023-11-24T08:09:56.335055+0000","last_active":"2023-11-24T08:09:57.356102+0000","last_peered":"2023-11-24T08:09:57.356102+0000","last_clean":"2023-11-24T08:09:57.356102+0000","last_became_active":"2023-11-24T08:09:56.333890+0000","last_became_peered":"2023-11-24T08:09:56.333890+0000","last_unstale":"2023-11-24T08:09:57.356102+0000","last_undegraded":"2023-11-24T08:09:57.356102+0000","last_fullsized":"2023-11-24T08:09:57.356102+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:27.338331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333830+0000","last_change":"2023-11-24T08:09:57.337726+0000","last_active":"2023-11-24T08:09:58.333830+0000","last_peered":"2023-11-24T08:09:58.333830+0000","last_clean":"2023-11-24T08:09:58.333830+0000","last_became_active":"2023-11-24T08:09:57.337408+0000","last_became_peered":"2023-11-24T08:09:57.337408+0000","last_unstale":"2023-11-24T08:09:58.333830+0000","last_undegraded":"2023-11-24T08:09:58.333830+0000","last_fullsized":"2023-11-24T08:09:58.333830+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:22.560748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"35'1","reported_seq":13,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352258+0000","last_change":"2023-11-24T08:09:56.334036+0000","last_active":"2023-11-24T08:09:58.352258+0000","last_peered":"2023-11-24T08:09:58.352258+0000","last_clean":"2023-11-24T08:09:58.352258+0000","last_became_active":"2023-11-24T08:09:56.333585+0000","last_became_peered":"2023-11-24T08:09:56.333585+0000","last_unstale":"2023-11-24T08:09:58.352258+0000","last_undegraded":"2023-11-24T08:09:58.352258+0000","last_fullsized":"2023-11-24T08:09:58.352258+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:04.110358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351362+0000","last_change":"2023-11-24T08:09:57.346217+0000","last_active":"2023-11-24T08:09:58.351362+0000","last_peered":"2023-11-24T08:09:58.351362+0000","last_clean":"2023-11-24T08:09:58.351362+0000","last_became_active":"2023-11-24T08:09:57.345105+0000","last_became_peered":"2023-11-24T08:09:57.345105+0000","last_unstale":"2023-11-24T08:09:58.351362+0000","last_undegraded":"2023-11-24T08:09:58.351362+0000","last_fullsized":"2023-11-24T08:09:58.351362+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:36.347389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.358219+0000","last_change":"2023-11-24T08:09:56.326820+0000","last_active":"2023-11-24T08:09:57.358219+0000","last_peered":"2023-11-24T08:09:57.358219+0000","last_clean":"2023-11-24T08:09:57.358219+0000","last_became_active":"2023-11-24T08:09:56.326554+0000","last_became_peered":"2023-11-24T08:09:56.326554+0000","last_unstale":"2023-11-24T08:09:57.358219+0000","last_undegraded":"2023-11-24T08:09:57.358219+0000","last_fullsized":"2023-11-24T08:09:57.358219+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:21.555016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332148+0000","last_change":"2023-11-24T08:09:57.342008+0000","last_active":"2023-11-24T08:09:58.332148+0000","last_peered":"2023-11-24T08:09:58.332148+0000","last_clean":"2023-11-24T08:09:58.332148+0000","last_became_active":"2023-11-24T08:09:57.341358+0000","last_became_peered":"2023-11-24T08:09:57.341358+0000","last_unstale":"2023-11-24T08:09:58.332148+0000","last_undegraded":"2023-11-24T08:09:58.332148+0000","last_fullsized":"2023-11-24T08:09:58.332148+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:32.521394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.357872+0000","last_change":"2023-11-24T08:09:56.331123+0000","last_active":"2023-11-24T08:09:57.357872+0000","last_peered":"2023-11-24T08:09:57.357872+0000","last_clean":"2023-11-24T08:09:57.357872+0000","last_became_active":"2023-11-24T08:09:56.326470+0000","last_became_peered":"2023-11-24T08:09:56.326470+0000","last_unstale":"2023-11-24T08:09:57.357872+0000","last_undegraded":"2023-11-24T08:09:57.357872+0000","last_fullsized":"2023-11-24T08:09:57.357872+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:56:24.340594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352316+0000","last_change":"2023-11-24T08:09:57.338671+0000","last_active":"2023-11-24T08:09:58.352316+0000","last_peered":"2023-11-24T08:09:58.352316+0000","last_clean":"2023-11-24T08:09:58.352316+0000","last_became_active":"2023-11-24T08:09:57.338343+0000","last_became_peered":"2023-11-24T08:09:57.338343+0000","last_unstale":"2023-11-24T08:09:58.352316+0000","last_undegraded":"2023-11-24T08:09:58.352316+0000","last_fullsized":"2023-11-24T08:09:58.352316+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:06:23.964175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.357995+0000","last_change":"2023-11-24T08:09:56.329938+0000","last_active":"2023-11-24T08:09:57.357995+0000","last_peered":"2023-11-24T08:09:57.357995+0000","last_clean":"2023-11-24T08:09:57.357995+0000","last_became_active":"2023-11-24T08:09:56.327630+0000","last_became_peered":"2023-11-24T08:09:56.327630+0000","last_unstale":"2023-11-24T08:09:57.357995+0000","last_undegraded":"2023-11-24T08:09:57.357995+0000","last_fullsized":"2023-11-24T08:09:57.357995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:21.555016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332299+0000","last_change":"2023-11-24T08:09:57.347128+0000","last_active":"2023-11-24T08:09:58.332299+0000","last_peered":"2023-11-24T08:09:58.332299+0000","last_clean":"2023-11-24T08:09:58.332299+0000","last_became_active":"2023-11-24T08:09:57.346788+0000","last_became_peered":"2023-11-24T08:09:57.346788+0000","last_unstale":"2023-11-24T08:09:58.332299+0000","last_undegraded":"2023-11-24T08:09:58.332299+0000","last_fullsized":"2023-11-24T08:09:58.332299+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:42.456620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.358072+0000","last_change":"2023-11-24T08:09:56.326659+0000","last_active":"2023-11-24T08:09:57.358072+0000","last_peered":"2023-11-24T08:09:57.358072+0000","last_clean":"2023-11-24T08:09:57.358072+0000","last_became_active":"2023-11-24T08:09:56.326157+0000","last_became_peered":"2023-11-24T08:09:56.326157+0000","last_unstale":"2023-11-24T08:09:57.358072+0000","last_undegraded":"2023-11-24T08:09:57.358072+0000","last_fullsized":"2023-11-24T08:09:57.358072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:18.292579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350023+0000","last_change":"2023-11-24T08:09:57.333696+0000","last_active":"2023-11-24T08:09:58.350023+0000","last_peered":"2023-11-24T08:09:58.350023+0000","last_clean":"2023-11-24T08:09:58.350023+0000","last_became_active":"2023-11-24T08:09:57.333416+0000","last_became_peered":"2023-11-24T08:09:57.333416+0000","last_unstale":"2023-11-24T08:09:58.350023+0000","last_undegraded":"2023-11-24T08:09:58.350023+0000","last_fullsized":"2023-11-24T08:09:58.350023+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.928845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.308635+0000","last_change":"2023-11-24T08:09:56.328940+0000","last_active":"2023-11-24T08:09:57.308635+0000","last_peered":"2023-11-24T08:09:57.308635+0000","last_clean":"2023-11-24T08:09:57.308635+0000","last_became_active":"2023-11-24T08:09:56.328528+0000","last_became_peered":"2023-11-24T08:09:56.328528+0000","last_unstale":"2023-11-24T08:09:57.308635+0000","last_undegraded":"2023-11-24T08:09:57.308635+0000","last_fullsized":"2023-11-24T08:09:57.308635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:30.102315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330406+0000","last_change":"2023-11-24T08:09:57.339427+0000","last_active":"2023-11-24T08:09:58.330406+0000","last_peered":"2023-11-24T08:09:58.330406+0000","last_clean":"2023-11-24T08:09:58.330406+0000","last_became_active":"2023-11-24T08:09:57.339122+0000","last_became_peered":"2023-11-24T08:09:57.339122+0000","last_unstale":"2023-11-24T08:09:58.330406+0000","last_undegraded":"2023-11-24T08:09:58.330406+0000","last_fullsized":"2023-11-24T08:09:58.330406+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:49:31.108048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.325896+0000","last_change":"2023-11-24T08:09:56.323625+0000","last_active":"2023-11-24T08:09:57.325896+0000","last_peered":"2023-11-24T08:09:57.325896+0000","last_clean":"2023-11-24T08:09:57.325896+0000","last_became_active":"2023-11-24T08:09:56.323161+0000","last_became_peered":"2023-11-24T08:09:56.323161+0000","last_unstale":"2023-11-24T08:09:57.325896+0000","last_undegraded":"2023-11-24T08:09:57.325896+0000","last_fullsized":"2023-11-24T08:09:57.325896+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:57.236491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333032+0000","last_change":"2023-11-24T08:09:57.349505+0000","last_active":"2023-11-24T08:09:58.333032+0000","last_peered":"2023-11-24T08:09:58.333032+0000","last_clean":"2023-11-24T08:09:58.333032+0000","last_became_active":"2023-11-24T08:09:57.349249+0000","last_became_peered":"2023-11-24T08:09:57.349249+0000","last_unstale":"2023-11-24T08:09:58.333032+0000","last_undegraded":"2023-11-24T08:09:58.333032+0000","last_fullsized":"2023-11-24T08:09:58.333032+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:24.386449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.309129+0000","last_change":"2023-11-24T08:09:56.335469+0000","last_active":"2023-11-24T08:09:57.309129+0000","last_peered":"2023-11-24T08:09:57.309129+0000","last_clean":"2023-11-24T08:09:57.309129+0000","last_became_active":"2023-11-24T08:09:56.335167+0000","last_became_peered":"2023-11-24T08:09:56.335167+0000","last_unstale":"2023-11-24T08:09:57.309129+0000","last_undegraded":"2023-11-24T08:09:57.309129+0000","last_fullsized":"2023-11-24T08:09:57.309129+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:53.571105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350342+0000","last_change":"2023-11-24T08:09:57.346589+0000","last_active":"2023-11-24T08:09:58.350342+0000","last_peered":"2023-11-24T08:09:58.350342+0000","last_clean":"2023-11-24T08:09:58.350342+0000","last_became_active":"2023-11-24T08:09:57.346290+0000","last_became_peered":"2023-11-24T08:09:57.346290+0000","last_unstale":"2023-11-24T08:09:58.350342+0000","last_undegraded":"2023-11-24T08:09:58.350342+0000","last_fullsized":"2023-11-24T08:09:58.350342+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:02.739444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352443+0000","last_change":"2023-11-24T08:09:56.328690+0000","last_active":"2023-11-24T08:09:58.352443+0000","last_peered":"2023-11-24T08:09:58.352443+0000","last_clean":"2023-11-24T08:09:58.352443+0000","last_became_active":"2023-11-24T08:09:56.327008+0000","last_became_peered":"2023-11-24T08:09:56.327008+0000","last_unstale":"2023-11-24T08:09:58.352443+0000","last_undegraded":"2023-11-24T08:09:58.352443+0000","last_fullsized":"2023-11-24T08:09:58.352443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:04.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349611+0000","last_change":"2023-11-24T08:09:57.326872+0000","last_active":"2023-11-24T08:09:58.349611+0000","last_peered":"2023-11-24T08:09:58.349611+0000","last_clean":"2023-11-24T08:09:58.349611+0000","last_became_active":"2023-11-24T08:09:57.325481+0000","last_became_peered":"2023-11-24T08:09:57.325481+0000","last_unstale":"2023-11-24T08:09:58.349611+0000","last_undegraded":"2023-11-24T08:09:58.349611+0000","last_fullsized":"2023-11-24T08:09:58.349611+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:26:29.840340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"35'3","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.355340+0000","last_change":"2023-11-24T08:09:56.335960+0000","last_active":"2023-11-24T08:09:57.355340+0000","last_peered":"2023-11-24T08:09:57.355340+0000","last_clean":"2023-11-24T08:09:57.355340+0000","last_became_active":"2023-11-24T08:09:56.335665+0000","last_became_peered":"2023-11-24T08:09:56.335665+0000","last_unstale":"2023-11-24T08:09:57.355340+0000","last_undegraded":"2023-11-24T08:09:57.355340+0000","last_fullsized":"2023-11-24T08:09:57.355340+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:05:04.045363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.332684+0000","last_change":"2023-11-24T08:09:57.346488+0000","last_active":"2023-11-24T08:09:58.332684+0000","last_peered":"2023-11-24T08:09:58.332684+0000","last_clean":"2023-11-24T08:09:58.332684+0000","last_became_active":"2023-11-24T08:09:57.346237+0000","last_became_peered":"2023-11-24T08:09:57.346237+0000","last_unstale":"2023-11-24T08:09:58.332684+0000","last_undegraded":"2023-11-24T08:09:58.332684+0000","last_fullsized":"2023-11-24T08:09:58.332684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:12:49.330012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"35'6","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.363084+0000","last_change":"2023-11-24T08:09:56.327612+0000","last_active":"2023-11-24T08:09:57.363084+0000","last_peered":"2023-11-24T08:09:57.363084+0000","last_clean":"2023-11-24T08:09:57.363084+0000","last_became_active":"2023-11-24T08:09:56.327332+0000","last_became_peered":"2023-11-24T08:09:56.327332+0000","last_unstale":"2023-11-24T08:09:57.363084+0000","last_undegraded":"2023-11-24T08:09:57.363084+0000","last_fullsized":"2023-11-24T08:09:57.363084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:02:20.336231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330532+0000","last_change":"2023-11-24T08:09:57.342440+0000","last_active":"2023-11-24T08:09:58.330532+0000","last_peered":"2023-11-24T08:09:58.330532+0000","last_clean":"2023-11-24T08:09:58.330532+0000","last_became_active":"2023-11-24T08:09:57.342154+0000","last_became_peered":"2023-11-24T08:09:57.342154+0000","last_unstale":"2023-11-24T08:09:58.330532+0000","last_undegraded":"2023-11-24T08:09:58.330532+0000","last_fullsized":"2023-11-24T08:09:58.330532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:54.576837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.353995+0000","last_change":"2023-11-24T08:09:56.325701+0000","last_active":"2023-11-24T08:09:57.353995+0000","last_peered":"2023-11-24T08:09:57.353995+0000","last_clean":"2023-11-24T08:09:57.353995+0000","last_became_active":"2023-11-24T08:09:56.325397+0000","last_became_peered":"2023-11-24T08:09:56.325397+0000","last_unstale":"2023-11-24T08:09:57.353995+0000","last_undegraded":"2023-11-24T08:09:57.353995+0000","last_fullsized":"2023-11-24T08:09:57.353995+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:04.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350313+0000","last_change":"2023-11-24T08:09:57.338102+0000","last_active":"2023-11-24T08:09:58.350313+0000","last_peered":"2023-11-24T08:09:58.350313+0000","last_clean":"2023-11-24T08:09:58.350313+0000","last_became_active":"2023-11-24T08:09:57.337854+0000","last_became_peered":"2023-11-24T08:09:57.337854+0000","last_unstale":"2023-11-24T08:09:58.350313+0000","last_undegraded":"2023-11-24T08:09:58.350313+0000","last_fullsized":"2023-11-24T08:09:58.350313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:01.999633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.329607+0000","last_change":"2023-11-24T08:09:56.322061+0000","last_active":"2023-11-24T08:09:57.329607+0000","last_peered":"2023-11-24T08:09:57.329607+0000","last_clean":"2023-11-24T08:09:57.329607+0000","last_became_active":"2023-11-24T08:09:56.321806+0000","last_became_peered":"2023-11-24T08:09:56.321806+0000","last_unstale":"2023-11-24T08:09:57.329607+0000","last_undegraded":"2023-11-24T08:09:57.329607+0000","last_fullsized":"2023-11-24T08:09:57.329607+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:51:04.815091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350082+0000","last_change":"2023-11-24T08:09:57.338019+0000","last_active":"2023-11-24T08:09:58.350082+0000","last_peered":"2023-11-24T08:09:58.350082+0000","last_clean":"2023-11-24T08:09:58.350082+0000","last_became_active":"2023-11-24T08:09:57.337328+0000","last_became_peered":"2023-11-24T08:09:57.337328+0000","last_unstale":"2023-11-24T08:09:58.350082+0000","last_undegraded":"2023-11-24T08:09:58.350082+0000","last_fullsized":"2023-11-24T08:09:58.350082+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:41:32.521394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.308006+0000","last_change":"2023-11-24T08:09:56.335379+0000","last_active":"2023-11-24T08:09:57.308006+0000","last_peered":"2023-11-24T08:09:57.308006+0000","last_clean":"2023-11-24T08:09:57.308006+0000","last_became_active":"2023-11-24T08:09:56.335102+0000","last_became_peered":"2023-11-24T08:09:56.335102+0000","last_unstale":"2023-11-24T08:09:57.308006+0000","last_undegraded":"2023-11-24T08:09:57.308006+0000","last_fullsized":"2023-11-24T08:09:57.308006+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333412+0000","last_change":"2023-11-24T08:09:57.337960+0000","last_active":"2023-11-24T08:09:58.333412+0000","last_peered":"2023-11-24T08:09:58.333412+0000","last_clean":"2023-11-24T08:09:58.333412+0000","last_became_active":"2023-11-24T08:09:57.337386+0000","last_became_peered":"2023-11-24T08:09:57.337386+0000","last_unstale":"2023-11-24T08:09:58.333412+0000","last_undegraded":"2023-11-24T08:09:58.333412+0000","last_fullsized":"2023-11-24T08:09:58.333412+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:28.999355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"35'8","reported_seq":20,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.360921+0000","last_change":"2023-11-24T08:09:56.322047+0000","last_active":"2023-11-24T08:09:57.360921+0000","last_peered":"2023-11-24T08:09:57.360921+0000","last_clean":"2023-11-24T08:09:57.360921+0000","last_became_active":"2023-11-24T08:09:56.321761+0000","last_became_peered":"2023-11-24T08:09:56.321761+0000","last_unstale":"2023-11-24T08:09:57.360921+0000","last_undegraded":"2023-11-24T08:09:57.360921+0000","last_fullsized":"2023-11-24T08:09:57.360921+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:27.993623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351482+0000","last_change":"2023-11-24T08:09:57.344842+0000","last_active":"2023-11-24T08:09:58.351482+0000","last_peered":"2023-11-24T08:09:58.351482+0000","last_clean":"2023-11-24T08:09:58.351482+0000","last_became_active":"2023-11-24T08:09:57.344499+0000","last_became_peered":"2023-11-24T08:09:57.344499+0000","last_unstale":"2023-11-24T08:09:58.351482+0000","last_undegraded":"2023-11-24T08:09:58.351482+0000","last_fullsized":"2023-11-24T08:09:58.351482+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:53:41.487984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.357012+0000","last_change":"2023-11-24T08:09:56.327820+0000","last_active":"2023-11-24T08:09:57.357012+0000","last_peered":"2023-11-24T08:09:57.357012+0000","last_clean":"2023-11-24T08:09:57.357012+0000","last_became_active":"2023-11-24T08:09:56.327550+0000","last_became_peered":"2023-11-24T08:09:56.327550+0000","last_unstale":"2023-11-24T08:09:57.357012+0000","last_undegraded":"2023-11-24T08:09:57.357012+0000","last_fullsized":"2023-11-24T08:09:57.357012+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:00:59.923113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333043+0000","last_change":"2023-11-24T08:09:57.348775+0000","last_active":"2023-11-24T08:09:58.333043+0000","last_peered":"2023-11-24T08:09:58.333043+0000","last_clean":"2023-11-24T08:09:58.333043+0000","last_became_active":"2023-11-24T08:09:57.348186+0000","last_became_peered":"2023-11-24T08:09:57.348186+0000","last_unstale":"2023-11-24T08:09:58.333043+0000","last_undegraded":"2023-11-24T08:09:58.333043+0000","last_fullsized":"2023-11-24T08:09:58.333043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:41:18.445825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.353657+0000","last_change":"2023-11-24T08:09:56.325891+0000","last_active":"2023-11-24T08:09:57.353657+0000","last_peered":"2023-11-24T08:09:57.353657+0000","last_clean":"2023-11-24T08:09:57.353657+0000","last_became_active":"2023-11-24T08:09:56.325495+0000","last_became_peered":"2023-11-24T08:09:56.325495+0000","last_unstale":"2023-11-24T08:09:57.353657+0000","last_undegraded":"2023-11-24T08:09:57.353657+0000","last_fullsized":"2023-11-24T08:09:57.353657+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:04.110358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.330843+0000","last_change":"2023-11-24T08:09:57.341510+0000","last_active":"2023-11-24T08:09:58.330843+0000","last_peered":"2023-11-24T08:09:58.330843+0000","last_clean":"2023-11-24T08:09:58.330843+0000","last_became_active":"2023-11-24T08:09:57.341193+0000","last_became_peered":"2023-11-24T08:09:57.341193+0000","last_unstale":"2023-11-24T08:09:58.330843+0000","last_undegraded":"2023-11-24T08:09:58.330843+0000","last_fullsized":"2023-11-24T08:09:58.330843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:55:37.663117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"35'1","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.359531+0000","last_change":"2023-11-24T08:09:56.329612+0000","last_active":"2023-11-24T08:09:57.359531+0000","last_peered":"2023-11-24T08:09:57.359531+0000","last_clean":"2023-11-24T08:09:57.359531+0000","last_became_active":"2023-11-24T08:09:56.329330+0000","last_became_peered":"2023-11-24T08:09:56.329330+0000","last_unstale":"2023-11-24T08:09:57.359531+0000","last_undegraded":"2023-11-24T08:09:57.359531+0000","last_fullsized":"2023-11-24T08:09:57.359531+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:48:17.660091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351185+0000","last_change":"2023-11-24T08:09:57.340436+0000","last_active":"2023-11-24T08:09:58.351185+0000","last_peered":"2023-11-24T08:09:58.351185+0000","last_clean":"2023-11-24T08:09:58.351185+0000","last_became_active":"2023-11-24T08:09:57.340164+0000","last_became_peered":"2023-11-24T08:09:57.340164+0000","last_unstale":"2023-11-24T08:09:58.351185+0000","last_undegraded":"2023-11-24T08:09:58.351185+0000","last_fullsized":"2023-11-24T08:09:58.351185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:19.298311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.322327+0000","last_change":"2023-11-24T08:09:56.330279+0000","last_active":"2023-11-24T08:09:57.322327+0000","last_peered":"2023-11-24T08:09:57.322327+0000","last_clean":"2023-11-24T08:09:57.322327+0000","last_became_active":"2023-11-24T08:09:56.329931+0000","last_became_peered":"2023-11-24T08:09:56.329931+0000","last_unstale":"2023-11-24T08:09:57.322327+0000","last_undegraded":"2023-11-24T08:09:57.322327+0000","last_fullsized":"2023-11-24T08:09:57.322327+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:27.338331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349852+0000","last_change":"2023-11-24T08:09:57.337668+0000","last_active":"2023-11-24T08:09:58.349852+0000","last_peered":"2023-11-24T08:09:58.349852+0000","last_clean":"2023-11-24T08:09:58.349852+0000","last_became_active":"2023-11-24T08:09:57.337408+0000","last_became_peered":"2023-11-24T08:09:57.337408+0000","last_unstale":"2023-11-24T08:09:58.349852+0000","last_undegraded":"2023-11-24T08:09:58.349852+0000","last_fullsized":"2023-11-24T08:09:58.349852+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:12.046326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"35'2","reported_seq":14,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352273+0000","last_change":"2023-11-24T08:09:56.319325+0000","last_active":"2023-11-24T08:09:58.352273+0000","last_peered":"2023-11-24T08:09:58.352273+0000","last_clean":"2023-11-24T08:09:58.352273+0000","last_became_active":"2023-11-24T08:09:56.318542+0000","last_became_peered":"2023-11-24T08:09:56.318542+0000","last_unstale":"2023-11-24T08:09:58.352273+0000","last_undegraded":"2023-11-24T08:09:58.352273+0000","last_fullsized":"2023-11-24T08:09:58.352273+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:32:50.843182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349949+0000","last_change":"2023-11-24T08:09:57.338376+0000","last_active":"2023-11-24T08:09:58.349949+0000","last_peered":"2023-11-24T08:09:58.349949+0000","last_clean":"2023-11-24T08:09:58.349949+0000","last_became_active":"2023-11-24T08:09:57.338128+0000","last_became_peered":"2023-11-24T08:09:57.338128+0000","last_unstale":"2023-11-24T08:09:58.349949+0000","last_undegraded":"2023-11-24T08:09:58.349949+0000","last_fullsized":"2023-11-24T08:09:58.349949+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:28.344063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.314575+0000","last_change":"2023-11-24T08:09:56.333904+0000","last_active":"2023-11-24T08:09:57.314575+0000","last_peered":"2023-11-24T08:09:57.314575+0000","last_clean":"2023-11-24T08:09:57.314575+0000","last_became_active":"2023-11-24T08:09:56.331340+0000","last_became_peered":"2023-11-24T08:09:56.331340+0000","last_unstale":"2023-11-24T08:09:57.314575+0000","last_undegraded":"2023-11-24T08:09:57.314575+0000","last_fullsized":"2023-11-24T08:09:57.314575+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:30.935923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.349861+0000","last_change":"2023-11-24T08:09:57.343688+0000","last_active":"2023-11-24T08:09:58.349861+0000","last_peered":"2023-11-24T08:09:58.349861+0000","last_clean":"2023-11-24T08:09:58.349861+0000","last_became_active":"2023-11-24T08:09:57.343488+0000","last_became_peered":"2023-11-24T08:09:57.343488+0000","last_unstale":"2023-11-24T08:09:58.349861+0000","last_undegraded":"2023-11-24T08:09:58.349861+0000","last_fullsized":"2023-11-24T08:09:58.349861+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:14:45.085173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.329834+0000","last_change":"2023-11-24T08:09:56.325039+0000","last_active":"2023-11-24T08:09:57.329834+0000","last_peered":"2023-11-24T08:09:57.329834+0000","last_clean":"2023-11-24T08:09:57.329834+0000","last_became_active":"2023-11-24T08:09:56.324552+0000","last_became_peered":"2023-11-24T08:09:56.324552+0000","last_unstale":"2023-11-24T08:09:57.329834+0000","last_undegraded":"2023-11-24T08:09:57.329834+0000","last_fullsized":"2023-11-24T08:09:57.329834+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:27.993623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351709+0000","last_change":"2023-11-24T08:09:57.328012+0000","last_active":"2023-11-24T08:09:58.351709+0000","last_peered":"2023-11-24T08:09:58.351709+0000","last_clean":"2023-11-24T08:09:58.351709+0000","last_became_active":"2023-11-24T08:09:57.327707+0000","last_became_peered":"2023-11-24T08:09:57.327707+0000","last_unstale":"2023-11-24T08:09:58.351709+0000","last_undegraded":"2023-11-24T08:09:58.351709+0000","last_fullsized":"2023-11-24T08:09:58.351709+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:10.423396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.333056+0000","last_change":"2023-11-24T08:09:56.338027+0000","last_active":"2023-11-24T08:09:57.333056+0000","last_peered":"2023-11-24T08:09:57.333056+0000","last_clean":"2023-11-24T08:09:57.333056+0000","last_became_active":"2023-11-24T08:09:56.337414+0000","last_became_peered":"2023-11-24T08:09:56.337414+0000","last_unstale":"2023-11-24T08:09:57.333056+0000","last_undegraded":"2023-11-24T08:09:57.333056+0000","last_fullsized":"2023-11-24T08:09:57.333056+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:47:16.239226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.351404+0000","last_change":"2023-11-24T08:09:57.349436+0000","last_active":"2023-11-24T08:09:58.351404+0000","last_peered":"2023-11-24T08:09:58.351404+0000","last_clean":"2023-11-24T08:09:58.351404+0000","last_became_active":"2023-11-24T08:09:57.349144+0000","last_became_peered":"2023-11-24T08:09:57.349144+0000","last_unstale":"2023-11-24T08:09:58.351404+0000","last_undegraded":"2023-11-24T08:09:58.351404+0000","last_fullsized":"2023-11-24T08:09:58.351404+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:32:58.242223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"35'2","reported_seq":14,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.356911+0000","last_change":"2023-11-24T08:09:56.332402+0000","last_active":"2023-11-24T08:09:57.356911+0000","last_peered":"2023-11-24T08:09:57.356911+0000","last_clean":"2023-11-24T08:09:57.356911+0000","last_became_active":"2023-11-24T08:09:56.332013+0000","last_became_peered":"2023-11-24T08:09:56.332013+0000","last_unstale":"2023-11-24T08:09:57.356911+0000","last_undegraded":"2023-11-24T08:09:57.356911+0000","last_fullsized":"2023-11-24T08:09:57.356911+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:03.938076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333768+0000","last_change":"2023-11-24T08:09:57.344032+0000","last_active":"2023-11-24T08:09:58.333768+0000","last_peered":"2023-11-24T08:09:58.333768+0000","last_clean":"2023-11-24T08:09:58.333768+0000","last_became_active":"2023-11-24T08:09:57.343763+0000","last_became_peered":"2023-11-24T08:09:57.343763+0000","last_unstale":"2023-11-24T08:09:58.333768+0000","last_undegraded":"2023-11-24T08:09:58.333768+0000","last_fullsized":"2023-11-24T08:09:58.333768+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:43:40.250323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352338+0000","last_change":"2023-11-24T08:09:56.316753+0000","last_active":"2023-11-24T08:09:58.352338+0000","last_peered":"2023-11-24T08:09:58.352338+0000","last_clean":"2023-11-24T08:09:58.352338+0000","last_became_active":"2023-11-24T08:09:56.316367+0000","last_became_peered":"2023-11-24T08:09:56.316367+0000","last_unstale":"2023-11-24T08:09:58.352338+0000","last_undegraded":"2023-11-24T08:09:58.352338+0000","last_fullsized":"2023-11-24T08:09:58.352338+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:02:21.555016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352091+0000","last_change":"2023-11-24T08:09:57.348112+0000","last_active":"2023-11-24T08:09:58.352091+0000","last_peered":"2023-11-24T08:09:58.352091+0000","last_clean":"2023-11-24T08:09:58.352091+0000","last_became_active":"2023-11-24T08:09:57.347745+0000","last_became_peered":"2023-11-24T08:09:57.347745+0000","last_unstale":"2023-11-24T08:09:58.352091+0000","last_undegraded":"2023-11-24T08:09:58.352091+0000","last_fullsized":"2023-11-24T08:09:58.352091+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:47:04.943808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.330200+0000","last_change":"2023-11-24T08:09:56.330274+0000","last_active":"2023-11-24T08:09:57.330200+0000","last_peered":"2023-11-24T08:09:57.330200+0000","last_clean":"2023-11-24T08:09:57.330200+0000","last_became_active":"2023-11-24T08:09:56.329610+0000","last_became_peered":"2023-11-24T08:09:56.329610+0000","last_unstale":"2023-11-24T08:09:57.330200+0000","last_undegraded":"2023-11-24T08:09:57.330200+0000","last_fullsized":"2023-11-24T08:09:57.330200+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:22:54.111691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.333679+0000","last_change":"2023-11-24T08:09:57.335971+0000","last_active":"2023-11-24T08:09:58.333679+0000","last_peered":"2023-11-24T08:09:58.333679+0000","last_clean":"2023-11-24T08:09:58.333679+0000","last_became_active":"2023-11-24T08:09:57.335638+0000","last_became_peered":"2023-11-24T08:09:57.335638+0000","last_unstale":"2023-11-24T08:09:58.333679+0000","last_undegraded":"2023-11-24T08:09:58.333679+0000","last_fullsized":"2023-11-24T08:09:58.333679+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:05.116090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":12,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352248+0000","last_change":"2023-11-24T08:09:56.329734+0000","last_active":"2023-11-24T08:09:58.352248+0000","last_peered":"2023-11-24T08:09:58.352248+0000","last_clean":"2023-11-24T08:09:58.352248+0000","last_became_active":"2023-11-24T08:09:56.329443+0000","last_became_peered":"2023-11-24T08:09:56.329443+0000","last_unstale":"2023-11-24T08:09:58.352248+0000","last_undegraded":"2023-11-24T08:09:58.352248+0000","last_fullsized":"2023-11-24T08:09:58.352248+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:33:27.993623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352112+0000","last_change":"2023-11-24T08:09:57.341345+0000","last_active":"2023-11-24T08:09:58.352112+0000","last_peered":"2023-11-24T08:09:58.352112+0000","last_clean":"2023-11-24T08:09:58.352112+0000","last_became_active":"2023-11-24T08:09:57.341027+0000","last_became_peered":"2023-11-24T08:09:57.341027+0000","last_unstale":"2023-11-24T08:09:58.352112+0000","last_undegraded":"2023-11-24T08:09:58.352112+0000","last_fullsized":"2023-11-24T08:09:58.352112+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:01:00.928845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.329742+0000","last_change":"2023-11-24T08:09:56.335501+0000","last_active":"2023-11-24T08:09:57.329742+0000","last_peered":"2023-11-24T08:09:57.329742+0000","last_clean":"2023-11-24T08:09:57.329742+0000","last_became_active":"2023-11-24T08:09:56.335205+0000","last_became_peered":"2023-11-24T08:09:56.335205+0000","last_unstale":"2023-11-24T08:09:57.329742+0000","last_undegraded":"2023-11-24T08:09:57.329742+0000","last_fullsized":"2023-11-24T08:09:57.329742+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.348908+0000","last_change":"2023-11-24T08:09:57.326634+0000","last_active":"2023-11-24T08:09:58.348908+0000","last_peered":"2023-11-24T08:09:58.348908+0000","last_clean":"2023-11-24T08:09:58.348908+0000","last_became_active":"2023-11-24T08:09:57.325999+0000","last_became_peered":"2023-11-24T08:09:57.325999+0000","last_unstale":"2023-11-24T08:09:58.348908+0000","last_undegraded":"2023-11-24T08:09:58.348908+0000","last_fullsized":"2023-11-24T08:09:58.348908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:01.583026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.331905+0000","last_change":"2023-11-24T08:09:56.335034+0000","last_active":"2023-11-24T08:09:57.331905+0000","last_peered":"2023-11-24T08:09:57.331905+0000","last_clean":"2023-11-24T08:09:57.331905+0000","last_became_active":"2023-11-24T08:09:56.334326+0000","last_became_peered":"2023-11-24T08:09:56.334326+0000","last_unstale":"2023-11-24T08:09:57.331905+0000","last_undegraded":"2023-11-24T08:09:57.331905+0000","last_fullsized":"2023-11-24T08:09:57.331905+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:14.290447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.350634+0000","last_change":"2023-11-24T08:09:57.351440+0000","last_active":"2023-11-24T08:09:58.350634+0000","last_peered":"2023-11-24T08:09:58.350634+0000","last_clean":"2023-11-24T08:09:58.350634+0000","last_became_active":"2023-11-24T08:09:57.351180+0000","last_became_peered":"2023-11-24T08:09:57.351180+0000","last_unstale":"2023-11-24T08:09:58.350634+0000","last_undegraded":"2023-11-24T08:09:58.350634+0000","last_fullsized":"2023-11-24T08:09:58.350634+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:55:37.663117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.307853+0000","last_change":"2023-11-24T08:09:56.337047+0000","last_active":"2023-11-24T08:09:57.307853+0000","last_peered":"2023-11-24T08:09:57.307853+0000","last_clean":"2023-11-24T08:09:57.307853+0000","last_became_active":"2023-11-24T08:09:56.336873+0000","last_became_peered":"2023-11-24T08:09:56.336873+0000","last_unstale":"2023-11-24T08:09:57.307853+0000","last_undegraded":"2023-11-24T08:09:57.307853+0000","last_fullsized":"2023-11-24T08:09:57.307853+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:52:04.110358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":11,"reported_epoch":36,"state":"active+clean","last_fresh":"2023-11-24T08:09:58.352219+0000","last_change":"2023-11-24T08:09:57.334475+0000","last_active":"2023-11-24T08:09:58.352219+0000","last_peered":"2023-11-24T08:09:58.352219+0000","last_clean":"2023-11-24T08:09:58.352219+0000","last_became_active":"2023-11-24T08:09:57.333265+0000","last_became_peered":"2023-11-24T08:09:57.333265+0000","last_unstale":"2023-11-24T08:09:58.352219+0000","last_undegraded":"2023-11-24T08:09:58.352219+0000","last_fullsized":"2023-11-24T08:09:58.352219+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":34,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:56.284674+0000","last_clean_scrub_stamp":"2023-11-24T08:09:56.284674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:41.039861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.311778+0000","last_change":"2023-11-24T08:09:56.332890+0000","last_active":"2023-11-24T08:09:57.311778+0000","last_peered":"2023-11-24T08:09:57.311778+0000","last_clean":"2023-11-24T08:09:57.311778+0000","last_became_active":"2023-11-24T08:09:56.332562+0000","last_became_peered":"2023-11-24T08:09:56.332562+0000","last_unstale":"2023-11-24T08:09:57.311778+0000","last_undegraded":"2023-11-24T08:09:57.311778+0000","last_fullsized":"2023-11-24T08:09:57.311778+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:09:55.278942+0000","last_clean_scrub_stamp":"2023-11-24T08:09:55.278942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:18:27.438236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":177,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-24T08:09:57.315434+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:09:57.315434+0000","last_peered":"2023-11-24T08:09:57.315434+0000","last_clean":"2023-11-24T08:09:57.315434+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:09:57.315434+0000","last_undegraded":"2023-11-24T08:09:57.315434+0000","last_fullsized":"2023-11-24T08:09:57.315434+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542164,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":1003520,"data_stored":750485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.141},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":1.2989999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.0740000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.643},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.246}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.536},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.353}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.7090000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.446},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.2070000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.76}]}]},{"osd":6,"up_from":14,"seq":60129542163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31592,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340248,"statfs":{"total":96636764160,"available":96604413952,"internally_reserved":0,"allocated":413696,"data_stored":160643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.0229999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.3640000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.0800000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.85699999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.145},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.512}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.4810000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.25},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.4410000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.536},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.5629999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542162,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31584,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340256,"statfs":{"total":96636764160,"available":96604422144,"internally_reserved":0,"allocated":413696,"data_stored":160699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.4359999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":0.76500000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":0.999},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.5900000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.546},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6870000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.766}]},{"osd":6,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":2.121},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.728}]},{"osd":7,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.8600000000000001},{"interface":"front","average":{"1min":1.752,"5min":1.752,"15min":1.752},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.9689999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":2.1989999999999998}]}]},{"osd":4,"up_from":14,"seq":60129542163,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31588,"kb_used_data":404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340252,"statfs":{"total":96636764160,"available":96604418048,"internally_reserved":0,"allocated":413696,"data_stored":160131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.73799999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.05}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.93999999999999995}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.329},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.097}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.212},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.1830000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4410000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542163,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27440,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344400,"statfs":{"total":96636764160,"available":96608665600,"internally_reserved":0,"allocated":421888,"data_stored":161661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.61299999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.88},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.755}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.58299999999999996}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.208}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.407},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.0129999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.494}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.248},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":0.86099999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542163,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27428,"kb_used_data":400,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344412,"statfs":{"total":96636764160,"available":96608677888,"internally_reserved":0,"allocated":409600,"data_stored":160097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":0.83199999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.96599999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.327},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.27}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.1970000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.536}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.4390000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.51}]}]},{"osd":1,"up_from":14,"seq":60129542163,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27436,"kb_used_data":412,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344404,"statfs":{"total":96636764160,"available":96608669696,"internally_reserved":0,"allocated":421888,"data_stored":161661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":1.0629999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.1180000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.75}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.2190000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.518},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3899999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.6930000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.4159999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":999424,"data_stored":750395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2010000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":0.85199999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.119},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.77200000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.345}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.4319999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.159},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.3680000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.3140000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:02.231 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:10:02.231 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:10:02.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.497+0000 7f814366b640 1 Processor -- start 2023-11-24T08:10:02.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.498+0000 7f814366b640 1 -- start start 2023-11-24T08:10:02.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f814366b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c06bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:02.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f814366b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f813c1605f0 0x7f813c160a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:02.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f814366b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f813c160fc0 con 0x7f813c0d2000 2023-11-24T08:10:02.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f814366b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f813c0ce410 con 0x7f813c1605f0 2023-11-24T08:10:02.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f814366b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0d06f0 con 0x7f813c0d1c20 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f8141c08640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f813c1605f0 0x7f813c160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f8140c06640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f813c0d2000 0x7f813c0cfcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39382/0 (socket says 172.21.15.106:39382) 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f8141c08640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f813c1605f0 0x7f813c160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49368/0 (socket says 172.21.15.106:49368) 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.499+0000 7f8140c06640 1 -- 172.21.15.106:0/1928856119 learned_addr learned my addr 172.21.15.106:0/1928856119 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.500+0000 7f8141407640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c06bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.500+0000 7f8141c08640 1 -- 172.21.15.106:0/1928856119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 msgr2=0x7f813c06bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.500+0000 7f8141c08640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c06bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.500+0000 7f8141c08640 1 -- 172.21.15.106:0/1928856119 >> v1:172.21.15.106:6789/0 conn(0x7f813c0d2000 legacy=0x7f813c0cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.500+0000 7f8141c08640 1 -- 172.21.15.106:0/1928856119 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f813c0d09d0 con 0x7f813c1605f0 2023-11-24T08:10:02.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.500+0000 7f8141407640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c06bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.501+0000 7f8141c08640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c160a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f81340097a0 tx=0x7f813402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1685ddae8322b8b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.501+0000 7f813bfff640 1 -- 172.21.15.106:0/1928856119 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813402d040 con 0x7f813c1605f0 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.501+0000 7f813bfff640 1 -- 172.21.15.106:0/1928856119 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8134031810 con 0x7f813c1605f0 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.501+0000 7f813bfff640 1 -- 172.21.15.106:0/1928856119 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134031ba0 con 0x7f813c1605f0 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 -- 172.21.15.106:0/1928856119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 msgr2=0x7f813c160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c160a80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f81340097a0 tx=0x7f813402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 -- 172.21.15.106:0/1928856119 shutdown_connections 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c160a80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 --2- 172.21.15.106:0/1928856119 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c06bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 -- 172.21.15.106:0/1928856119 >> 172.21.15.106:0/1928856119 conn(0x7f813c08cea0 msgr2=0x7f813c09c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:02.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 -- 172.21.15.106:0/1928856119 shutdown_connections 2023-11-24T08:10:02.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.502+0000 7f814366b640 1 -- 172.21.15.106:0/1928856119 wait complete. 2023-11-24T08:10:02.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.504+0000 7f814366b640 1 Processor -- start 2023-11-24T08:10:02.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.504+0000 7f814366b640 1 -- start start 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f814366b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c0a0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f814366b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f813c0d2000 0x7f813c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f814366b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c0d3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f814366b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c154e50 con 0x7f813c1605f0 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f814366b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0a0000 con 0x7f813c0d2000 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f814366b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813c0a02e0 con 0x7f813c0d1c20 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f8141c08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c0d3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f8141c08640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c0d3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49384/0 (socket says 172.21.15.106:49384) 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f8141c08640 1 -- 172.21.15.106:0/1760193253 learned_addr learned my addr 172.21.15.106:0/1760193253 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:02.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.505+0000 7f8141407640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c0a0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8140c06640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f813c0d2000 0x7f813c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8141c08640 1 -- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 msgr2=0x7f813c0a0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8141c08640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c0a0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8141c08640 1 -- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f813c0d2000 msgr2=0x7f813c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8141c08640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f813c0d2000 0x7f813c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8141c08640 1 -- 172.21.15.106:0/1760193253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134009400 con 0x7f813c1605f0 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8140c06640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f813c0d2000 0x7f813c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8141c08640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c0d3100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f813402b8f0 tx=0x7f81340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813403ace0 con 0x7f813c1605f0 2023-11-24T08:10:02.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813403ae60 con 0x7f813c1605f0 2023-11-24T08:10:02.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8134044630 con 0x7f813c1605f0 2023-11-24T08:10:02.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.506+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f813c0a05c0 con 0x7f813c1605f0 2023-11-24T08:10:02.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.507+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f813c09fcf0 con 0x7f813c1605f0 2023-11-24T08:10:02.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.508+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f813c0a0ae0 con 0x7f813c1605f0 2023-11-24T08:10:02.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.509+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f8134044790 con 0x7f813c1605f0 2023-11-24T08:10:02.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.509+0000 7f8139ffb640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812806ae40 0x7f812806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:02.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.510+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81340abad0 con 0x7f813c1605f0 2023-11-24T08:10:02.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.510+0000 7f8141407640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812806ae40 0x7f812806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:02.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.510+0000 7f8141407640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812806ae40 0x7f812806d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f812c007510 tx=0x7f812c0074a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:02.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.514+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f813c0a0ae0 con 0x7f813c1605f0 2023-11-24T08:10:02.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.771+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f813c0a0ae0 con 0x7f813c1605f0 2023-11-24T08:10:02.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.775+0000 7f8139ffb640 1 -- 172.21.15.106:0/1760193253 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 190+0+147 (secure 0 0 0) 0x7f81340350b0 con 0x7f813c1605f0 2023-11-24T08:10:02.777 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2023-11-24T08:10:02.777 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:10:02.777 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2023-11-24T08:10:02.778 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2023-11-24T08:10:02.778 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:10:02.778 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2023-11-24T08:10:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812806ae40 msgr2=0x7f812806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812806ae40 0x7f812806d300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f812c007510 tx=0x7f812c0074a0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 msgr2=0x7f813c0d3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c0d3100 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f813402b8f0 tx=0x7f81340046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 shutdown_connections 2023-11-24T08:10:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f813c1605f0 0x7f813c0d3100 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f813c0d2000 0x7f813c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812806ae40 0x7f812806d300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 --2- 172.21.15.106:0/1760193253 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f813c0d1c20 0x7f813c0a0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 >> 172.21.15.106:0/1760193253 conn(0x7f813c08cea0 msgr2=0x7f813c0b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:02.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 shutdown_connections 2023-11-24T08:10:02.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:02.778+0000 7f814366b640 1 -- 172.21.15.106:0/1760193253 wait complete. 2023-11-24T08:10:02.797 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:03.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.003+0000 7f08abfff640 1 Processor -- start 2023-11-24T08:10:03.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.003+0000 7f08abfff640 1 -- start start 2023-11-24T08:10:03.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.004+0000 7f08abfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 0x7f08ac168080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.004+0000 7f08abfff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac0ce990 con 0x7f08ac15d4e0 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.004+0000 7f08abfff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac0cec70 con 0x7f08ac0d1e20 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.004+0000 7f08abfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0d4110 con 0x7f08ac0d0240 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.005+0000 7f08aaffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f08ac15d4e0 0x7f08ac0d38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39396/0 (socket says 172.21.15.106:39396) 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.005+0000 7f08aaffd640 1 -- 172.21.15.106:0/510704746 learned_addr learned my addr 172.21.15.106:0/510704746 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.005+0000 7f08ab7fe640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 0x7f08ac168080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.005+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074092846 0 0) 0x7f08ac0ce990 con 0x7f08ac15d4e0 2023-11-24T08:10:03.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.005+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0894003520 con 0x7f08ac15d4e0 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260761165 0 0) 0x7f0894003520 con 0x7f08ac15d4e0 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 msgr2=0x7f08ac168080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 0x7f08ac168080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 >> v1:172.21.15.156:6789/0 conn(0x7f08ac0d1e20 legacy=0x7f08ac0cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0cef50 con 0x7f08ac15d4e0 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f089c001670 con 0x7f08ac15d4e0 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 >> v1:172.21.15.106:6789/0 conn(0x7f08ac15d4e0 legacy=0x7f08ac0d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08940039a0 0x7f0894003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940047c0 0x7f0894004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 0x7f0894005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0d4110 con 0x7f08940039a0 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f089400ba30 con 0x7f08940047c0 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f089400bbf0 con 0x7f08ac0d0240 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aaffd640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08940039a0 0x7f0894003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aa7fc640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940047c0 0x7f0894004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aaffd640 1 -- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 msgr2=0x7f0894005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aaffd640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0d0240 0x7f0894005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aaffd640 1 -- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940047c0 msgr2=0x7f0894004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aaffd640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940047c0 0x7f0894004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aaffd640 1 -- 172.21.15.106:0/510704746 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0894003520 con 0x7f08940039a0 2023-11-24T08:10:03.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.006+0000 7f08aa7fc640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940047c0 0x7f0894004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:03.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.007+0000 7f08aaffd640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08940039a0 0x7f0894003dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f089c007fd0 tx=0x7f08940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75aaaa5e58c15d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c005ad0 con 0x7f08940039a0 2023-11-24T08:10:03.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f089c005c30 con 0x7f08940039a0 2023-11-24T08:10:03.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08a9ffb640 1 -- 172.21.15.106:0/510704746 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f089c02c850 con 0x7f08940039a0 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08abfff640 1 -- 172.21.15.106:0/510704746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08940039a0 msgr2=0x7f0894003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08abfff640 1 --2- 172.21.15.106:0/510704746 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08940039a0 0x7f0894003dc0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f089c007fd0 tx=0x7f08940057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08ab7fe640 1 -- 172.21.15.106:0/510704746 reap_dead start 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08abfff640 1 -- 172.21.15.106:0/510704746 shutdown_connections 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08abfff640 1 -- 172.21.15.106:0/510704746 >> 172.21.15.106:0/510704746 conn(0x7f08ac09c510 msgr2=0x7f08ac0d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08abfff640 1 -- 172.21.15.106:0/510704746 shutdown_connections 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.008+0000 7f08abfff640 1 -- 172.21.15.106:0/510704746 wait complete. 2023-11-24T08:10:03.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.010+0000 7f08abfff640 1 Processor -- start 2023-11-24T08:10:03.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.010+0000 7f08abfff640 1 -- start start 2023-11-24T08:10:03.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08abfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940039a0 0x7f08ac1601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08abfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0ccf20 0x7f08ac0b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 0x7f08ac14d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08abfff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac14dba0 con 0x7f08ac14d1e0 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08abfff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac160a70 con 0x7f08940039a0 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08abfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0ca630 con 0x7f08ac0ccf20 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.011+0000 7f08ab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 0x7f08ac14d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 0x7f08ac14d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49398/0 (socket says 172.21.15.106:49398) 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 -- 172.21.15.106:0/1222751626 learned_addr learned my addr 172.21.15.106:0/1222751626 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08aa7fc640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0ccf20 0x7f08ac0b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08aaffd640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940039a0 0x7f08ac1601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 -- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0ccf20 msgr2=0x7f08ac0b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0ccf20 0x7f08ac0b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 -- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940039a0 msgr2=0x7f08ac1601b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940039a0 0x7f08ac1601b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 -- 172.21.15.106:0/1222751626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0ca910 con 0x7f08ac14d1e0 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08aa7fc640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0ccf20 0x7f08ac0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08aaffd640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940039a0 0x7f08ac1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08ab7fe640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 0x7f08ac14d660 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08ac08da00 tx=0x7f08a400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.012+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4010040 con 0x7f08ac14d1e0 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.013+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400bae0 con 0x7f08ac14d1e0 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.013+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400be70 con 0x7f08ac14d1e0 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.013+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac0cabf0 con 0x7f08ac14d1e0 2023-11-24T08:10:03.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.013+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac1606f0 con 0x7f08ac14d1e0 2023-11-24T08:10:03.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.014+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac0cb110 con 0x7f08ac14d1e0 2023-11-24T08:10:03.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.015+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f08a401e410 con 0x7f08ac14d1e0 2023-11-24T08:10:03.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.016+0000 7f08937fe640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f089806ae40 0x7f089806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.016+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08a4089810 con 0x7f08ac14d1e0 2023-11-24T08:10:03.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.016+0000 7f08aaffd640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f089806ae40 0x7f089806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.017+0000 7f08aaffd640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f089806ae40 0x7f089806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f089c034000 tx=0x7f089c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.020+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08ac0cb110 con 0x7f08ac14d1e0 2023-11-24T08:10:03.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.250+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08ac0cb110 con 0x7f08ac14d1e0 2023-11-24T08:10:03.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.252+0000 7f08937fe640 1 -- 172.21.15.106:0/1222751626 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f08a4058350 con 0x7f08ac14d1e0 2023-11-24T08:10:03.253 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:03.253 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:09:56.761029+0000","modified":"2023-11-24T08:09:59.823336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6836/3854927347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3854927347},{"type":"v1","addr":"172.21.15.106:6836","nonce":3854927347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:10:03.254 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 14 2023-11-24T08:10:03.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f089806ae40 msgr2=0x7f089806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f089806ae40 0x7f089806d300 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f089c034000 tx=0x7f089c009030 comp rx=0 tx=0).stop 2023-11-24T08:10:03.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 msgr2=0x7f08ac14d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 0x7f08ac14d660 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f08ac08da00 tx=0x7f08a400b480 comp rx=0 tx=0).stop 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 shutdown_connections 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08ac14d1e0 0x7f08ac14d660 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f089806ae40 0x7f089806d300 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08ac0ccf20 0x7f08ac0b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.254+0000 7f08abfff640 1 --2- 172.21.15.106:0/1222751626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08940039a0 0x7f08ac1601b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.255+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 >> 172.21.15.106:0/1222751626 conn(0x7f08ac09c510 msgr2=0x7f08ac06c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.255+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 shutdown_connections 2023-11-24T08:10:03.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.255+0000 7f08abfff640 1 -- 172.21.15.106:0/1222751626 wait complete. 2023-11-24T08:10:03.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:09:56.761029+0000', 'modified': '2023-11-24T08:09:59.823336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5082}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5082': {'gid': 5082, 'name': 'c', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.106:6836/3854927347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6834', 'nonce': 3854927347}, {'type': 'v1', 'addr': '172.21.15.106:6836', 'nonce': 3854927347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:10:03.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:10:03.678 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T08:10:03.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.904+0000 7f5c7c806640 1 Processor -- start 2023-11-24T08:10:03.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.904+0000 7f5c7c806640 1 -- start start 2023-11-24T08:10:03.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.905+0000 7f5c7c806640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5c78070ea0 0x7f5c780b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7c806640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 0x7f5c7806da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7c806640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c7806dfd0 con 0x7f5c7815a1c0 2023-11-24T08:10:03.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7c806640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780cd920 con 0x7f5c78070ea0 2023-11-24T08:10:03.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7c806640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815e580 con 0x7f5c7806d640 2023-11-24T08:10:03.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c76d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5c7815a1c0 0x7f5c7815dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:39398/0 (socket says 172.21.15.106:39398) 2023-11-24T08:10:03.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c76d9d640 1 -- 172.21.15.106:0/3463631124 learned_addr learned my addr 172.21.15.106:0/3463631124 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:03.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7759e640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 0x7f5c7806da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727692869 0 0) 0x7f5c68001110 con 0x7f5c7815a1c0 2023-11-24T08:10:03.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/3463631124 >> v2:172.21.15.156:3300/0 conn(0x7f5c78070ea0 0x7f5c780b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.906+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c64005730 con 0x7f5c7815a1c0 2023-11-24T08:10:03.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 560809780 0 0) 0x7f5c64005730 con 0x7f5c7815a1c0 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 msgr2=0x7f5c7806da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 0x7f5c7806da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 >> v2:172.21.15.156:3300/0 conn(0x7f5c78070ea0 msgr2=0x7f5c780b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 --2- 172.21.15.106:0/3463631124 >> v2:172.21.15.156:3300/0 conn(0x7f5c78070ea0 0x7f5c780b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7815e860 con 0x7f5c7815a1c0 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f5c68001800 con 0x7f5c7815a1c0 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 >> v1:172.21.15.106:6789/0 conn(0x7f5c7815a1c0 legacy=0x7f5c7815dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c64005b80 0x7f5c64005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c64006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 0x7f5c64007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c780cd920 con 0x7f5c640069d0 2023-11-24T08:10:03.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7815e580 con 0x7f5c64005b80 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6400dc70 con 0x7f5c7806d640 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c64006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c64005b80 0x7f5c64005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.907+0000 7f5c7759e640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 0x7f5c64007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.908+0000 7f5c6ffff640 1 -- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 msgr2=0x7f5c64007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.908+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c7806d640 0x7f5c64007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.908+0000 7f5c6ffff640 1 -- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c64005b80 msgr2=0x7f5c64005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.908+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c64005b80 0x7f5c64005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.908+0000 7f5c6ffff640 1 -- 172.21.15.106:0/3463631124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c64005730 con 0x7f5c640069d0 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.908+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c64005b80 0x7f5c64005fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c64006e50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ba30 tx=0x7f5c64007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3936e957a35c3b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60032680 con 0x7f5c640069d0 2023-11-24T08:10:03.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c600327e0 con 0x7f5c640069d0 2023-11-24T08:10:03.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c7659c640 1 -- 172.21.15.106:0/3463631124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60032b70 con 0x7f5c640069d0 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c7c806640 1 -- 172.21.15.106:0/3463631124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 msgr2=0x7f5c64006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c7c806640 1 --2- 172.21.15.106:0/3463631124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c64006e50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ba30 tx=0x7f5c64007a00 comp rx=0 tx=0).stop 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.909+0000 7f5c7759e640 1 -- 172.21.15.106:0/3463631124 reap_dead start 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.910+0000 7f5c7c806640 1 -- 172.21.15.106:0/3463631124 shutdown_connections 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.910+0000 7f5c7c806640 1 -- 172.21.15.106:0/3463631124 >> 172.21.15.106:0/3463631124 conn(0x7f5c780cbb40 msgr2=0x7f5c780cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.910+0000 7f5c7c806640 1 -- 172.21.15.106:0/3463631124 shutdown_connections 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.910+0000 7f5c7c806640 1 -- 172.21.15.106:0/3463631124 wait complete. 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.911+0000 7f5c7c806640 1 Processor -- start 2023-11-24T08:10:03.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.912+0000 7f5c7c806640 1 -- start start 2023-11-24T08:10:03.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c7c806640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c78156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c7c806640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c7814b940 0x7f5c78157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c7c806640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c78157d60 0x7f5c781510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c7c806640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78150bc0 con 0x7f5c640069d0 2023-11-24T08:10:03.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c7c806640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c781581e0 con 0x7f5c7814b940 2023-11-24T08:10:03.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c7c806640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78156fe0 con 0x7f5c78157d60 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c78156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c78156aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49426/0 (socket says 172.21.15.106:49426) 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 -- 172.21.15.106:0/221419610 learned_addr learned my addr 172.21.15.106:0/221419610 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 -- 172.21.15.106:0/221419610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c78157d60 msgr2=0x7f5c781510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c78157d60 0x7f5c781510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 -- 172.21.15.106:0/221419610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c7814b940 msgr2=0x7f5c78157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c7814b940 0x7f5c78157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.913+0000 7f5c76d9d640 1 -- 172.21.15.106:0/221419610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780af570 con 0x7f5c640069d0 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.914+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c78156aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5c68001eb0 tx=0x7f5c6800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.914+0000 7f5c6f7fe640 1 -- 172.21.15.106:0/221419610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680072d0 con 0x7f5c640069d0 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.914+0000 7f5c6f7fe640 1 -- 172.21.15.106:0/221419610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68007430 con 0x7f5c640069d0 2023-11-24T08:10:03.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.914+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c780af850 con 0x7f5c640069d0 2023-11-24T08:10:03.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.914+0000 7f5c6f7fe640 1 -- 172.21.15.106:0/221419610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680047e0 con 0x7f5c640069d0 2023-11-24T08:10:03.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.914+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780afe10 con 0x7f5c640069d0 2023-11-24T08:10:03.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.916+0000 7f5c6f7fe640 1 -- 172.21.15.106:0/221419610 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f5c68004b50 con 0x7f5c640069d0 2023-11-24T08:10:03.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.917+0000 7f5c6f7fe640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5c5806ae70 0x7f5c5806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.917+0000 7f5c6f7fe640 1 -- 172.21.15.106:0/221419610 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c68089300 con 0x7f5c640069d0 2023-11-24T08:10:03.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.917+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5c5806ae70 0x7f5c5806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.918+0000 7f5c7fb9b740 1 Processor -- start 2023-11-24T08:10:03.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.918+0000 7f5c7fb9b740 1 -- start start 2023-11-24T08:10:03.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.918+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5c5806ae70 0x7f5c5806d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ba00 tx=0x7f5c60032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bb9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2bb8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bfc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2b17880 con 0x55ccc2bb9070 2023-11-24T08:10:03.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2b26ae0 con 0x55ccc2bb87b0 2023-11-24T08:10:03.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2b241d0 con 0x55ccc2bb3d70 2023-11-24T08:10:03.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bb9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bb9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:49440/0 (socket says 172.21.15.106:49440) 2023-11-24T08:10:03.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 -- 172.21.15.106:0/422336640 learned_addr learned my addr 172.21.15.106:0/422336640 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:03.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 -- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 msgr2=0x55ccc2bb8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2bb8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c7759e640 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bfc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 -- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 msgr2=0x55ccc2bfc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bfc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.919+0000 7f5c76d9d640 1 -- 172.21.15.106:0/422336640 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccc2ae7860 con 0x55ccc2bb9070 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.920+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bb9450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5c68057f60 tx=0x7f5c68057a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75e952a67aea711 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.920+0000 7f5c6cff9640 1 -- 172.21.15.106:0/422336640 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c680602d0 con 0x55ccc2bb9070 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.920+0000 7f5c6cff9640 1 -- 172.21.15.106:0/422336640 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680890c0 con 0x55ccc2bb9070 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.920+0000 7f5c6cff9640 1 -- 172.21.15.106:0/422336640 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68056370 con 0x55ccc2bb9070 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/422336640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 msgr2=0x55ccc2bb9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bb9450 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5c68057f60 tx=0x7f5c68057a40 comp rx=0 tx=0).stop 2023-11-24T08:10:03.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/422336640 shutdown_connections 2023-11-24T08:10:03.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bfc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2bb8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/422336640 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bb9450 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/422336640 >> 172.21.15.106:0/422336640 conn(0x55ccc2b7ac70 msgr2=0x55ccc2bfeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/422336640 shutdown_connections 2023-11-24T08:10:03.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.921+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/422336640 wait complete. 2023-11-24T08:10:03.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.922+0000 7f5c7fb9b740 1 -- start start 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.922+0000 7f5c7fb9b740 1 Processor -- start 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 0x55ccc2bd8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c04260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb9070 0x55ccc2bdffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2b1fac0 con 0x55ccc2bb3d70 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2abc2f0 con 0x55ccc2bb87b0 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2ad9c70 con 0x55ccc2bb9070 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7fb9b740 10 client.0 _ll_register_callbacks cb 0x55ccc2ade3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 0x55ccc2bd8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 0x55ccc2bd8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58184/0 (socket says 172.21.15.106:58184) 2023-11-24T08:10:03.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 -- 172.21.15.106:0/1338022921 learned_addr learned my addr 172.21.15.106:0/1338022921 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c7759e640 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb9070 0x55ccc2bdffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 -- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb9070 msgr2=0x55ccc2bdffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb9070 0x55ccc2bdffe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 -- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 msgr2=0x55ccc2c04260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c04260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.923+0000 7f5c76d9d640 1 -- 172.21.15.106:0/1338022921 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c780afe10 con 0x55ccc2bb3d70 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 0x55ccc2bd8ed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800b4a0 tx=0x7f5c680077a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ba7d0c77e7fda server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c55ffb640 1 -- 172.21.15.106:0/1338022921 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68087330 con 0x55ccc2bb3d70 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c55ffb640 1 -- 172.21.15.106:0/1338022921 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c68087490 con 0x55ccc2bb3d70 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ccc2b18aa0 con 0x55ccc2bb3d70 2023-11-24T08:10:03.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c55ffb640 1 -- 172.21.15.106:0/1338022921 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c6805e030 con 0x55ccc2bb3d70 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c7fb9b740 10 client.5265 fetch_fsmap learned FSMap version 14 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55ccc2b1e140 con 0x55ccc2bb3d70 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.924+0000 7f5c55ffb640 1 -- 172.21.15.106:0/1338022921 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ccc2b18aa0 con 0x55ccc2bb3d70 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c55ffb640 1 -- 172.21.15.106:0/1338022921 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c6806bd20 con 0x55ccc2bb3d70 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c7fb9b740 10 client.5265 fetch_fsmap finished waiting for FSMap version 14 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c7fb9b740 10 client.5265 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c7fb9b740 20 client.5265 populate_metadata read hostname 'smithi106' 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2c002b0 0x55ccc2c00690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c7fb9b740 4 client.5265 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 --> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ccc2b187c0 con 0x55ccc2c002b0 2023-11-24T08:10:03.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.925+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2c002b0 0x55ccc2c00690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.926+0000 7f5c55ffb640 10 client.5265 ms_handle_connect on v2:172.21.15.106:6834/3854927347 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.926+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2c002b0 0x55ccc2c00690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27a292f1a66bf71 server_cookie=a82093c9450ed6b in_seq=0 out_seq=0 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.926+0000 7f5c55ffb640 1 -- 172.21.15.106:0/1338022921 <== mds.0 v2:172.21.15.106:6834/3854927347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ccc2b187c0 con 0x55ccc2c002b0 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.926+0000 7f5c55ffb640 10 client.5265 handle_command_reply: tid=0 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.926+0000 7f5c7fb9b740 1 client.5265 shutdown 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 msgr2=0x55ccc2bd8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 0x55ccc2bd8ed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800b4a0 tx=0x7f5c680077a0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 shutdown_connections 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2c002b0 0x55ccc2c00690 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb9070 0x55ccc2bdffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c04260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c55ffb640 0 client.5265 ms_handle_reset on v2:172.21.15.106:6834/3854927347 2023-11-24T08:10:03.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1338022921 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb3d70 0x55ccc2bd8ed0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 >> 172.21.15.106:0/1338022921 conn(0x55ccc2b7ac70 msgr2=0x55ccc2bfeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 shutdown_connections 2023-11-24T08:10:03.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1338022921 wait complete. 2023-11-24T08:10:03.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.927+0000 7f5c7fb9b740 20 client.5265 trim_cache size 0 max 16384 2023-11-24T08:10:03.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.944+0000 7f5c7fb9b740 1 Processor -- start 2023-11-24T08:10:03.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.944+0000 7f5c7fb9b740 1 -- start start 2023-11-24T08:10:03.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2bda0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bb7960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bfeb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2882f20 con 0x55ccc2bb9070 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2bb3610 con 0x55ccc2bb87b0 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2c06670 con 0x55ccc2bb3d70 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bfeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bfeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58198/0 (socket says 172.21.15.106:58198) 2023-11-24T08:10:03.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.945+0000 7f5c7759e640 1 -- 172.21.15.106:0/1606468062 learned_addr learned my addr 172.21.15.106:0/1606468062 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:03.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c7759e640 1 -- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 msgr2=0x55ccc2bda0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c7759e640 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2bda0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c7759e640 1 -- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 msgr2=0x55ccc2bb7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bb7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c7759e640 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bb7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c7759e640 1 -- 172.21.15.106:0/1606468062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ccc2abbb80 con 0x55ccc2bb9070 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c7759e640 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bfeb60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c780cbaa0 tx=0x7f5c70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5829a85ecd3840a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c6d7fa640 1 -- 172.21.15.106:0/1606468062 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7001a070 con 0x55ccc2bb9070 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c6d7fa640 1 -- 172.21.15.106:0/1606468062 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c70011050 con 0x55ccc2bb9070 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.946+0000 7f5c6d7fa640 1 -- 172.21.15.106:0/1606468062 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70003bb0 con 0x55ccc2bb9070 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1606468062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 msgr2=0x55ccc2bfeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bfeb60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5c780cbaa0 tx=0x7f5c70009750 comp rx=0 tx=0).stop 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1606468062 shutdown_connections 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb9070 0x55ccc2bfeb60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb87b0 0x55ccc2bb7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/1606468062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2bda0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1606468062 >> 172.21.15.106:0/1606468062 conn(0x55ccc2b7ac70 msgr2=0x55ccc2bb2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1606468062 shutdown_connections 2023-11-24T08:10:03.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.947+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/1606468062 wait complete. 2023-11-24T08:10:03.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.948+0000 7f5c7fb9b740 1 -- start start 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.948+0000 7f5c7fb9b740 1 Processor -- start 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2c70670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c0ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb9070 0x55ccc2c186c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2c70bb0 con 0x55ccc2bb87b0 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2bde860 con 0x55ccc2bb9070 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ccc2bfb9d0 con 0x55ccc2bb3d70 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c7fb9b740 10 client.0 _ll_register_callbacks cb 0x7f5c680586d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c0ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c0ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58210/0 (socket says 172.21.15.106:58210) 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 -- 172.21.15.106:0/4131500978 learned_addr learned my addr 172.21.15.106:0/4131500978 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:03.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2c70670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 -- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 msgr2=0x55ccc2c70670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2c70670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 -- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb9070 msgr2=0x55ccc2c186c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb9070 0x55ccc2c186c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c6ffff640 1 -- 172.21.15.106:0/4131500978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c70009de0 con 0x55ccc2bb87b0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.949+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2c70670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c6ffff640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c0ddc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5c60000c00 tx=0x7f5c600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d99b33d952dc61 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c55ffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c55ffb640 1 -- 172.21.15.106:0/4131500978 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6004c040 con 0x55ccc2bb87b0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c55ffb640 1 -- 172.21.15.106:0/4131500978 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c60032e10 con 0x55ccc2bb87b0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c680890c0 con 0x55ccc2bb87b0 2023-11-24T08:10:03.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c55ffb640 1 -- 172.21.15.106:0/4131500978 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=14) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c6004f740 con 0x55ccc2bb87b0 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c7fb9b740 10 client.5271 fetch_fsmap learned FSMap version 14 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.950+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=14,monmap=2+}) v3 -- 0x55ccc2ae1170 con 0x55ccc2bb87b0 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c55ffb640 1 -- 172.21.15.106:0/4131500978 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680890c0 con 0x55ccc2bb87b0 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c55ffb640 1 -- 172.21.15.106:0/4131500978 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 14) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5c6004f050 con 0x55ccc2bb87b0 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c7fb9b740 10 client.5271 fetch_fsmap finished waiting for FSMap version 14 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c7fb9b740 10 client.5271 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c7fb9b740 20 client.5271 populate_metadata read hostname 'smithi106' 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2bdf6b0 0x55ccc2bdfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c7fb9b740 4 client.5271 mds_command: new command op to 5082 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 --> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ccc28206c0 con 0x55ccc2bdf6b0 2023-11-24T08:10:03.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.951+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2bdf6b0 0x55ccc2bdfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c76d9d640 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2bdf6b0 0x55ccc2bdfa90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d80229318cf566 server_cookie=88fa6bb83d84a33c in_seq=0 out_seq=0 2023-11-24T08:10:03.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c55ffb640 10 client.5271 ms_handle_connect on v2:172.21.15.106:6834/3854927347 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c55ffb640 1 -- 172.21.15.106:0/4131500978 <== mds.0 v2:172.21.15.106:6834/3854927347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55ccc28206c0 con 0x55ccc2bdf6b0 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c55ffb640 10 client.5271 handle_command_reply: tid=0 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 client.5271 shutdown 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 msgr2=0x55ccc2c0ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c0ddc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5c60000c00 tx=0x7f5c600046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 shutdown_connections 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:6834/3854927347,v1:172.21.15.106:6836/3854927347] conn(0x55ccc2bdf6b0 0x55ccc2bdfa90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ccc2bb9070 0x55ccc2c186c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55ccc2bb87b0 0x55ccc2c0ddc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 --2- 172.21.15.106:0/4131500978 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55ccc2bb3d70 0x55ccc2c70670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c55ffb640 0 client.5271 ms_handle_reset on v2:172.21.15.106:6834/3854927347 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 >> 172.21.15.106:0/4131500978 conn(0x55ccc2b7ac70 msgr2=0x55ccc2bb2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 shutdown_connections 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 1 -- 172.21.15.106:0/4131500978 wait complete. 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.952+0000 7f5c7fb9b740 20 client.5271 trim_cache size 0 max 16384 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d1a897c2-15ce-4272-ad66-4b3723970cb1", 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5082, 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2023-11-24T08:10:03.958 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 6.6298935800000001, 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 14, 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 36, 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 36, 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 16.169245935999999 2023-11-24T08:10:03.959 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5c5806ae70 msgr2=0x7f5c5806d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5c5806ae70 0x7f5c5806d330 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000ba00 tx=0x7f5c60032000 comp rx=0 tx=0).stop 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 msgr2=0x7f5c78156aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c78156aa0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5c68001eb0 tx=0x7f5c6800db90 comp rx=0 tx=0).stop 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 shutdown_connections 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5c5806ae70 0x7f5c5806d330 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5c78157d60 0x7f5c781510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c7814b940 0x7f5c78157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 --2- 172.21.15.106:0/221419610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5c640069d0 0x7f5c78156aa0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 >> 172.21.15.106:0/221419610 conn(0x7f5c780cbb40 msgr2=0x7f5c780ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 shutdown_connections 2023-11-24T08:10:03.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:03.953+0000 7f5c7c806640 1 -- 172.21.15.106:0/221419610 wait complete. 2023-11-24T08:10:03.971 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:10:04.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.172+0000 7f657874c640 1 Processor -- start 2023-11-24T08:10:04.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.172+0000 7f657874c640 1 -- start start 2023-11-24T08:10:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f657874c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 0x7f657006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f657874c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f65700cf450 0x7f65700cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f657874c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65700cece0 con 0x7f65700cdfb0 2023-11-24T08:10:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f657874c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6570156010 con 0x7f65700cf450 2023-11-24T08:10:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f657874c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657006ce30 con 0x7f657015c7f0 2023-11-24T08:10:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f6576ce9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f65700cdfb0 0x7f65700ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36610/0 (socket says 172.21.15.106:36610) 2023-11-24T08:10:04.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f6576ce9640 1 -- 172.21.15.106:0/1172687591 learned_addr learned my addr 172.21.15.106:0/1172687591 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f65764e8640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 0x7f657006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.174+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953561340 0 0) 0x7f65700cece0 con 0x7f65700cdfb0 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f655c003520 con 0x7f65700cdfb0 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f6575ce7640 1 --2- 172.21.15.106:0/1172687591 >> v2:172.21.15.156:3300/0 conn(0x7f65700cf450 0x7f65700cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1549603834 0 0) 0x7f655c003520 con 0x7f65700cdfb0 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 msgr2=0x7f657006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:04.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 0x7f657006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 >> v2:172.21.15.156:3300/0 conn(0x7f65700cf450 msgr2=0x7f65700cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 --2- 172.21.15.106:0/1172687591 >> v2:172.21.15.156:3300/0 conn(0x7f65700cf450 0x7f65700cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657006d040 con 0x7f65700cdfb0 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f65680019f0 con 0x7f65700cdfb0 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 >> v1:172.21.15.106:6789/0 conn(0x7f65700cdfb0 legacy=0x7f65700ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 0x7f655c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c003fa0 0x7f655c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c004d00 0x7f655c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6570156010 con 0x7f655c004d00 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657006ce30 con 0x7f655c003fa0 2023-11-24T08:10:04.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655c00ba80 con 0x7f657015c7f0 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.175+0000 7f6576ce9640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c004d00 0x7f655c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6576ce9640 1 -- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 msgr2=0x7f655c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6576ce9640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f657015c7f0 0x7f655c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6575ce7640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c003fa0 0x7f655c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6576ce9640 1 -- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c003fa0 msgr2=0x7f655c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6576ce9640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c003fa0 0x7f655c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6576ce9640 1 -- 172.21.15.106:0/1172687591 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c003520 con 0x7f655c004d00 2023-11-24T08:10:04.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.176+0000 7f6575ce7640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c003fa0 0x7f655c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.177+0000 7f6576ce9640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c004d00 0x7f655c005180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f656800a860 tx=0x7f655c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77492574695133c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:04.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.177+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568034630 con 0x7f655c004d00 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.177+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656802f070 con 0x7f655c004d00 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.177+0000 7f65754e6640 1 -- 172.21.15.106:0/1172687591 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568034a30 con 0x7f655c004d00 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f657874c640 1 -- 172.21.15.106:0/1172687591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c004d00 msgr2=0x7f655c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f657874c640 1 --2- 172.21.15.106:0/1172687591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f655c004d00 0x7f655c005180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f656800a860 tx=0x7f655c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f6576ce9640 1 -- 172.21.15.106:0/1172687591 reap_dead start 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f657874c640 1 -- 172.21.15.106:0/1172687591 shutdown_connections 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f657874c640 1 -- 172.21.15.106:0/1172687591 >> 172.21.15.106:0/1172687591 conn(0x7f657006a920 msgr2=0x7f657006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f657874c640 1 -- 172.21.15.106:0/1172687591 shutdown_connections 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.178+0000 7f657874c640 1 -- 172.21.15.106:0/1172687591 wait complete. 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.180+0000 7f657874c640 1 Processor -- start 2023-11-24T08:10:04.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.180+0000 7f657874c640 1 -- start start 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f657874c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c004d00 0x7f6570099140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f657874c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 0x7f65701473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f657874c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65700ad010 0x7f65700ad490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f657874c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65700ad9d0 con 0x7f65701f4510 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f657874c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657009b9c0 con 0x7f655c004d00 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f657874c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65701f4990 con 0x7f65700ad010 2023-11-24T08:10:04.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f6575ce7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 0x7f65701473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f6575ce7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 0x7f65701473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58228/0 (socket says 172.21.15.106:58228) 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.181+0000 7f6575ce7640 1 -- 172.21.15.106:0/3120718593 learned_addr learned my addr 172.21.15.106:0/3120718593 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6575ce7640 1 -- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65700ad010 msgr2=0x7f65700ad490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6575ce7640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65700ad010 0x7f65700ad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f65764e8640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c004d00 0x7f6570099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6575ce7640 1 -- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c004d00 msgr2=0x7f6570099140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6575ce7640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c004d00 0x7f6570099140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6575ce7640 1 -- 172.21.15.106:0/3120718593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65701f4c40 con 0x7f65701f4510 2023-11-24T08:10:04.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6575ce7640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 0x7f65701473e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f656000e790 tx=0x7f656000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:04.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000c6d0 con 0x7f65701f4510 2023-11-24T08:10:04.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f656000c830 con 0x7f65701f4510 2023-11-24T08:10:04.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656000cba0 con 0x7f65701f4510 2023-11-24T08:10:04.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65701f4f20 con 0x7f65701f4510 2023-11-24T08:10:04.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.182+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f657009bc40 con 0x7f65701f4510 2023-11-24T08:10:04.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.183+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65701f5440 con 0x7f65701f4510 2023-11-24T08:10:04.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.184+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f656000cd00 con 0x7f65701f4510 2023-11-24T08:10:04.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.185+0000 7f6566ffd640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f655406ae40 0x7f655406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:04.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.185+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65600893c0 con 0x7f65701f4510 2023-11-24T08:10:04.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.186+0000 7f65764e8640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f655406ae40 0x7f655406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:04.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.186+0000 7f65764e8640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f655406ae40 0x7f655406d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f656c006a90 tx=0x7f656c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:04.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.190+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6560054460 con 0x7f65701f4510 2023-11-24T08:10:04.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.422+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6570067470 con 0x7f65701f4510 2023-11-24T08:10:04.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.423+0000 7f6566ffd640 1 -- 172.21.15.106:0/3120718593 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6560057a80 con 0x7f65701f4510 2023-11-24T08:10:04.425 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:10:04.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.426+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f655406ae40 msgr2=0x7f655406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:04.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.426+0000 7f657874c640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f655406ae40 0x7f655406d300 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f656c006a90 tx=0x7f656c008040 comp rx=0 tx=0).stop 2023-11-24T08:10:04.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.426+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 msgr2=0x7f65701473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:04.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.426+0000 7f657874c640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 0x7f65701473e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f656000e790 tx=0x7f656000ec50 comp rx=0 tx=0).stop 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 shutdown_connections 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f655406ae40 0x7f655406d300 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f65700ad010 0x7f65700ad490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f65701f4510 0x7f65701473e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 --2- 172.21.15.106:0/3120718593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f655c004d00 0x7f6570099140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 >> 172.21.15.106:0/3120718593 conn(0x7f657006a920 msgr2=0x7f657015eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 shutdown_connections 2023-11-24T08:10:04.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:04.427+0000 7f657874c640 1 -- 172.21.15.106:0/3120718593 wait complete. 2023-11-24T08:10:04.447 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:10:04.447 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:10:04.447 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:10:04.447 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:10:04.447 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:04.448 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:10:04.448 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:10:04.448 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:10:04.448 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:10:04.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:04.448 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.501 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:10:04.502 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42312sec preferred_lft 42312sec 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a407:f7ff:fe84:63d0/64 scope link 2023-11-24T08:10:04.503 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.504 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:04.504 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:10:04.504 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9079:d8ff:febb:c08c/64 scope link 2023-11-24T08:10:04.504 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.504 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:04.504 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:10:04.505 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::20e3:f2ff:fe93:487e/64 scope link 2023-11-24T08:10:04.505 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:04.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:04.505 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:10:04.594 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:10:04.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:04.598 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:10:04.652 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:10:04.652 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:10:04.652 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:10:04.652 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:10:04.653 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:10:04.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:04.653 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:10:04.653 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:10:04.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:10:04.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:10:04.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:10:04.653 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:10:04.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:04.839 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:10:04.891 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:10:04.891 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:10:04.893 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:10:04.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:04.893 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2023-11-24T08:10:04.990 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:04.990 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:05.046 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:10:05.046 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:05.046 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:05.097 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:10:05.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:05.098 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:05.242 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:05.242 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:05.276 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:05.276 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:05.276 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:10:05.276 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:10:05.276 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:10:05.278 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-24T08:10:05.278 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:10:05.278 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:10:05.278 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:10:05.278 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:10:05.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:10:05.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:05.281 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:05.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:05.371 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:10:05.437 INFO:teuthology.orchestra.run.smithi106.stdout:client4774 2023-11-24T08:10:05.438 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:05.438 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:05.487 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:10:05.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.733+0000 7f333b0ee640 1 Processor -- start 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.733+0000 7f333b0ee640 1 -- start start 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f333b0ee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f333415c7f0 0x7f333406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f333b0ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33340cf450 0x7f33340cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f333b0ee640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340cece0 con 0x7f33340cdfb0 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f333b0ee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3334156010 con 0x7f333415c7f0 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f333b0ee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333406ce30 con 0x7f33340cf450 2023-11-24T08:10:05.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f3338e8a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f333415c7f0 0x7f333406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f3338e8a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f333415c7f0 0x7f333406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58238/0 (socket says 172.21.15.106:58238) 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.735+0000 7f3338e8a640 1 -- 172.21.15.106:0/3478953875 learned_addr learned my addr 172.21.15.106:0/3478953875 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.736+0000 7f3338e8a640 1 -- 172.21.15.106:0/3478953875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33340cf450 msgr2=0x7f33340cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.736+0000 7f3333fff640 1 --2- 172.21.15.106:0/3478953875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33340cf450 0x7f33340cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.736+0000 7f3338e8a640 1 --2- 172.21.15.106:0/3478953875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33340cf450 0x7f33340cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.736+0000 7f3338e8a640 1 -- 172.21.15.106:0/3478953875 >> v1:172.21.15.156:6789/0 conn(0x7f33340cdfb0 legacy=0x7f33340ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.736+0000 7f3338e8a640 1 -- 172.21.15.106:0/3478953875 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f333406d040 con 0x7f333415c7f0 2023-11-24T08:10:05.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.737+0000 7f3338e8a640 1 --2- 172.21.15.106:0/3478953875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f333415c7f0 0x7f333406db20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3328009870 tx=0x7f332802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc9d1e8aec50ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.737+0000 7f33337fe640 1 -- 172.21.15.106:0/3478953875 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328031640 con 0x7f333415c7f0 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.737+0000 7f33337fe640 1 -- 172.21.15.106:0/3478953875 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33280317a0 con 0x7f333415c7f0 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.737+0000 7f33337fe640 1 -- 172.21.15.106:0/3478953875 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328031b30 con 0x7f333415c7f0 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 -- 172.21.15.106:0/3478953875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f333415c7f0 msgr2=0x7f333406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 --2- 172.21.15.106:0/3478953875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f333415c7f0 0x7f333406db20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3328009870 tx=0x7f332802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 -- 172.21.15.106:0/3478953875 shutdown_connections 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 --2- 172.21.15.106:0/3478953875 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f33340cf450 0x7f33340cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 --2- 172.21.15.106:0/3478953875 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f333415c7f0 0x7f333406db20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 -- 172.21.15.106:0/3478953875 >> 172.21.15.106:0/3478953875 conn(0x7f333406a920 msgr2=0x7f333406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 -- 172.21.15.106:0/3478953875 shutdown_connections 2023-11-24T08:10:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.738+0000 7f333b0ee640 1 -- 172.21.15.106:0/3478953875 wait complete. 2023-11-24T08:10:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.740+0000 7f333b0ee640 1 Processor -- start 2023-11-24T08:10:05.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.740+0000 7f333b0ee640 1 -- start start 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.741+0000 7f333b0ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 0x7f3334156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.741+0000 7f333b0ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33340cf450 0x7f3334157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.741+0000 7f333b0ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f333415c7f0 0x7f33340994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.741+0000 7f333b0ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334156730 con 0x7f33340cdfb0 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.741+0000 7f333b0ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3334156a10 con 0x7f33340cf450 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.741+0000 7f333b0ee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3334156cf0 con 0x7f333415c7f0 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 0x7f3334156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:05.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 0x7f3334156150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58240/0 (socket says 172.21.15.106:58240) 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 -- 172.21.15.106:0/1871548457 learned_addr learned my addr 172.21.15.106:0/1871548457 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f333968b640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f333415c7f0 0x7f33340994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3333fff640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33340cf450 0x7f3334157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 -- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f333415c7f0 msgr2=0x7f33340994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f333415c7f0 0x7f33340994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 -- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33340cf450 msgr2=0x7f3334157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33340cf450 0x7f3334157600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3338e8a640 1 -- 172.21.15.106:0/1871548457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33280094d0 con 0x7f33340cdfb0 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.742+0000 7f3333fff640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33340cf450 0x7f3334157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.743+0000 7f3338e8a640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 0x7f3334156150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f33280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.743+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332803ac60 con 0x7f33340cdfb0 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.743+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332803ade0 con 0x7f33340cdfb0 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.743+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3328044630 con 0x7f33340cdfb0 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.743+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3334156fd0 con 0x7f33340cdfb0 2023-11-24T08:10:05.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.743+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333409bdb0 con 0x7f33340cdfb0 2023-11-24T08:10:05.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.744+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3334067470 con 0x7f33340cdfb0 2023-11-24T08:10:05.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.745+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f332802f050 con 0x7f33340cdfb0 2023-11-24T08:10:05.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.746+0000 7f33317fa640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f331806ae40 0x7f331806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:05.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.746+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33280ab730 con 0x7f33340cdfb0 2023-11-24T08:10:05.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.746+0000 7f3333fff640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f331806ae40 0x7f331806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:05.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.747+0000 7f3333fff640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f331806ae40 0x7f331806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33341f5780 tx=0x7f3320008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:05.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.750+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f33280767e0 con 0x7f33340cdfb0 2023-11-24T08:10:05.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.981+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33340cd730 con 0x7f33340cdfb0 2023-11-24T08:10:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.982+0000 7f33317fa640 1 -- 172.21.15.106:0/1871548457 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v14) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3328079e00 con 0x7f33340cdfb0 2023-11-24T08:10:05.983 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:10:05.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f331806ae40 msgr2=0x7f331806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:05.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f331806ae40 0x7f331806d300 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f33341f5780 tx=0x7f3320008040 comp rx=0 tx=0).stop 2023-11-24T08:10:05.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 msgr2=0x7f3334156150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:05.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 0x7f3334156150 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3328000c00 tx=0x7f33280046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 shutdown_connections 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f331806ae40 0x7f331806d300 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f333415c7f0 0x7f33340994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33340cf450 0x7f3334157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.985+0000 7f333b0ee640 1 --2- 172.21.15.106:0/1871548457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f33340cdfb0 0x7f3334156150 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.986+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 >> 172.21.15.106:0/1871548457 conn(0x7f333406a920 msgr2=0x7f333415eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.986+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 shutdown_connections 2023-11-24T08:10:05.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:05.986+0000 7f333b0ee640 1 -- 172.21.15.106:0/1871548457 wait complete. 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:10:06.012 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:10:06.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.013 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:10:06.075 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:10:06.075 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:10:06.075 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:10:06.075 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:06.076 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42326sec preferred_lft 42326sec 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::1831:a6ff:fe65:552d/64 scope link 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:06.077 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:10:06.078 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::683b:9fff:fe67:7152/64 scope link 2023-11-24T08:10:06.078 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.078 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:06.078 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether a2:a3:a7:89:e5:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:10:06.078 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a0a3:a7ff:fe89:e507/64 scope link 2023-11-24T08:10:06.078 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:06.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.079 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:10:06.171 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:10:06.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.176 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:10:06.233 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:10:06.233 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:10:06.234 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:10:06.234 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:10:06.234 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:10:06.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.234 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:10:06.235 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:10:06.235 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:10:06.235 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:10:06.235 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:10:06.235 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:10:06.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.423 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:10:06.473 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:10:06.473 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:10:06.474 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:10:06.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.475 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-24T08:10:06.566 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:06.567 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:06.622 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:10:06.623 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:06.623 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:06.676 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:10:06.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.676 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:06.817 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:06.818 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:06.852 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:06.852 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:06.852 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:10:06.852 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 1@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:10:06.852 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:10:06.855 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-24T08:10:06.855 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:10:06.855 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:10:06.855 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:10:06.855 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:10:06.859 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:10:06.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.860 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:06.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:06.947 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:10:07.003 INFO:teuthology.orchestra.run.smithi156.stdout:client4568 2023-11-24T08:10:07.003 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:07.004 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:07.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:10:07.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.284+0000 7f53754ad640 1 Processor -- start 2023-11-24T08:10:07.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.284+0000 7f53754ad640 1 -- start start 2023-11-24T08:10:07.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.285+0000 7f53754ad640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f537015c7f0 0x7f537006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:07.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f53754ad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53700ce6c0 0x7f53700cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f53754ad640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53700cece0 con 0x7f53700cf450 2023-11-24T08:10:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f53754ad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700cfe40 con 0x7f53700ce6c0 2023-11-24T08:10:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f53754ad640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53700ceaf0 con 0x7f537015c7f0 2023-11-24T08:10:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53700ce6c0 0x7f53700cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53700ce6c0 0x7f53700cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58246/0 (socket says 172.21.15.106:58246) 2023-11-24T08:10:07.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536f7fe640 1 -- 172.21.15.106:0/3013899618 learned_addr learned my addr 172.21.15.106:0/3013899618 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536effd640 1 --2- 172.21.15.106:0/3013899618 >> v2:172.21.15.156:3300/0 conn(0x7f537015c7f0 0x7f537006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536f7fe640 1 -- 172.21.15.106:0/3013899618 >> v2:172.21.15.156:3300/0 conn(0x7f537015c7f0 msgr2=0x7f537006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536f7fe640 1 --2- 172.21.15.106:0/3013899618 >> v2:172.21.15.156:3300/0 conn(0x7f537015c7f0 0x7f537006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.286+0000 7f536f7fe640 1 -- 172.21.15.106:0/3013899618 >> v1:172.21.15.156:6789/0 conn(0x7f53700cf450 legacy=0x7f53700cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.287+0000 7f536f7fe640 1 -- 172.21.15.106:0/3013899618 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537006d040 con 0x7f53700ce6c0 2023-11-24T08:10:07.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.288+0000 7f536f7fe640 1 --2- 172.21.15.106:0/3013899618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53700ce6c0 0x7f53700cf900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53700d0130 tx=0x7f536802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9ae8221790c946e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.288+0000 7f536e7fc640 1 -- 172.21.15.106:0/3013899618 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536802ca80 con 0x7f53700ce6c0 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.288+0000 7f536e7fc640 1 -- 172.21.15.106:0/3013899618 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536802cbe0 con 0x7f53700ce6c0 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.288+0000 7f536e7fc640 1 -- 172.21.15.106:0/3013899618 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53680343f0 con 0x7f53700ce6c0 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.288+0000 7f53754ad640 1 -- 172.21.15.106:0/3013899618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53700ce6c0 msgr2=0x7f53700cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.288+0000 7f53754ad640 1 --2- 172.21.15.106:0/3013899618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53700ce6c0 0x7f53700cf900 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f53700d0130 tx=0x7f536802bee0 comp rx=0 tx=0).stop 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.289+0000 7f53754ad640 1 -- 172.21.15.106:0/3013899618 shutdown_connections 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.289+0000 7f53754ad640 1 --2- 172.21.15.106:0/3013899618 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53700ce6c0 0x7f53700cf900 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.289+0000 7f53754ad640 1 --2- 172.21.15.106:0/3013899618 >> v2:172.21.15.156:3300/0 conn(0x7f537015c7f0 0x7f537006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:07.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.289+0000 7f53754ad640 1 -- 172.21.15.106:0/3013899618 >> 172.21.15.106:0/3013899618 conn(0x7f537006a920 msgr2=0x7f537006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:07.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.289+0000 7f53754ad640 1 -- 172.21.15.106:0/3013899618 shutdown_connections 2023-11-24T08:10:07.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.289+0000 7f53754ad640 1 -- 172.21.15.106:0/3013899618 wait complete. 2023-11-24T08:10:07.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.290+0000 7f53754ad640 1 Processor -- start 2023-11-24T08:10:07.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.291+0000 7f53754ad640 1 -- start start 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.291+0000 7f53754ad640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53700ce6c0 0x7f53700abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.291+0000 7f53754ad640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53700cf450 0x7f53701f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f53754ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 0x7f53700ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f53754ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700ac650 con 0x7f537015c7f0 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f53754ad640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53700ac930 con 0x7f53700ce6c0 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f53754ad640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53700acc10 con 0x7f53700cf450 2023-11-24T08:10:07.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 0x7f53700ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 0x7f53700ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58258/0 (socket says 172.21.15.106:58258) 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 -- 172.21.15.106:0/1267190293 learned_addr learned my addr 172.21.15.106:0/1267190293 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536effd640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53700ce6c0 0x7f53700abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 -- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53700cf450 msgr2=0x7f53701f3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f5367fff640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53700cf450 0x7f53701f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53700cf450 0x7f53701f3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 -- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53700ce6c0 msgr2=0x7f53700abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:07.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53700ce6c0 0x7f53700abfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536f7fe640 1 -- 172.21.15.106:0/1267190293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5368009b30 con 0x7f537015c7f0 2023-11-24T08:10:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f5367fff640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53700cf450 0x7f53701f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.292+0000 7f536effd640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53700ce6c0 0x7f53700abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:07.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.293+0000 7f536f7fe640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 0x7f53700ad6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5368036040 tx=0x7f5368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:07.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.293+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5368009510 con 0x7f537015c7f0 2023-11-24T08:10:07.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.293+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f536803e450 con 0x7f537015c7f0 2023-11-24T08:10:07.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.293+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f536803e800 con 0x7f537015c7f0 2023-11-24T08:10:07.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.293+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53700acef0 con 0x7f537015c7f0 2023-11-24T08:10:07.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.293+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53700b3aa0 con 0x7f537015c7f0 2023-11-24T08:10:07.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.294+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53700ad3b0 con 0x7f537015c7f0 2023-11-24T08:10:07.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.295+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f536802cd50 con 0x7f537015c7f0 2023-11-24T08:10:07.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.296+0000 7f53677fe640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f535006ae40 0x7f535006d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:07.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.297+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53680ac2a0 con 0x7f537015c7f0 2023-11-24T08:10:07.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.297+0000 7f536effd640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f535006ae40 0x7f535006d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:07.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.297+0000 7f536effd640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f535006ae40 0x7f535006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f536000b820 tx=0x7f5360008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:07.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.301+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5368077220 con 0x7f537015c7f0 2023-11-24T08:10:07.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:07.531+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f5370067470 con 0x7f537015c7f0 2023-11-24T08:10:08.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.381+0000 7f53677fe640 1 -- 172.21.15.106:0/1267190293 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v15) v1 ==== 136+0+0 (secure 0 0 0) 0x7f536807a840 con 0x7f537015c7f0 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f535006ae40 msgr2=0x7f535006d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f535006ae40 0x7f535006d300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f536000b820 tx=0x7f5360008040 comp rx=0 tx=0).stop 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 msgr2=0x7f53700ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 0x7f53700ad6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5368036040 tx=0x7f5368004a40 comp rx=0 tx=0).stop 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 shutdown_connections 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f537015c7f0 0x7f53700ad6f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f535006ae40 0x7f535006d300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53700cf450 0x7f53701f3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 --2- 172.21.15.106:0/1267190293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53700ce6c0 0x7f53700abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.385+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 >> 172.21.15.106:0/1267190293 conn(0x7f537006a920 msgr2=0x7f53700734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.386+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 shutdown_connections 2023-11-24T08:10:08.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.386+0000 7f53754ad640 1 -- 172.21.15.106:0/1267190293 wait complete. 2023-11-24T08:10:08.412 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:10:08.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.621+0000 7f87b0b90640 1 Processor -- start 2023-11-24T08:10:08.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.621+0000 7f87b0b90640 1 -- start start 2023-11-24T08:10:08.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87b0b90640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f87ac0ccbd0 0x7f87ac1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:08.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87b0b90640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f87ac0c9650 0x7f87ac0c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:08.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87b0b90640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87ac1655f0 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87b0b90640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac0cc380 con 0x7f87ac0ccbd0 2023-11-24T08:10:08.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87b0b90640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87ac162b30 con 0x7f87ac0c9650 2023-11-24T08:10:08.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87a3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f87ac0ccfb0 0x7f87ac162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36618/0 (socket says 172.21.15.106:36618) 2023-11-24T08:10:08.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87a3fff640 1 -- 172.21.15.106:0/2460073026 learned_addr learned my addr 172.21.15.106:0/2460073026 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:08.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.623+0000 7f87aa59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f87ac0ccbd0 0x7f87ac1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:08.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87aad9d640 1 --2- 172.21.15.106:0/2460073026 >> v2:172.21.15.156:3300/0 conn(0x7f87ac0c9650 0x7f87ac0c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:08.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87a9d9b640 1 -- 172.21.15.106:0/2460073026 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920814887 0 0) 0x7f87ac1655f0 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87a9d9b640 1 -- 172.21.15.106:0/2460073026 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8798003520 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87aa59c640 1 -- 172.21.15.106:0/2460073026 >> v2:172.21.15.156:3300/0 conn(0x7f87ac0c9650 msgr2=0x7f87ac0c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:08.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87aa59c640 1 --2- 172.21.15.106:0/2460073026 >> v2:172.21.15.156:3300/0 conn(0x7f87ac0c9650 0x7f87ac0c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87aa59c640 1 -- 172.21.15.106:0/2460073026 >> v1:172.21.15.106:6789/0 conn(0x7f87ac0ccfb0 legacy=0x7f87ac162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87aa59c640 1 -- 172.21.15.106:0/2460073026 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87ac162e10 con 0x7f87ac0ccbd0 2023-11-24T08:10:08.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.624+0000 7f87aad9d640 1 --2- 172.21.15.106:0/2460073026 >> v2:172.21.15.156:3300/0 conn(0x7f87ac0c9650 0x7f87ac0c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:10:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.626+0000 7f87aa59c640 1 --2- 172.21.15.106:0/2460073026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccbd0 0x7f87ac1660b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0c5fe0 tx=0x7f879c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9476d6c03a40f8fa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.626+0000 7f87a9d9b640 1 -- 172.21.15.106:0/2460073026 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c031830 con 0x7f87ac0ccbd0 2023-11-24T08:10:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.626+0000 7f87a9d9b640 1 -- 172.21.15.106:0/2460073026 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f879c031990 con 0x7f87ac0ccbd0 2023-11-24T08:10:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.626+0000 7f87a9d9b640 1 -- 172.21.15.106:0/2460073026 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f879c031d00 con 0x7f87ac0ccbd0 2023-11-24T08:10:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 -- 172.21.15.106:0/2460073026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccbd0 msgr2=0x7f87ac1660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 --2- 172.21.15.106:0/2460073026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccbd0 0x7f87ac1660b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f87ac0c5fe0 tx=0x7f879c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 -- 172.21.15.106:0/2460073026 shutdown_connections 2023-11-24T08:10:08.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 --2- 172.21.15.106:0/2460073026 >> v2:172.21.15.156:3300/0 conn(0x7f87ac0c9650 0x7f87ac0c9ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 --2- 172.21.15.106:0/2460073026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccbd0 0x7f87ac1660b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 -- 172.21.15.106:0/2460073026 >> 172.21.15.106:0/2460073026 conn(0x7f87ac0b0c50 msgr2=0x7f87ac0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:08.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 -- 172.21.15.106:0/2460073026 shutdown_connections 2023-11-24T08:10:08.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.627+0000 7f87b0b90640 1 -- 172.21.15.106:0/2460073026 wait complete. 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.629+0000 7f87b0b90640 1 Processor -- start 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.629+0000 7f87b0b90640 1 -- start start 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87b0b90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87ac0c9650 0x7f87ac1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87b0b90640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87ac0ccbd0 0x7f87ac158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87b0b90640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 0x7f87ac1632c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87b0b90640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac165230 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87b0b90640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac164cb0 con 0x7f87ac0c9650 2023-11-24T08:10:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87b0b90640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac164f50 con 0x7f87ac0ccbd0 2023-11-24T08:10:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87aad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 0x7f87ac1632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87aad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 0x7f87ac1632c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58284/0 (socket says 172.21.15.106:58284) 2023-11-24T08:10:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.630+0000 7f87aad9d640 1 -- 172.21.15.106:0/4032946664 learned_addr learned my addr 172.21.15.106:0/4032946664 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:08.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aa59c640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87ac0c9650 0x7f87ac1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87a3fff640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87ac0ccbd0 0x7f87ac158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:08.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aad9d640 1 -- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87ac0ccbd0 msgr2=0x7f87ac158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aad9d640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87ac0ccbd0 0x7f87ac158830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aad9d640 1 -- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87ac0c9650 msgr2=0x7f87ac1582f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aad9d640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87ac0c9650 0x7f87ac1582f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aad9d640 1 -- 172.21.15.106:0/4032946664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a4009a70 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aa59c640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87ac0c9650 0x7f87ac1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87aad9d640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 0x7f87ac1632c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f87a400efc0 tx=0x7f87a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4009cd0 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a400f040 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.631+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c009400 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.632+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4003990 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.632+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac1617d0 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.633+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac15c9a0 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.634+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f87a4014480 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.635+0000 7f87a2ffd640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f878c06ae40 0x7f878c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:08.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.635+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87a408a190 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.635+0000 7f87aa59c640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f878c06ae40 0x7f878c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:08.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.636+0000 7f87aa59c640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f878c06ae40 0x7f878c06d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f879c036000 tx=0x7f879c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:08.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.639+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f87a40552b0 con 0x7f87ac0ccfb0 2023-11-24T08:10:08.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.852+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f87ac0cd410 con 0x7f878c06ae40 2023-11-24T08:10:08.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.855+0000 7f87a2ffd640 1 -- 172.21.15.106:0/4032946664 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f879c031c00 con 0x7f878c06ae40 2023-11-24T08:10:08.857 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:10:08.857 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:08.857 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2023-11-24T08:10:08.858 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:08.858 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:10:08.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f878c06ae40 msgr2=0x7f878c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f878c06ae40 0x7f878c06d300 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f879c036000 tx=0x7f879c03c040 comp rx=0 tx=0).stop 2023-11-24T08:10:08.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 msgr2=0x7f87ac1632c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:08.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 0x7f87ac1632c0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f87a400efc0 tx=0x7f87a400c490 comp rx=0 tx=0).stop 2023-11-24T08:10:08.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 shutdown_connections 2023-11-24T08:10:08.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f87ac0ccfb0 0x7f87ac1632c0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f878c06ae40 0x7f878c06d300 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f87ac0ccbd0 0x7f87ac158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 --2- 172.21.15.106:0/4032946664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87ac0c9650 0x7f87ac1582f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:08.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.857+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 >> 172.21.15.106:0/4032946664 conn(0x7f87ac0b0c50 msgr2=0x7f87ac0aea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:08.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.858+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 shutdown_connections 2023-11-24T08:10:08.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:08.858+0000 7f87b0b90640 1 -- 172.21.15.106:0/4032946664 wait complete. 2023-11-24T08:10:08.875 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:10:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.085+0000 7f454d390640 1 Processor -- start 2023-11-24T08:10:09.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.085+0000 7f454d390640 1 -- start start 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.087+0000 7f454d390640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4548140520 0x7f45480c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.087+0000 7f454d390640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45480c65f0 0x7f4548161400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f454d390640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f454815de00 con 0x7f4548161940 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f454d390640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45480c6a70 con 0x7f45480c65f0 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f454d390640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f454815e0e0 con 0x7f4548140520 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45480c65f0 0x7f4548161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45480c65f0 0x7f4548161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58286/0 (socket says 172.21.15.106:58286) 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 -- 172.21.15.106:0/3821602676 learned_addr learned my addr 172.21.15.106:0/3821602676 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f4546ffd640 1 --2- 172.21.15.106:0/3821602676 >> v2:172.21.15.156:3300/0 conn(0x7f4548140520 0x7f45480c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 -- 172.21.15.106:0/3821602676 >> v2:172.21.15.156:3300/0 conn(0x7f4548140520 msgr2=0x7f45480c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:09.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 --2- 172.21.15.106:0/3821602676 >> v2:172.21.15.156:3300/0 conn(0x7f4548140520 0x7f45480c60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 -- 172.21.15.106:0/3821602676 >> v1:172.21.15.156:6789/0 conn(0x7f4548161940 legacy=0x7f4548142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:09.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.088+0000 7f453e7fc640 1 -- 172.21.15.106:0/3821602676 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454815e3c0 con 0x7f45480c65f0 2023-11-24T08:10:09.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.089+0000 7f453e7fc640 1 --2- 172.21.15.106:0/3821602676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548161400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f454806e0b0 tx=0x7f453002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42906fc826f4f2b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:09.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f45467fc640 1 -- 172.21.15.106:0/3821602676 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530031870 con 0x7f45480c65f0 2023-11-24T08:10:09.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f45467fc640 1 -- 172.21.15.106:0/3821602676 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45300319d0 con 0x7f45480c65f0 2023-11-24T08:10:09.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f45467fc640 1 -- 172.21.15.106:0/3821602676 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4530031d60 con 0x7f45480c65f0 2023-11-24T08:10:09.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f454d390640 1 -- 172.21.15.106:0/3821602676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 msgr2=0x7f4548161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f454d390640 1 --2- 172.21.15.106:0/3821602676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548161400 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f454806e0b0 tx=0x7f453002b910 comp rx=0 tx=0).stop 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f454d390640 1 -- 172.21.15.106:0/3821602676 shutdown_connections 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f454d390640 1 --2- 172.21.15.106:0/3821602676 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548161400 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f454d390640 1 --2- 172.21.15.106:0/3821602676 >> v2:172.21.15.156:3300/0 conn(0x7f4548140520 0x7f45480c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.090+0000 7f454d390640 1 -- 172.21.15.106:0/3821602676 >> 172.21.15.106:0/3821602676 conn(0x7f45480b2e40 msgr2=0x7f45480b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.091+0000 7f454d390640 1 -- 172.21.15.106:0/3821602676 shutdown_connections 2023-11-24T08:10:09.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.091+0000 7f454d390640 1 -- 172.21.15.106:0/3821602676 wait complete. 2023-11-24T08:10:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.092+0000 7f454d390640 1 Processor -- start 2023-11-24T08:10:09.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.092+0000 7f454d390640 1 -- start start 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.093+0000 7f454d390640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.093+0000 7f454d390640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4548140520 0x7f45480ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.093+0000 7f454d390640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4548161940 0x7f45480cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.093+0000 7f454d390640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45480c82e0 con 0x7f45480c65f0 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.093+0000 7f454d390640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4548153a40 con 0x7f4548140520 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.093+0000 7f454d390640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45480cf6e0 con 0x7f4548161940 2023-11-24T08:10:09.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58302/0 (socket says 172.21.15.106:58302) 2023-11-24T08:10:09.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 -- 172.21.15.106:0/2219564386 learned_addr learned my addr 172.21.15.106:0/2219564386 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f453e7fc640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4548140520 0x7f45480ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 -- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4548161940 msgr2=0x7f45480cf0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4548161940 0x7f45480cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 -- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4548140520 msgr2=0x7f45480ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4548140520 0x7f45480ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 -- 172.21.15.106:0/2219564386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45300094d0 con 0x7f45480c65f0 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.094+0000 7f4546ffd640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548153500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f453800c7f0 tx=0x7f453800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.095+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538004220 con 0x7f45480c65f0 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.095+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45380043a0 con 0x7f45480c65f0 2023-11-24T08:10:09.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.095+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45480cf9c0 con 0x7f45480c65f0 2023-11-24T08:10:09.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.095+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4538007860 con 0x7f45480c65f0 2023-11-24T08:10:09.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.095+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45480cff80 con 0x7f45480c65f0 2023-11-24T08:10:09.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.096+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f454804e8b0 con 0x7f45480c65f0 2023-11-24T08:10:09.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.097+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f4538007bd0 con 0x7f45480c65f0 2023-11-24T08:10:09.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.098+0000 7f453ffff640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f452806ae40 0x7f452806d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.098+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45380896e0 con 0x7f45480c65f0 2023-11-24T08:10:09.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.098+0000 7f453e7fc640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f452806ae40 0x7f452806d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.099+0000 7f453e7fc640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f452806ae40 0x7f452806d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4530036000 tx=0x7f453003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:09.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.102+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45380546f0 con 0x7f45480c65f0 2023-11-24T08:10:09.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.290+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f45481590c0 con 0x7f45480c65f0 2023-11-24T08:10:09.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.291+0000 7f453ffff640 1 -- 172.21.15.106:0/2219564386 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4538057d10 con 0x7f45480c65f0 2023-11-24T08:10:09.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f452806ae40 msgr2=0x7f452806d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f452806ae40 0x7f452806d300 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f4530036000 tx=0x7f453003c040 comp rx=0 tx=0).stop 2023-11-24T08:10:09.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 msgr2=0x7f4548153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548153500 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f453800c7f0 tx=0x7f453800ccb0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 shutdown_connections 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f452806ae40 0x7f452806d300 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4548161940 0x7f45480cf0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4548140520 0x7f45480ceb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 --2- 172.21.15.106:0/2219564386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45480c65f0 0x7f4548153500 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 >> 172.21.15.106:0/2219564386 conn(0x7f45480b2e40 msgr2=0x7f45480cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 shutdown_connections 2023-11-24T08:10:09.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.294+0000 7f454d390640 1 -- 172.21.15.106:0/2219564386 wait complete. 2023-11-24T08:10:09.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp2797 2023-11-24T08:10:09.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.537+0000 7f80061c4640 1 Processor -- start 2023-11-24T08:10:09.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.538+0000 7f80061c4640 1 -- start start 2023-11-24T08:10:09.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.539+0000 7f80061c4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d1c20 0x7f800006bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.539+0000 7f80061c4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80000d2000 0x7f80000cfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.539+0000 7f80061c4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f80000d0200 0x7f80001605f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.539+0000 7f80061c4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80000ce410 con 0x7f80000d0200 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.539+0000 7f80061c4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80000d06f0 con 0x7f80000d2000 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.539+0000 7f80061c4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80000d09d0 con 0x7f80000d1c20 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f80000d0200 0x7f80001605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f80000d0200 0x7f80001605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58318/0 (socket says 172.21.15.106:58318) 2023-11-24T08:10:09.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 -- 172.21.15.106:0/3307492458 learned_addr learned my addr 172.21.15.106:0/3307492458 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:09.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffeffd640 1 --2- 172.21.15.106:0/3307492458 >> v2:172.21.15.156:3300/0 conn(0x7f80000d2000 0x7f80000cfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7fff7fe640 1 --2- 172.21.15.106:0/3307492458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d1c20 0x7f800006bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 -- 172.21.15.106:0/3307492458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d1c20 msgr2=0x7f800006bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:09.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 --2- 172.21.15.106:0/3307492458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d1c20 0x7f800006bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 -- 172.21.15.106:0/3307492458 >> v2:172.21.15.156:3300/0 conn(0x7f80000d2000 msgr2=0x7f80000cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 --2- 172.21.15.106:0/3307492458 >> v2:172.21.15.156:3300/0 conn(0x7f80000d2000 0x7f80000cfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.540+0000 7f7ffffff640 1 -- 172.21.15.106:0/3307492458 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80000ce0a0 con 0x7f80000d0200 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.541+0000 7f7ffffff640 1 --2- 172.21.15.106:0/3307492458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001605f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80000b5f50 tx=0x7f7ff802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792c5c87ba2e1bda server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.542+0000 7f7ffe7fc640 1 -- 172.21.15.106:0/3307492458 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8031870 con 0x7f80000d0200 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.542+0000 7f7ffe7fc640 1 -- 172.21.15.106:0/3307492458 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff80319d0 con 0x7f80000d0200 2023-11-24T08:10:09.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.542+0000 7f7ffe7fc640 1 -- 172.21.15.106:0/3307492458 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff8031d60 con 0x7f80000d0200 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.542+0000 7f80061c4640 1 -- 172.21.15.106:0/3307492458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 msgr2=0x7f80001605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.542+0000 7f80061c4640 1 --2- 172.21.15.106:0/3307492458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001605f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f80000b5f50 tx=0x7f7ff802b910 comp rx=0 tx=0).stop 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 -- 172.21.15.106:0/3307492458 shutdown_connections 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 --2- 172.21.15.106:0/3307492458 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001605f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 --2- 172.21.15.106:0/3307492458 >> v2:172.21.15.156:3300/0 conn(0x7f80000d2000 0x7f80000cfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 --2- 172.21.15.106:0/3307492458 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d1c20 0x7f800006bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 -- 172.21.15.106:0/3307492458 >> 172.21.15.106:0/3307492458 conn(0x7f800008cea0 msgr2=0x7f800009c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 -- 172.21.15.106:0/3307492458 shutdown_connections 2023-11-24T08:10:09.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.543+0000 7f80061c4640 1 -- 172.21.15.106:0/3307492458 wait complete. 2023-11-24T08:10:09.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.544+0000 7f80061c4640 1 Processor -- start 2023-11-24T08:10:09.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 -- start start 2023-11-24T08:10:09.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001f4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80000d1c20 0x7f8000157510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d2000 0x7f8000157a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80001f54a0 con 0x7f80000d0200 2023-11-24T08:10:09.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80001f5780 con 0x7f80000d1c20 2023-11-24T08:10:09.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.545+0000 7f80061c4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80001f5a60 con 0x7f80000d2000 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001f4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001f4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58322/0 (socket says 172.21.15.106:58322) 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 -- 172.21.15.106:0/253223622 learned_addr learned my addr 172.21.15.106:0/253223622 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 -- 172.21.15.106:0/253223622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d2000 msgr2=0x7f8000157a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d2000 0x7f8000157a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 -- 172.21.15.106:0/253223622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80000d1c20 msgr2=0x7f8000157510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80000d1c20 0x7f8000157510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 -- 172.21.15.106:0/253223622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80094d0 con 0x7f80000d0200 2023-11-24T08:10:09.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.546+0000 7f7fff7fe640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001f4c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d5f0 tx=0x7f7fe800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:09.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.547+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe80125f0 con 0x7f80000d0200 2023-11-24T08:10:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.547+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fe8012750 con 0x7f80000d0200 2023-11-24T08:10:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.547+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8012b00 con 0x7f80000d0200 2023-11-24T08:10:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.547+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80001f5d40 con 0x7f80000d0200 2023-11-24T08:10:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.547+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8000157250 con 0x7f80000d0200 2023-11-24T08:10:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.548+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800004e8b0 con 0x7f80000d0200 2023-11-24T08:10:09.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.549+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f7fe8012c60 con 0x7f80000d0200 2023-11-24T08:10:09.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.550+0000 7f7ff7fff640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fe006ae70 0x7f7fe006d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:09.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.550+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fe8089b40 con 0x7f80000d0200 2023-11-24T08:10:09.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.550+0000 7f7ffeffd640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fe006ae70 0x7f7fe006d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:09.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.551+0000 7f7ffeffd640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fe006ae70 0x7f7fe006d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009540 tx=0x7f7ff0009450 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:09.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.554+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7fe8054b50 con 0x7f80000d0200 2023-11-24T08:10:09.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.767+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp2797", "target": ["mon-mgr", ""]}) v1 -- 0x7f800008af60 con 0x7f7fe006ae70 2023-11-24T08:10:09.776 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:09.775+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:09.777 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:09.775+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:09.777 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:09.775+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:09.777 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:09.775+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:09.777 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:09.775+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:09.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.792+0000 7f7ff7fff640 1 -- 172.21.15.106:0/253223622 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f800008af60 con 0x7f7fe006ae70 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fe006ae70 msgr2=0x7f7fe006d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fe006ae70 0x7f7fe006d330 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009540 tx=0x7f7ff0009450 comp rx=0 tx=0).stop 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 msgr2=0x7f80001f4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001f4c50 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d5f0 tx=0x7f7fe800dab0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 shutdown_connections 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fe006ae70 0x7f7fe006d330 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80000d2000 0x7f8000157a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80000d1c20 0x7f8000157510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 --2- 172.21.15.106:0/253223622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80000d0200 0x7f80001f4c50 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:09.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 >> 172.21.15.106:0/253223622 conn(0x7f800008cea0 msgr2=0x7f80000b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:09.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 shutdown_connections 2023-11-24T08:10:09.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:09.795+0000 7f80061c4640 1 -- 172.21.15.106:0/253223622 wait complete. 2023-11-24T08:10:09.813 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create cephfs subvol_grp2797 subvol_snap2809 2023-11-24T08:10:10.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.005+0000 7f2cba6ad640 1 Processor -- start 2023-11-24T08:10:10.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.005+0000 7f2cba6ad640 1 -- start start 2023-11-24T08:10:10.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cba6ad640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2cb406ddb0 0x7f2cb4071650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cba6ad640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb4143c10 con 0x7f2cb40b2ad0 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cba6ad640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb40705b0 con 0x7f2cb41431f0 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cba6ad640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb40b4fd0 con 0x7f2cb406ddb0 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cb9eac640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2cb406ddb0 0x7f2cb4071650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cb9eac640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2cb406ddb0 0x7f2cb4071650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58344/0 (socket says 172.21.15.106:58344) 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cb9eac640 1 -- 172.21.15.106:0/1897724246 learned_addr learned my addr 172.21.15.106:0/1897724246 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:10.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.007+0000 7f2cb9eac640 1 -- 172.21.15.106:0/1897724246 >> v1:172.21.15.156:6789/0 conn(0x7f2cb41431f0 legacy=0x7f2cb415e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:10.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.008+0000 7f2cb9eac640 1 -- 172.21.15.106:0/1897724246 >> v1:172.21.15.106:6789/0 conn(0x7f2cb40b2ad0 legacy=0x7f2cb406d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.008+0000 7f2cb9eac640 1 -- 172.21.15.106:0/1897724246 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb4070890 con 0x7f2cb406ddb0 2023-11-24T08:10:10.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.009+0000 7f2cb9eac640 1 --2- 172.21.15.106:0/1897724246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb4071650 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40b5110 tx=0x7f2cac02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b59ba863a62c5ebc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:10.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.009+0000 7f2cabfff640 1 -- 172.21.15.106:0/1897724246 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac02cb00 con 0x7f2cb406ddb0 2023-11-24T08:10:10.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.009+0000 7f2cabfff640 1 -- 172.21.15.106:0/1897724246 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac02cc60 con 0x7f2cb406ddb0 2023-11-24T08:10:10.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.009+0000 7f2cabfff640 1 -- 172.21.15.106:0/1897724246 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0343f0 con 0x7f2cb406ddb0 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1897724246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 msgr2=0x7f2cb4071650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 --2- 172.21.15.106:0/1897724246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb4071650 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2cb40b5110 tx=0x7f2cac02bf60 comp rx=0 tx=0).stop 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1897724246 shutdown_connections 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 --2- 172.21.15.106:0/1897724246 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb4071650 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1897724246 >> 172.21.15.106:0/1897724246 conn(0x7f2cb40cb8a0 msgr2=0x7f2cb40cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1897724246 shutdown_connections 2023-11-24T08:10:10.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.010+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1897724246 wait complete. 2023-11-24T08:10:10.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.012+0000 7f2cba6ad640 1 Processor -- start 2023-11-24T08:10:10.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.012+0000 7f2cba6ad640 1 -- start start 2023-11-24T08:10:10.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cba6ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb41f6890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cba6ad640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb40b2ad0 0x7f2cb413f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cba6ad640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cb41431f0 0x7f2cb413f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cba6ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb415c330 con 0x7f2cb406ddb0 2023-11-24T08:10:10.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cba6ad640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40c86a0 con 0x7f2cb40b2ad0 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cba6ad640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb41f6dd0 con 0x7f2cb41431f0 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cb96ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb41f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cb96ab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb41f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58358/0 (socket says 172.21.15.106:58358) 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.013+0000 7f2cb96ab640 1 -- 172.21.15.106:0/1771557732 learned_addr learned my addr 172.21.15.106:0/1771557732 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cb96ab640 1 -- 172.21.15.106:0/1771557732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cb41431f0 msgr2=0x7f2cb413f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cb96ab640 1 --2- 172.21.15.106:0/1771557732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cb41431f0 0x7f2cb413f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cb96ab640 1 -- 172.21.15.106:0/1771557732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb40b2ad0 msgr2=0x7f2cb413f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cb96ab640 1 --2- 172.21.15.106:0/1771557732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cb40b2ad0 0x7f2cb413f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cb96ab640 1 -- 172.21.15.106:0/1771557732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac009b30 con 0x7f2cb406ddb0 2023-11-24T08:10:10.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cb96ab640 1 --2- 172.21.15.106:0/1771557732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cb406ddb0 0x7f2cb41f6890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000c400 tx=0x7f2cb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:10.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00052d0 con 0x7f2cb406ddb0 2023-11-24T08:10:10.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb0005430 con 0x7f2cb406ddb0 2023-11-24T08:10:10.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1771557732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cb41f7000 con 0x7f2cb406ddb0 2023-11-24T08:10:10.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.014+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0003830 con 0x7f2cb406ddb0 2023-11-24T08:10:10.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.015+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1771557732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb41f75c0 con 0x7f2cb406ddb0 2023-11-24T08:10:10.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.016+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1771557732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb404e840 con 0x7f2cb406ddb0 2023-11-24T08:10:10.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.017+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f2cb0003990 con 0x7f2cb406ddb0 2023-11-24T08:10:10.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.018+0000 7f2ca9ffb640 1 --2- 172.21.15.106:0/1771557732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c9806ae70 0x7f2c9806d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.019+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cb0088ba0 con 0x7f2cb406ddb0 2023-11-24T08:10:10.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.019+0000 7f2cb8eaa640 1 --2- 172.21.15.106:0/1771557732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c9806ae70 0x7f2c9806d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.019+0000 7f2cb8eaa640 1 --2- 172.21.15.106:0/1771557732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c9806ae70 0x7f2c9806d330 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2ca4000fd0 tx=0x7f2ca4007950 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:10.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.024+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cb0053c40 con 0x7f2cb406ddb0 2023-11-24T08:10:10.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.241+0000 7f2cba6ad640 1 -- 172.21.15.106:0/1771557732 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "cephfs", "group_name": "subvol_grp2797", "snap_name": "subvol_snap2809", "target": ["mon-mgr", ""]}) v1 -- 0x7f2cb415e450 con 0x7f2c9806ae70 2023-11-24T08:10:10.246 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:10.244+0000 7f67aa1bf640 -1 mgr.server reply reply (38) Function not implemented subvolume group snapshots are not supported 2023-11-24T08:10:10.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.244+0000 7f2ca9ffb640 1 -- 172.21.15.106:0/1771557732 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: -38 subvolume group snapshots are not supported) v1 ==== 51+0+0 (secure 0 0 0) 0x7f2cb415e450 con 0x7f2c9806ae70 2023-11-24T08:10:10.246 INFO:teuthology.orchestra.run.smithi106.stderr:Error ENOSYS: subvolume group snapshots are not supported 2023-11-24T08:10:10.258 DEBUG:teuthology.orchestra.run:got remote process result: 38 2023-11-24T08:10:10.258 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp2797 2023-11-24T08:10:10.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.477+0000 7fad91e0e640 1 Processor -- start 2023-11-24T08:10:10.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.477+0000 7fad91e0e640 1 -- start start 2023-11-24T08:10:10.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.479+0000 7fad91e0e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 0x7fad8c166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.479+0000 7fad91e0e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c15eea0 con 0x7fad8c15b440 2023-11-24T08:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.479+0000 7fad91e0e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad8c06d600 con 0x7fad8c0cc0f0 2023-11-24T08:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.479+0000 7fad91e0e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0c5e00 con 0x7fad8c0c59b0 2023-11-24T08:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.479+0000 7fad8b7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fad8c15b440 0x7fad8c0cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36632/0 (socket says 172.21.15.106:36632) 2023-11-24T08:10:10.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.479+0000 7fad8b7fe640 1 -- 172.21.15.106:0/1875882649 learned_addr learned my addr 172.21.15.106:0/1875882649 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:10.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 670831564 0 0) 0x7fad8c15eea0 con 0x7fad8c15b440 2023-11-24T08:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad78003520 con 0x7fad8c15b440 2023-11-24T08:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8bfff640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 0x7fad8c166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3972735199 0 0) 0x7fad78003520 con 0x7fad8c15b440 2023-11-24T08:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 msgr2=0x7fad8c166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 0x7fad8c166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 >> v1:172.21.15.156:6789/0 conn(0x7fad8c0cc0f0 legacy=0x7fad8c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c0abf30 con 0x7fad8c15b440 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fad7c001670 con 0x7fad8c15b440 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 >> v1:172.21.15.106:6789/0 conn(0x7fad8c15b440 legacy=0x7fad8c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad780039a0 0x7fad78003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad78004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 0x7fad7800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad780054b0 con 0x7fad780047c0 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78005620 con 0x7fad780039a0 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.480+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad7800bec0 con 0x7fad8c0c59b0 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad78004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad8bfff640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 0x7fad7800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad82ffd640 1 -- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 msgr2=0x7fad7800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c0c59b0 0x7fad7800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad82ffd640 1 -- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad780039a0 msgr2=0x7fad78003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad780039a0 0x7fad78003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.481+0000 7fad82ffd640 1 -- 172.21.15.106:0/1875882649 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78003520 con 0x7fad780047c0 2023-11-24T08:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.482+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad78004c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fad74001750 tx=0x7fad780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d045f24c00bfa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.482+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740052d0 con 0x7fad780047c0 2023-11-24T08:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.482+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7402e050 con 0x7fad780047c0 2023-11-24T08:10:10.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.482+0000 7fad8affd640 1 -- 172.21.15.106:0/1875882649 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad740035f0 con 0x7fad780047c0 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad91e0e640 1 -- 172.21.15.106:0/1875882649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 msgr2=0x7fad78004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1875882649 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad78004c40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fad74001750 tx=0x7fad780057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad8bfff640 1 -- 172.21.15.106:0/1875882649 reap_dead start 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad91e0e640 1 -- 172.21.15.106:0/1875882649 shutdown_connections 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad91e0e640 1 -- 172.21.15.106:0/1875882649 >> 172.21.15.106:0/1875882649 conn(0x7fad8c0b0c50 msgr2=0x7fad8c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad91e0e640 1 -- 172.21.15.106:0/1875882649 shutdown_connections 2023-11-24T08:10:10.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.483+0000 7fad91e0e640 1 -- 172.21.15.106:0/1875882649 wait complete. 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.485+0000 7fad91e0e640 1 Processor -- start 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.485+0000 7fad91e0e640 1 -- start start 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad91e0e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad8c153c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad91e0e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c1f4260 0x7fad8c0cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad91e0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad8c0cf470 0x7fad8c0ac410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad91e0e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0cf8f0 con 0x7fad780047c0 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad91e0e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c1541b0 con 0x7fad8c0cf470 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad91e0e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c0ad190 con 0x7fad8c1f4260 2023-11-24T08:10:10.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad8b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad8c153c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad8b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad8c153c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58382/0 (socket says 172.21.15.106:58382) 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.486+0000 7fad8b7fe640 1 -- 172.21.15.106:0/1759310764 learned_addr learned my addr 172.21.15.106:0/1759310764 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c1f4260 0x7fad8c0cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8bfff640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad8c0cf470 0x7fad8c0ac410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8b7fe640 1 -- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c1f4260 msgr2=0x7fad8c0cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8b7fe640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c1f4260 0x7fad8c0cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8b7fe640 1 -- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad8c0cf470 msgr2=0x7fad8c0ac410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8b7fe640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad8c0cf470 0x7fad8c0ac410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8b7fe640 1 -- 172.21.15.106:0/1759310764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c1f34a0 con 0x7fad780047c0 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8bfff640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad8c0cf470 0x7fad8c0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:10.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad8b7fe640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad8c153c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00c700 tx=0x7fad7c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:10.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c0074e0 con 0x7fad780047c0 2023-11-24T08:10:10.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7c007640 con 0x7fad780047c0 2023-11-24T08:10:10.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c004a00 con 0x7fad780047c0 2023-11-24T08:10:10.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c1f3780 con 0x7fad780047c0 2023-11-24T08:10:10.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.487+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c1f3d40 con 0x7fad780047c0 2023-11-24T08:10:10.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.488+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c04e8b0 con 0x7fad780047c0 2023-11-24T08:10:10.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.490+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7fad7c004b60 con 0x7fad780047c0 2023-11-24T08:10:10.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.490+0000 7fad88ff9640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fad6c06ae40 0x7fad6c06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:10.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.491+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad7c088b10 con 0x7fad780047c0 2023-11-24T08:10:10.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.491+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fad6c06ae40 0x7fad6c06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:10.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.492+0000 7fad82ffd640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fad6c06ae40 0x7fad6c06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad74033000 tx=0x7fad74009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:10.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.495+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fad7c053bc0 con 0x7fad780047c0 2023-11-24T08:10:10.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.711+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp2797", "target": ["mon-mgr", ""]}) v1 -- 0x7fad8c15ee90 con 0x7fad6c06ae40 2023-11-24T08:10:10.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.718+0000 7fad88ff9640 1 -- 172.21.15.106:0/1759310764 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fad8c15ee90 con 0x7fad6c06ae40 2023-11-24T08:10:10.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.720+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fad6c06ae40 msgr2=0x7fad6c06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.720+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fad6c06ae40 0x7fad6c06d300 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fad74033000 tx=0x7fad74009030 comp rx=0 tx=0).stop 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.720+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 msgr2=0x7fad8c153c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.720+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad8c153c70 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00c700 tx=0x7fad7c00cbc0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 shutdown_connections 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad8c0cf470 0x7fad8c0ac410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fad6c06ae40 0x7fad6c06d300 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fad8c1f4260 0x7fad8c0cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 --2- 172.21.15.106:0/1759310764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fad780047c0 0x7fad8c153c70 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 >> 172.21.15.106:0/1759310764 conn(0x7fad8c0b0c50 msgr2=0x7fad8c0cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 shutdown_connections 2023-11-24T08:10:10.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:10.721+0000 7fad91e0e640 1 -- 172.21.15.106:0/1759310764 wait complete. 2023-11-24T08:10:10.741 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:10:10.742 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:10:10.851 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:10:10.976 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:10:10.976 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:10.976 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:11.037 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:11.037 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:11.092 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:11.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.306+0000 7f9cceb2e640 1 Processor -- start 2023-11-24T08:10:11.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.306+0000 7f9cceb2e640 1 -- start start 2023-11-24T08:10:11.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9cceb2e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cc80cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9cceb2e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80d04b0 con 0x7f9cc815d4e0 2023-11-24T08:10:11.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9cceb2e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cc80ce990 con 0x7f9cc80d0070 2023-11-24T08:10:11.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9cceb2e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80d4110 con 0x7f9cc80d1e20 2023-11-24T08:10:11.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9ccdb2c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9cc815d4e0 0x7f9cc80d38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36640/0 (socket says 172.21.15.106:36640) 2023-11-24T08:10:11.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9ccdb2c640 1 -- 172.21.15.106:0/1242781567 learned_addr learned my addr 172.21.15.106:0/1242781567 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:11.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.308+0000 7f9ccd32b640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cc80cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2561173072 0 0) 0x7f9cc80d04b0 con 0x7f9cc815d4e0 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cb4003520 con 0x7f9cc815d4e0 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4194864751 0 0) 0x7f9cb4003520 con 0x7f9cc815d4e0 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 msgr2=0x7f9cc80cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cc80cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 >> v1:172.21.15.156:6789/0 conn(0x7f9cc80d0070 legacy=0x7f9cc80b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80cec70 con 0x7f9cc815d4e0 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f9cc4001670 con 0x7f9cc815d4e0 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 >> v1:172.21.15.106:6789/0 conn(0x7f9cc815d4e0 legacy=0x7f9cc80d38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb40039a0 0x7f9cb4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80d4110 con 0x7f9cb4004d00 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb400ba30 con 0x7f9cb40039a0 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb400bbf0 con 0x7f9cc80d1e20 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.309+0000 7f9cce32d640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9ccdb2c640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb40039a0 0x7f9cb4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9ccd32b640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9cce32d640 1 -- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 msgr2=0x7f9cb40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9cce32d640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cb40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9cce32d640 1 -- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb40039a0 msgr2=0x7f9cb4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9cce32d640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb40039a0 0x7f9cb4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9cce32d640 1 -- 172.21.15.106:0/1242781567 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb4003520 con 0x7f9cb4004d00 2023-11-24T08:10:11.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.310+0000 7f9ccd32b640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80d1e20 0x7f9cb40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.311+0000 7f9cce32d640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0001e20 tx=0x7f9cb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402a13b0943e4aef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.311+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0003b70 con 0x7f9cb4004d00 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.311+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc0003cd0 con 0x7f9cb4004d00 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.311+0000 7f9cccb2a640 1 -- 172.21.15.106:0/1242781567 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0005310 con 0x7f9cb4004d00 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cceb2e640 1 -- 172.21.15.106:0/1242781567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cb4004d00 msgr2=0x7f9cb4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/1242781567 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cb4004d00 0x7f9cb4005180 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0001e20 tx=0x7f9cb40057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cce32d640 1 -- 172.21.15.106:0/1242781567 reap_dead start 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cceb2e640 1 -- 172.21.15.106:0/1242781567 shutdown_connections 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cceb2e640 1 -- 172.21.15.106:0/1242781567 >> 172.21.15.106:0/1242781567 conn(0x7f9cc809c510 msgr2=0x7f9cc80d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:11.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cceb2e640 1 -- 172.21.15.106:0/1242781567 shutdown_connections 2023-11-24T08:10:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.312+0000 7f9cceb2e640 1 -- 172.21.15.106:0/1242781567 wait complete. 2023-11-24T08:10:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.314+0000 7f9cceb2e640 1 Processor -- start 2023-11-24T08:10:11.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.314+0000 7f9cceb2e640 1 -- start start 2023-11-24T08:10:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9cceb2e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb4004d00 0x7f9cc815ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9cceb2e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 0x7f9cc815fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9cceb2e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80b8890 0x7f9cc815ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9cceb2e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80b7150 con 0x7f9cc80c9700 2023-11-24T08:10:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9cceb2e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80b7430 con 0x7f9cb4004d00 2023-11-24T08:10:11.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9cceb2e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80b7710 con 0x7f9cc80b8890 2023-11-24T08:10:11.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9ccd32b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 0x7f9cc815fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9ccd32b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 0x7f9cc815fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58398/0 (socket says 172.21.15.106:58398) 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.315+0000 7f9ccd32b640 1 -- 172.21.15.106:0/3951096387 learned_addr learned my addr 172.21.15.106:0/3951096387 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccdb2c640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb4004d00 0x7f9cc815ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9cce32d640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80b8890 0x7f9cc815ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccd32b640 1 -- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80b8890 msgr2=0x7f9cc815ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccd32b640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80b8890 0x7f9cc815ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccd32b640 1 -- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb4004d00 msgr2=0x7f9cc815ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccd32b640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb4004d00 0x7f9cc815ba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccd32b640 1 -- 172.21.15.106:0/3951096387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80b79f0 con 0x7f9cc80c9700 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9ccd32b640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 0x7f9cc815fa50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80108b0 tx=0x7f9cb8010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:11.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb800eb60 con 0x7f9cc80c9700 2023-11-24T08:10:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.316+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cb800ecc0 con 0x7f9cc80c9700 2023-11-24T08:10:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.317+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc80b7cd0 con 0x7f9cc80c9700 2023-11-24T08:10:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.317+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb801c400 con 0x7f9cc80c9700 2023-11-24T08:10:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.317+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc80b6de0 con 0x7f9cc80c9700 2023-11-24T08:10:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.318+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc80b2aa0 con 0x7f9cc80c9700 2023-11-24T08:10:11.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.319+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 5) v1 ==== 89231+0+0 (secure 0 0 0) 0x7f9cb801c560 con 0x7f9cc80c9700 2023-11-24T08:10:11.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.320+0000 7f9cbe7fc640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9cac06ae40 0x7f9cac06d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.320+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9cb8089ce0 con 0x7f9cc80c9700 2023-11-24T08:10:11.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.320+0000 7f9ccdb2c640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9cac06ae40 0x7f9cac06d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.321+0000 7f9ccdb2c640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9cac06ae40 0x7f9cac06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4001c80 tx=0x7f9cc4001d50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:11.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.324+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9cb8054da0 con 0x7f9cc80c9700 2023-11-24T08:10:11.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.425+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9cb8055170 con 0x7f9cc80c9700 2023-11-24T08:10:11.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.520+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cc804e840 con 0x7f9cc80c9700 2023-11-24T08:10:11.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.523+0000 7f9cbe7fc640 1 -- 172.21.15.106:0/3951096387 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f9cb8085de0 con 0x7f9cc80c9700 2023-11-24T08:10:11.525 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:11.525 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:58.320664+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:11.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9cac06ae40 msgr2=0x7f9cac06d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9cac06ae40 0x7f9cac06d300 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9cc4001c80 tx=0x7f9cc4001d50 comp rx=0 tx=0).stop 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 msgr2=0x7f9cc815fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 0x7f9cc815fa50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9cb80108b0 tx=0x7f9cb8010d70 comp rx=0 tx=0).stop 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 shutdown_connections 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9cac06ae40 0x7f9cac06d300 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9cc80b8890 0x7f9cc815ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9cc80c9700 0x7f9cc815fa50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 --2- 172.21.15.106:0/3951096387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9cb4004d00 0x7f9cc815ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 >> 172.21.15.106:0/3951096387 conn(0x7f9cc809c510 msgr2=0x7f9cc80b2390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:11.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 shutdown_connections 2023-11-24T08:10:11.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.526+0000 7f9cceb2e640 1 -- 172.21.15.106:0/3951096387 wait complete. 2023-11-24T08:10:11.547 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:09:54.590990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:09:55.792078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:10:11.548 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:10:11.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.763+0000 7fc9ac22b640 1 Processor -- start 2023-11-24T08:10:11.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.763+0000 7fc9ac22b640 1 -- start start 2023-11-24T08:10:11.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.764+0000 7fc9ac22b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a415b440 0x7fc9a40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9ac22b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9a40cbf20 0x7fc9a40c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9ac22b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc9a40c57e0 0x7fc9a40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9ac22b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a415b820 con 0x7fc9a40cbf20 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9ac22b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a40cc3a0 con 0x7fc9a40c57e0 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9ac22b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a415eea0 con 0x7fc9a415b440 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9a40cbf20 0x7fc9a40c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc9a40cbf20 0x7fc9a40c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58408/0 (socket says 172.21.15.106:58408) 2023-11-24T08:10:11.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 -- 172.21.15.106:0/3477558373 learned_addr learned my addr 172.21.15.106:0/3477558373 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 -- 172.21.15.106:0/3477558373 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a415b440 msgr2=0x7fc9a40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 --2- 172.21.15.106:0/3477558373 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a415b440 0x7fc9a40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9aa7c8640 1 --2- 172.21.15.106:0/3477558373 >> v2:172.21.15.156:3300/0 conn(0x7fc9a40c57e0 0x7fc9a40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 -- 172.21.15.106:0/3477558373 >> v2:172.21.15.156:3300/0 conn(0x7fc9a40c57e0 msgr2=0x7fc9a40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.765+0000 7fc9a97c6640 1 --2- 172.21.15.106:0/3477558373 >> v2:172.21.15.156:3300/0 conn(0x7fc9a40c57e0 0x7fc9a40c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.766+0000 7fc9a97c6640 1 -- 172.21.15.106:0/3477558373 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40abf30 con 0x7fc9a40cbf20 2023-11-24T08:10:11.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.766+0000 7fc9aa7c8640 1 --2- 172.21.15.106:0/3477558373 >> v2:172.21.15.156:3300/0 conn(0x7fc9a40c57e0 0x7fc9a40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:11.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.767+0000 7fc9a97c6640 1 --2- 172.21.15.106:0/3477558373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a40c52a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40cb910 tx=0x7fc99402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c98bc1dac6a7c21 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:11.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.767+0000 7fc9a8fc5640 1 -- 172.21.15.106:0/3477558373 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9940317e0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.767+0000 7fc9a8fc5640 1 -- 172.21.15.106:0/3477558373 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994031940 con 0x7fc9a40cbf20 2023-11-24T08:10:11.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.767+0000 7fc9a8fc5640 1 -- 172.21.15.106:0/3477558373 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994031cd0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 -- 172.21.15.106:0/3477558373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 msgr2=0x7fc9a40c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/3477558373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a40c52a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc9a40cb910 tx=0x7fc99402b880 comp rx=0 tx=0).stop 2023-11-24T08:10:11.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 -- 172.21.15.106:0/3477558373 shutdown_connections 2023-11-24T08:10:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/3477558373 >> v2:172.21.15.156:3300/0 conn(0x7fc9a40c57e0 0x7fc9a40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/3477558373 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a40c52a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/3477558373 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a415b440 0x7fc9a40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 -- 172.21.15.106:0/3477558373 >> 172.21.15.106:0/3477558373 conn(0x7fc9a40b0c50 msgr2=0x7fc9a40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 -- 172.21.15.106:0/3477558373 shutdown_connections 2023-11-24T08:10:11.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.768+0000 7fc9ac22b640 1 -- 172.21.15.106:0/3477558373 wait complete. 2023-11-24T08:10:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.770+0000 7fc9ac22b640 1 Processor -- start 2023-11-24T08:10:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.770+0000 7fc9ac22b640 1 -- start start 2023-11-24T08:10:11.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9ac22b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a40c57e0 0x7fc9a4154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9ac22b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a4156890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9ac22b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9a415b440 0x7fc9a40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9ac22b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a409a4a0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9ac22b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a41550b0 con 0x7fc9a415b440 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9ac22b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4155a30 con 0x7fc9a40c57e0 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9a97c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a4156890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9a97c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a4156890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58424/0 (socket says 172.21.15.106:58424) 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.771+0000 7fc9a97c6640 1 -- 172.21.15.106:0/1322602052 learned_addr learned my addr 172.21.15.106:0/1322602052 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9a97c6640 1 -- 172.21.15.106:0/1322602052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a40c57e0 msgr2=0x7fc9a4154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9a97c6640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a40c57e0 0x7fc9a4154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9a97c6640 1 -- 172.21.15.106:0/1322602052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9a415b440 msgr2=0x7fc9a40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9a97c6640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9a415b440 0x7fc9a40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9a97c6640 1 -- 172.21.15.106:0/1322602052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9940094d0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9a97c6640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a4156890 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc99402b8c0 tx=0x7fc9940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994032940 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc994032aa0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a4155d10 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994032e50 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.772+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a41562d0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.774+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a404e8b0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.775+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc99404e450 con 0x7fc9a40cbf20 2023-11-24T08:10:11.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.777+0000 7fc99a7fc640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc98806aed0 0x7fc98806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:11.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.777+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9940ab940 con 0x7fc9a40cbf20 2023-11-24T08:10:11.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.777+0000 7fc9a9fc7640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc98806aed0 0x7fc98806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:11.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.778+0000 7fc9a9fc7640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc98806aed0 0x7fc98806d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009670 tx=0x7fc9a0006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:11.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.781+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc9940769a0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.971+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc9a406dc80 con 0x7fc9a40cbf20 2023-11-24T08:10:11.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.972+0000 7fc99a7fc640 1 -- 172.21.15.106:0/1322602052 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc994079fc0 con 0x7fc9a40cbf20 2023-11-24T08:10:11.974 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:10:11.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc98806aed0 msgr2=0x7fc98806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc98806aed0 0x7fc98806d390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0009670 tx=0x7fc9a0006bf0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 msgr2=0x7fc9a4156890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:11.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a4156890 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc99402b8c0 tx=0x7fc9940046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 shutdown_connections 2023-11-24T08:10:11.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9a415b440 0x7fc9a40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc9a40cbf20 0x7fc9a4156890 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc98806aed0 0x7fc98806d390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 --2- 172.21.15.106:0/1322602052 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc9a40c57e0 0x7fc9a4154b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:11.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 >> 172.21.15.106:0/1322602052 conn(0x7fc9a40b0c50 msgr2=0x7fc9a4158ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:11.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 shutdown_connections 2023-11-24T08:10:11.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:11.975+0000 7fc9ac22b640 1 -- 172.21.15.106:0/1322602052 wait complete. 2023-11-24T08:10:11.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:10:12.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.202+0000 7f5f3ace6640 1 Processor -- start 2023-11-24T08:10:12.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.202+0000 7f5f3ace6640 1 -- start start 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f3ace6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f340b0b40 0x7f5f340b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f3ace6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f3406d430 con 0x7f5f3415ded0 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f3ace6640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f340b14e0 con 0x7f5f3415a1c0 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f3ace6640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f3415ea70 con 0x7f5f340b0b40 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f394e3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f340b0b40 0x7f5f340b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f394e3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5f340b0b40 0x7f5f340b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58432/0 (socket says 172.21.15.106:58432) 2023-11-24T08:10:12.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.204+0000 7f5f394e3640 1 -- 172.21.15.106:0/1580151082 learned_addr learned my addr 172.21.15.106:0/1580151082 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:12.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.205+0000 7f5f394e3640 1 -- 172.21.15.106:0/1580151082 >> v1:172.21.15.156:6789/0 conn(0x7f5f3415a1c0 legacy=0x7f5f3406d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:12.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.205+0000 7f5f394e3640 1 -- 172.21.15.106:0/1580151082 >> v1:172.21.15.106:6789/0 conn(0x7f5f3415ded0 legacy=0x7f5f3415e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.205+0000 7f5f394e3640 1 -- 172.21.15.106:0/1580151082 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3415ec50 con 0x7f5f340b0b40 2023-11-24T08:10:12.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.206+0000 7f5f394e3640 1 --2- 172.21.15.106:0/1580151082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f340b0fa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f2402dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d860a4b40065b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:12.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.206+0000 7f5f38ce2640 1 -- 172.21.15.106:0/1580151082 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2402e960 con 0x7f5f340b0b40 2023-11-24T08:10:12.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.206+0000 7f5f38ce2640 1 -- 172.21.15.106:0/1580151082 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f2402eac0 con 0x7f5f340b0b40 2023-11-24T08:10:12.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.206+0000 7f5f38ce2640 1 -- 172.21.15.106:0/1580151082 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f2402ee50 con 0x7f5f340b0b40 2023-11-24T08:10:12.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 -- 172.21.15.106:0/1580151082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 msgr2=0x7f5f340b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/1580151082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f340b0fa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009870 tx=0x7f5f2402dbb0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 -- 172.21.15.106:0/1580151082 shutdown_connections 2023-11-24T08:10:12.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/1580151082 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f340b0fa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 -- 172.21.15.106:0/1580151082 >> 172.21.15.106:0/1580151082 conn(0x7f5f340cbb40 msgr2=0x7f5f340cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:12.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 -- 172.21.15.106:0/1580151082 shutdown_connections 2023-11-24T08:10:12.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.207+0000 7f5f3ace6640 1 -- 172.21.15.106:0/1580151082 wait complete. 2023-11-24T08:10:12.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.209+0000 7f5f3ace6640 1 Processor -- start 2023-11-24T08:10:12.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.209+0000 7f5f3ace6640 1 -- start start 2023-11-24T08:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f3ace6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f34155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f3ace6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3415a1c0 0x7f5f341f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f3ace6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3415ded0 0x7f5f340c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f3ace6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3415ce90 con 0x7f5f340b0b40 2023-11-24T08:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f3ace6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3415c330 con 0x7f5f3415a1c0 2023-11-24T08:10:12.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f3ace6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f3415c5f0 con 0x7f5f3415ded0 2023-11-24T08:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f39ce4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f34155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f39ce4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f34155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58444/0 (socket says 172.21.15.106:58444) 2023-11-24T08:10:12.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.210+0000 7f5f39ce4640 1 -- 172.21.15.106:0/2401365257 learned_addr learned my addr 172.21.15.106:0/2401365257 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:12.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f39ce4640 1 -- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3415ded0 msgr2=0x7f5f340c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f394e3640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3415a1c0 0x7f5f341f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f39ce4640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3415ded0 0x7f5f340c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f39ce4640 1 -- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3415a1c0 msgr2=0x7f5f341f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f39ce4640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3415a1c0 0x7f5f341f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f39ce4640 1 -- 172.21.15.106:0/2401365257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f240094d0 con 0x7f5f340b0b40 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f394e3640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3415a1c0 0x7f5f341f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f39ce4640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f34155870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c430 tx=0x7f5f3000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.211+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011790 con 0x7f5f340b0b40 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.212+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f300118f0 con 0x7f5f340b0b40 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.212+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f30011ce0 con 0x7f5f340b0b40 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.212+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f3415c8d0 con 0x7f5f340b0b40 2023-11-24T08:10:12.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.212+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f341f6710 con 0x7f5f340b0b40 2023-11-24T08:10:12.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.213+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f341f6570 con 0x7f5f340b0b40 2023-11-24T08:10:12.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.214+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5f3001c460 con 0x7f5f340b0b40 2023-11-24T08:10:12.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.215+0000 7f5f2a7fc640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f1806af00 0x7f5f1806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.215+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f30089640 con 0x7f5f340b0b40 2023-11-24T08:10:12.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.215+0000 7f5f394e3640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f1806af00 0x7f5f1806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.216+0000 7f5f394e3640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f1806af00 0x7f5f1806d3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5f24038000 tx=0x7f5f24032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:12.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.219+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5f341f6570 con 0x7f5f340b0b40 2023-11-24T08:10:12.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.410+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5f341f6570 con 0x7f5f340b0b40 2023-11-24T08:10:12.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.411+0000 7f5f2a7fc640 1 -- 172.21.15.106:0/2401365257 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5f341f6570 con 0x7f5f340b0b40 2023-11-24T08:10:12.413 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:10:12.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f1806af00 msgr2=0x7f5f1806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f1806af00 0x7f5f1806d3c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5f24038000 tx=0x7f5f24032000 comp rx=0 tx=0).stop 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 msgr2=0x7f5f34155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f34155870 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5f3000c430 tx=0x7f5f3000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 shutdown_connections 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5f1806af00 0x7f5f1806d3c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5f3415ded0 0x7f5f340c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f3415a1c0 0x7f5f341f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 --2- 172.21.15.106:0/2401365257 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5f340b0b40 0x7f5f34155870 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 >> 172.21.15.106:0/2401365257 conn(0x7f5f340cbb40 msgr2=0x7f5f34070df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 shutdown_connections 2023-11-24T08:10:12.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.414+0000 7f5f3ace6640 1 -- 172.21.15.106:0/2401365257 wait complete. 2023-11-24T08:10:12.434 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.646+0000 7f48ee65c640 1 Processor -- start 2023-11-24T08:10:12.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.646+0000 7f48ee65c640 1 -- start start 2023-11-24T08:10:12.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.648+0000 7f48ee65c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48e80cc0f0 0x7f48e80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.648+0000 7f48ee65c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e815eea0 con 0x7f48e80c57e0 2023-11-24T08:10:12.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.648+0000 7f48ee65c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48e80abf30 con 0x7f48e815b440 2023-11-24T08:10:12.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.648+0000 7f48ee65c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e815b820 con 0x7f48e80cc0f0 2023-11-24T08:10:12.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.648+0000 7f48ecbf9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f48e80c57e0 0x7f48e806c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36658/0 (socket says 172.21.15.106:36658) 2023-11-24T08:10:12.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48ecbf9640 1 -- 172.21.15.106:0/1606097161 learned_addr learned my addr 172.21.15.106:0/1606097161 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:12.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48dffff640 1 --2- 172.21.15.106:0/1606097161 >> v2:172.21.15.156:3300/0 conn(0x7f48e80cc0f0 0x7f48e80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 248074070 0 0) 0x7f48e815eea0 con 0x7f48e80c57e0 2023-11-24T08:10:12.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48d4003520 con 0x7f48e80c57e0 2023-11-24T08:10:12.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 58174134 0 0) 0x7f48d4003520 con 0x7f48e80c57e0 2023-11-24T08:10:12.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 >> v2:172.21.15.156:3300/0 conn(0x7f48e80cc0f0 msgr2=0x7f48e80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:12.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 --2- 172.21.15.106:0/1606097161 >> v2:172.21.15.156:3300/0 conn(0x7f48e80cc0f0 0x7f48e80c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 >> v1:172.21.15.156:6789/0 conn(0x7f48e815b440 legacy=0x7f48e80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e815f0f0 con 0x7f48e80c57e0 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f48e00019f0 con 0x7f48e80c57e0 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.649+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 >> v1:172.21.15.106:6789/0 conn(0x7f48e80c57e0 legacy=0x7f48e806c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e77fe640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48d40039c0 0x7f48d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e77fe640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40047c0 0x7f48d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e77fe640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48d40051d0 0x7f48d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e815b820 con 0x7f48d40039c0 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48d400c120 con 0x7f48d40047c0 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48d400c2e0 con 0x7f48d40051d0 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e7fff640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48d40039c0 0x7f48d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48dffff640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40047c0 0x7f48d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e7fff640 1 -- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48d40051d0 msgr2=0x7f48d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e7fff640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48d40051d0 0x7f48d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e7fff640 1 -- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40047c0 msgr2=0x7f48d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:12.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e7fff640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40047c0 0x7f48d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48e7fff640 1 -- 172.21.15.106:0/1606097161 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48d4003520 con 0x7f48d40039c0 2023-11-24T08:10:12.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.650+0000 7f48ecbf9640 1 -- 172.21.15.106:0/1606097161 reap_dead start 2023-11-24T08:10:12.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.651+0000 7f48e7fff640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48d40039c0 0x7f48d4003de0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f48d8001720 tx=0x7f48d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab1f0646d730e23e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d80052d0 con 0x7f48d40039c0 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d802e050 con 0x7f48d40039c0 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48e77fe640 1 -- 172.21.15.106:0/1606097161 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d8003830 con 0x7f48d40039c0 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 -- 172.21.15.106:0/1606097161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48d40039c0 msgr2=0x7f48d4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48d40039c0 0x7f48d4003de0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f48d8001720 tx=0x7f48d40057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 -- 172.21.15.106:0/1606097161 shutdown_connections 2023-11-24T08:10:12.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 --2- 172.21.15.106:0/1606097161 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48d40039c0 0x7f48d4003de0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 -- 172.21.15.106:0/1606097161 >> 172.21.15.106:0/1606097161 conn(0x7f48e80b0c50 msgr2=0x7f48e80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 -- 172.21.15.106:0/1606097161 shutdown_connections 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.652+0000 7f48ee65c640 1 -- 172.21.15.106:0/1606097161 wait complete. 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.654+0000 7f48ee65c640 1 Processor -- start 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.654+0000 7f48ee65c640 1 -- start start 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.655+0000 7f48ee65c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40039c0 0x7f48e80c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.655+0000 7f48ee65c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 0x7f48e8071550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.655+0000 7f48ee65c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80c7af0 0x7f48e8071a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.655+0000 7f48ee65c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e81598e0 con 0x7f48e80c8840 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.655+0000 7f48ee65c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e8156ac0 con 0x7f48d40039c0 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.655+0000 7f48ee65c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e8158c60 con 0x7f48e80c7af0 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 0x7f48e8071550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 0x7f48e8071550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58456/0 (socket says 172.21.15.106:58456) 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 -- 172.21.15.106:0/553792951 learned_addr learned my addr 172.21.15.106:0/553792951 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48e7fff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40039c0 0x7f48e80c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 -- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80c7af0 msgr2=0x7f48e8071a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80c7af0 0x7f48e8071a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 -- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40039c0 msgr2=0x7f48e80c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40039c0 0x7f48e80c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 -- 172.21.15.106:0/553792951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e8158f40 con 0x7f48e80c8840 2023-11-24T08:10:12.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48e7fff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40039c0 0x7f48e80c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:12.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.656+0000 7f48dffff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 0x7f48e8071550 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48d00116d0 tx=0x7f48d0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:12.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.657+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000f960 con 0x7f48e80c8840 2023-11-24T08:10:12.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.657+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d000fac0 con 0x7f48e80c8840 2023-11-24T08:10:12.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.657+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d000fe30 con 0x7f48e80c8840 2023-11-24T08:10:12.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.657+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e8159220 con 0x7f48e80c8840 2023-11-24T08:10:12.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.657+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e81587f0 con 0x7f48e80c8840 2023-11-24T08:10:12.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.658+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e8158a10 con 0x7f48e80c8840 2023-11-24T08:10:12.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.659+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f48d0015460 con 0x7f48e80c8840 2023-11-24T08:10:12.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.660+0000 7f48e57fa640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48c806af00 0x7f48c806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:12.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.660+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48d0089a90 con 0x7f48e80c8840 2023-11-24T08:10:12.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.661+0000 7f48e7fff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48c806af00 0x7f48c806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:12.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.661+0000 7f48e7fff640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48c806af00 0x7f48c806d3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48d8033000 tx=0x7f48d8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:12.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.665+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48d0054b90 con 0x7f48e80c8840 2023-11-24T08:10:12.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.858+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f48e804e8b0 con 0x7f48e80c8840 2023-11-24T08:10:12.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.859+0000 7f48e57fa640 1 -- 172.21.15.106:0/553792951 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f48d00220a0 con 0x7f48e80c8840 2023-11-24T08:10:12.861 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:10:12.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.861+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48c806af00 msgr2=0x7f48c806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.861+0000 7f48ee65c640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48c806af00 0x7f48c806d3c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f48d8033000 tx=0x7f48d8009030 comp rx=0 tx=0).stop 2023-11-24T08:10:12.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 msgr2=0x7f48e8071550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 0x7f48e8071550 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f48d00116d0 tx=0x7f48d0011b90 comp rx=0 tx=0).stop 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 shutdown_connections 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48c806af00 0x7f48c806d3c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48e80c7af0 0x7f48e8071a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48e80c8840 0x7f48e8071550 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 --2- 172.21.15.106:0/553792951 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48d40039c0 0x7f48e80c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 >> 172.21.15.106:0/553792951 conn(0x7f48e80b0c50 msgr2=0x7f48e80c3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 shutdown_connections 2023-11-24T08:10:12.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:12.862+0000 7f48ee65c640 1 -- 172.21.15.106:0/553792951 wait complete. 2023-11-24T08:10:12.882 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:10:13.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.092+0000 7f26bc5a9640 1 Processor -- start 2023-11-24T08:10:13.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.093+0000 7f26bc5a9640 1 -- start start 2023-11-24T08:10:13.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.094+0000 7f26bc5a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26b415b440 0x7f26b40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.094+0000 7f26bc5a9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f26b40c59b0 0x7f26b406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.094+0000 7f26bc5a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b40c5dc0 con 0x7f26b40cbf20 2023-11-24T08:10:13.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.094+0000 7f26bc5a9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b40cc3d0 con 0x7f26b415b440 2023-11-24T08:10:13.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.094+0000 7f26bc5a9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b415b820 con 0x7f26b40c59b0 2023-11-24T08:10:13.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26b415b440 0x7f26b40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f26b415b440 0x7f26b40cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58460/0 (socket says 172.21.15.106:58460) 2023-11-24T08:10:13.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 -- 172.21.15.106:0/1483431425 learned_addr learned my addr 172.21.15.106:0/1483431425 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:13.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 -- 172.21.15.106:0/1483431425 >> v2:172.21.15.156:3300/0 conn(0x7f26b40c59b0 msgr2=0x7f26b406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:13.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 --2- 172.21.15.106:0/1483431425 >> v2:172.21.15.156:3300/0 conn(0x7f26b40c59b0 0x7f26b406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 -- 172.21.15.106:0/1483431425 >> v1:172.21.15.106:6789/0 conn(0x7f26b40cbf20 legacy=0x7f26b40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.095+0000 7f26ba345640 1 -- 172.21.15.106:0/1483431425 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b415eea0 con 0x7f26b415b440 2023-11-24T08:10:13.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.096+0000 7f26ba345640 1 --2- 172.21.15.106:0/1483431425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b40cb9e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26b40cb910 tx=0x7f26b002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6cb5294620aa51e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.096+0000 7f26b9343640 1 -- 172.21.15.106:0/1483431425 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0031870 con 0x7f26b415b440 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.096+0000 7f26b9343640 1 -- 172.21.15.106:0/1483431425 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00319d0 con 0x7f26b415b440 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.096+0000 7f26b9343640 1 -- 172.21.15.106:0/1483431425 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0031d60 con 0x7f26b415b440 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 -- 172.21.15.106:0/1483431425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 msgr2=0x7f26b40cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/1483431425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b40cb9e0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f26b40cb910 tx=0x7f26b002b910 comp rx=0 tx=0).stop 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 -- 172.21.15.106:0/1483431425 shutdown_connections 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/1483431425 >> v2:172.21.15.156:3300/0 conn(0x7f26b40c59b0 0x7f26b406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/1483431425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b40cb9e0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 -- 172.21.15.106:0/1483431425 >> 172.21.15.106:0/1483431425 conn(0x7f26b40b0c50 msgr2=0x7f26b40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:13.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 -- 172.21.15.106:0/1483431425 shutdown_connections 2023-11-24T08:10:13.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.097+0000 7f26bc5a9640 1 -- 172.21.15.106:0/1483431425 wait complete. 2023-11-24T08:10:13.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.099+0000 7f26bc5a9640 1 Processor -- start 2023-11-24T08:10:13.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.099+0000 7f26bc5a9640 1 -- start start 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bc5a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40c59b0 0x7f26b409a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bc5a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26b40cbf20 0x7f26b4150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bc5a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b415ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bc5a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b4150750 con 0x7f26b415b440 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bc5a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b409a9c0 con 0x7f26b40c59b0 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bc5a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b409aca0 con 0x7f26b40cbf20 2023-11-24T08:10:13.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bab46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b415ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.100+0000 7f26bab46640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b415ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58472/0 (socket says 172.21.15.106:58472) 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 -- 172.21.15.106:0/564547166 learned_addr learned my addr 172.21.15.106:0/564547166 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26ba345640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40c59b0 0x7f26b409a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 -- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26b40cbf20 msgr2=0x7f26b4150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26b40cbf20 0x7f26b4150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 -- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40c59b0 msgr2=0x7f26b409a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40c59b0 0x7f26b409a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 -- 172.21.15.106:0/564547166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b00094d0 con 0x7f26b415b440 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26ba345640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40c59b0 0x7f26b409a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26bab46640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b415ce70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b970 tx=0x7f26ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00cb40 con 0x7f26b415b440 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.101+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac00ccc0 con 0x7f26b415b440 2023-11-24T08:10:13.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.102+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b409b1f0 con 0x7f26b415b440 2023-11-24T08:10:13.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.102+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac014680 con 0x7f26b415b440 2023-11-24T08:10:13.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.102+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b409b7b0 con 0x7f26b415b440 2023-11-24T08:10:13.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.103+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b409b9d0 con 0x7f26b415b440 2023-11-24T08:10:13.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.104+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f26ac01d3b0 con 0x7f26b415b440 2023-11-24T08:10:13.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.105+0000 7f26aaffd640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f269806af00 0x7f269806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.105+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26ac089690 con 0x7f26b415b440 2023-11-24T08:10:13.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.105+0000 7f26ba345640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f269806af00 0x7f269806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.106+0000 7f26ba345640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f269806af00 0x7f269806d3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26b0036000 tx=0x7f26b003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:13.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.109+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f26ac054650 con 0x7f26b415b440 2023-11-24T08:10:13.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.316+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f26b404e8b0 con 0x7f26b415b440 2023-11-24T08:10:13.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.317+0000 7f26aaffd640 1 -- 172.21.15.106:0/564547166 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7f26ac057c70 con 0x7f26b415b440 2023-11-24T08:10:13.321 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.319+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f269806af00 msgr2=0x7f269806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.319+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f269806af00 0x7f269806d3c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f26b0036000 tx=0x7f26b003c040 comp rx=0 tx=0).stop 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 msgr2=0x7f26b415ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b415ce70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00b970 tx=0x7f26ac00be30 comp rx=0 tx=0).stop 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 shutdown_connections 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f26b415b440 0x7f26b415ce70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f269806af00 0x7f269806d3c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f26b40cbf20 0x7f26b4150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 --2- 172.21.15.106:0/564547166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f26b40c59b0 0x7f26b409a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 >> 172.21.15.106:0/564547166 conn(0x7f26b40b0c50 msgr2=0x7f26b40c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:13.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 shutdown_connections 2023-11-24T08:10:13.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.320+0000 7f26bc5a9640 1 -- 172.21.15.106:0/564547166 wait complete. 2023-11-24T08:10:13.341 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:10:13.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:13.341 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:13.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:10:13.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:13.441 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:10:13.492 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:13.517 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:13.579 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:13.579 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:13.631 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:13.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.849+0000 7f53e94c2640 1 Processor -- start 2023-11-24T08:10:13.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.849+0000 7f53e94c2640 1 -- start start 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.851+0000 7f53e94c2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f53e4159ce0 0x7f53e406de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.851+0000 7f53e94c2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e40cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.851+0000 7f53e94c2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53e40cbf80 0x7f53e415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.851+0000 7f53e94c2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40cc410 con 0x7f53e40cbf80 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.851+0000 7f53e94c2640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53e40b3a40 con 0x7f53e4159ce0 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.851+0000 7f53e94c2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e40cdfa0 con 0x7f53e40b21d0 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53e40cbf80 0x7f53e415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53e40cbf80 0x7f53e415d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58482/0 (socket says 172.21.15.106:58482) 2023-11-24T08:10:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 -- 172.21.15.106:0/680186951 learned_addr learned my addr 172.21.15.106:0/680186951 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/680186951 >> v2:172.21.15.156:3300/0 conn(0x7f53e4159ce0 0x7f53e406de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 -- 172.21.15.106:0/680186951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 msgr2=0x7f53e40cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e27fc640 1 --2- 172.21.15.106:0/680186951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e40cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 --2- 172.21.15.106:0/680186951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e40cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 -- 172.21.15.106:0/680186951 >> v2:172.21.15.156:3300/0 conn(0x7f53e4159ce0 msgr2=0x7f53e406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 --2- 172.21.15.106:0/680186951 >> v2:172.21.15.156:3300/0 conn(0x7f53e4159ce0 0x7f53e406de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e37fe640 1 -- 172.21.15.106:0/680186951 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e40cd9e0 con 0x7f53e40cbf80 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.852+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/680186951 >> v2:172.21.15.156:3300/0 conn(0x7f53e4159ce0 0x7f53e406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.853+0000 7f53e37fe640 1 --2- 172.21.15.106:0/680186951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e415d760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f53e406e770 tx=0x7f53dc02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1efc43e60baaf591 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.853+0000 7f53e1ffb640 1 -- 172.21.15.106:0/680186951 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc031860 con 0x7f53e40cbf80 2023-11-24T08:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.853+0000 7f53e1ffb640 1 -- 172.21.15.106:0/680186951 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53dc0319c0 con 0x7f53e40cbf80 2023-11-24T08:10:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.853+0000 7f53e1ffb640 1 -- 172.21.15.106:0/680186951 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53dc031d50 con 0x7f53e40cbf80 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 -- 172.21.15.106:0/680186951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 msgr2=0x7f53e415d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 --2- 172.21.15.106:0/680186951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e415d760 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f53e406e770 tx=0x7f53dc02b900 comp rx=0 tx=0).stop 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 -- 172.21.15.106:0/680186951 shutdown_connections 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 --2- 172.21.15.106:0/680186951 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e415d760 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 --2- 172.21.15.106:0/680186951 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e40cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 --2- 172.21.15.106:0/680186951 >> v2:172.21.15.156:3300/0 conn(0x7f53e4159ce0 0x7f53e406de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 -- 172.21.15.106:0/680186951 >> 172.21.15.106:0/680186951 conn(0x7f53e406a920 msgr2=0x7f53e406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 -- 172.21.15.106:0/680186951 shutdown_connections 2023-11-24T08:10:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.854+0000 7f53e94c2640 1 -- 172.21.15.106:0/680186951 wait complete. 2023-11-24T08:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.856+0000 7f53e94c2640 1 Processor -- start 2023-11-24T08:10:13.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.857+0000 7f53e94c2640 1 -- start start 2023-11-24T08:10:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e94c2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e414f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e94c2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e41f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e94c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e4159ce0 0x7f53e414d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e94c2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e414f1b0 con 0x7f53e40cbf80 2023-11-24T08:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e94c2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e41f4020 con 0x7f53e4159ce0 2023-11-24T08:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e94c2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e414e5a0 con 0x7f53e40b21d0 2023-11-24T08:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e41f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e41f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58492/0 (socket says 172.21.15.106:58492) 2023-11-24T08:10:13.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e27fc640 1 -- 172.21.15.106:0/1518612520 learned_addr learned my addr 172.21.15.106:0/1518612520 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.858+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e414f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e27fc640 1 -- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 msgr2=0x7f53e414f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e37fe640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e4159ce0 0x7f53e414d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e27fc640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e414f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e27fc640 1 -- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e4159ce0 msgr2=0x7f53e414d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e27fc640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e4159ce0 0x7f53e414d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:13.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e27fc640 1 -- 172.21.15.106:0/1518612520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53dc0094d0 con 0x7f53e40cbf80 2023-11-24T08:10:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e27fc640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e41f3ae0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c820 tx=0x7f53cc00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc004220 con 0x7f53e40cbf80 2023-11-24T08:10:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc0043a0 con 0x7f53e40cbf80 2023-11-24T08:10:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc007a70 con 0x7f53e40cbf80 2023-11-24T08:10:13.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53e414e880 con 0x7f53e40cbf80 2023-11-24T08:10:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.859+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53e414ee40 con 0x7f53e40cbf80 2023-11-24T08:10:13.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.861+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53e414f360 con 0x7f53e40cbf80 2023-11-24T08:10:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.862+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f53cc007bd0 con 0x7f53e40cbf80 2023-11-24T08:10:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.863+0000 7f53db7fe640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f53c406aed0 0x7f53c406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.863+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f53cc0894d0 con 0x7f53e40cbf80 2023-11-24T08:10:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.863+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f53c406aed0 0x7f53c406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:13.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.864+0000 7f53e2ffd640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f53c406aed0 0x7f53c406d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53e41f0c50 tx=0x7f53d4007430 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:13.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:13.868+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53e414f360 con 0x7f53e40cbf80 2023-11-24T08:10:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.044+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53e414f360 con 0x7f53e40cbf80 2023-11-24T08:10:14.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.047+0000 7f53db7fe640 1 -- 172.21.15.106:0/1518612520 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f53e414f360 con 0x7f53e40cbf80 2023-11-24T08:10:14.050 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:14.050 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:58.320664+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:14.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f53c406aed0 msgr2=0x7f53c406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f53c406aed0 0x7f53c406d390 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f53e41f0c50 tx=0x7f53d4007430 comp rx=0 tx=0).stop 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 msgr2=0x7f53e41f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e41f3ae0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f53cc00c820 tx=0x7f53cc00cce0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 shutdown_connections 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53e4159ce0 0x7f53e414d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53e40cbf80 0x7f53e41f3ae0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f53c406aed0 0x7f53c406d390 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 --2- 172.21.15.106:0/1518612520 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53e40b21d0 0x7f53e414f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 >> 172.21.15.106:0/1518612520 conn(0x7f53e406a920 msgr2=0x7f53e4070630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 shutdown_connections 2023-11-24T08:10:14.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.050+0000 7f53e94c2640 1 -- 172.21.15.106:0/1518612520 wait complete. 2023-11-24T08:10:14.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:09:54.590990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:09:55.792078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:10:14.071 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:10:14.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.278+0000 7f73038ee640 1 Processor -- start 2023-11-24T08:10:14.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.278+0000 7f73038ee640 1 -- start start 2023-11-24T08:10:14.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f73038ee640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f72fc15b440 0x7f72fc0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f73038ee640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72fc0c5dc0 con 0x7f72fc0cbf20 2023-11-24T08:10:14.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f73038ee640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72fc15eea0 con 0x7f72fc0c59b0 2023-11-24T08:10:14.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f73038ee640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72fc0cc3d0 con 0x7f72fc15b440 2023-11-24T08:10:14.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f7300e89640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f72fc0cbf20 0x7f72fc0c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43800/0 (socket says 172.21.15.106:43800) 2023-11-24T08:10:14.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f7300e89640 1 -- 172.21.15.106:0/1583502313 learned_addr learned my addr 172.21.15.106:0/1583502313 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f730168a640 1 --2- 172.21.15.106:0/1583502313 >> v2:172.21.15.156:3300/0 conn(0x7f72fc15b440 0x7f72fc0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.280+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3881914005 0 0) 0x7f72fc0c5dc0 con 0x7f72fc0cbf20 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72e8003520 con 0x7f72fc0cbf20 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100390646 0 0) 0x7f72e8003520 con 0x7f72fc0cbf20 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 >> v2:172.21.15.156:3300/0 conn(0x7f72fc15b440 msgr2=0x7f72fc0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 --2- 172.21.15.106:0/1583502313 >> v2:172.21.15.156:3300/0 conn(0x7f72fc15b440 0x7f72fc0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 >> v1:172.21.15.156:6789/0 conn(0x7f72fc0c59b0 legacy=0x7f72fc06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc0abf30 con 0x7f72fc0cbf20 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f72ec001670 con 0x7f72fc0cbf20 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 >> v1:172.21.15.106:6789/0 conn(0x7f72fc0cbf20 legacy=0x7f72fc0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e80047c0 0x7f72e8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e8005250 0x7f72e800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0cc3d0 con 0x7f72e80039c0 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e800c1a0 con 0x7f72e8005250 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e800c360 con 0x7f72e80047c0 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.281+0000 7f730168a640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.282+0000 7f730168a640 1 -- 172.21.15.106:0/1583502313 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e80047c0 msgr2=0x7f72e8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.282+0000 7f730168a640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e80047c0 0x7f72e8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.282+0000 7f730168a640 1 -- 172.21.15.106:0/1583502313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e8005250 msgr2=0x7f72e800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.282+0000 7f730168a640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e8005250 0x7f72e800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.282+0000 7f730168a640 1 -- 172.21.15.106:0/1583502313 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e8003520 con 0x7f72e80039c0 2023-11-24T08:10:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.282+0000 7f7301e8b640 1 -- 172.21.15.106:0/1583502313 reap_dead start 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.283+0000 7f730168a640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72e8003dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f72f800ba30 tx=0x7f72e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992caa1fa1f9f5ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.283+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8032680 con 0x7f72e80039c0 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.283+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f80327e0 con 0x7f72e80039c0 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.283+0000 7f72f3fff640 1 -- 172.21.15.106:0/1583502313 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8032b70 con 0x7f72e80039c0 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.283+0000 7f73038ee640 1 -- 172.21.15.106:0/1583502313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 msgr2=0x7f72e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.283+0000 7f73038ee640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72e8003dc0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f72f800ba30 tx=0x7f72e80057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.284+0000 7f73038ee640 1 -- 172.21.15.106:0/1583502313 shutdown_connections 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.284+0000 7f73038ee640 1 --2- 172.21.15.106:0/1583502313 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72e8003dc0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.284+0000 7f73038ee640 1 -- 172.21.15.106:0/1583502313 >> 172.21.15.106:0/1583502313 conn(0x7f72fc0b0c50 msgr2=0x7f72fc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.284+0000 7f73038ee640 1 -- 172.21.15.106:0/1583502313 shutdown_connections 2023-11-24T08:10:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.284+0000 7f73038ee640 1 -- 172.21.15.106:0/1583502313 wait complete. 2023-11-24T08:10:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.285+0000 7f73038ee640 1 Processor -- start 2023-11-24T08:10:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.286+0000 7f73038ee640 1 -- start start 2023-11-24T08:10:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.286+0000 7f73038ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72fc0ad780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f73038ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72fc1f0ea0 0x7f72fc0c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f73038ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc0c8270 0x7f72fc1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f73038ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0b1cb0 con 0x7f72e80039c0 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f73038ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0c6fa0 con 0x7f72fc0c8270 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f73038ee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72fc0b1f00 con 0x7f72fc1f0ea0 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72fc0ad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72fc0ad780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42144/0 (socket says 172.21.15.106:42144) 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 -- 172.21.15.106:0/4031499879 learned_addr learned my addr 172.21.15.106:0/4031499879 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 -- 172.21.15.106:0/4031499879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72fc1f0ea0 msgr2=0x7f72fc0c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72fc1f0ea0 0x7f72fc0c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 -- 172.21.15.106:0/4031499879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc0c8270 msgr2=0x7f72fc1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc0c8270 0x7f72fc1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 -- 172.21.15.106:0/4031499879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72fc0b21e0 con 0x7f72e80039c0 2023-11-24T08:10:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.287+0000 7f730168a640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72fc0ad780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f72f800bb50 tx=0x7f72f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.288+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8030aa0 con 0x7f72e80039c0 2023-11-24T08:10:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.288+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f8030c20 con 0x7f72e80039c0 2023-11-24T08:10:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.288+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f803b420 con 0x7f72e80039c0 2023-11-24T08:10:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.288+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72fc0b24c0 con 0x7f72e80039c0 2023-11-24T08:10:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.288+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72fc0b2a80 con 0x7f72e80039c0 2023-11-24T08:10:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.289+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72fc04e8b0 con 0x7f72e80039c0 2023-11-24T08:10:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.290+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f72f803b580 con 0x7f72e80039c0 2023-11-24T08:10:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.291+0000 7f72f1ffb640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72e006aed0 0x7f72e006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.291+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f72f80abff0 con 0x7f72e80039c0 2023-11-24T08:10:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.291+0000 7f7300e89640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72e006aed0 0x7f72e006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.291+0000 7f7300e89640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72e006aed0 0x7f72e006d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c7ab0 tx=0x7f72ec001570 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:14.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.295+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72f8076fa0 con 0x7f72e80039c0 2023-11-24T08:10:14.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.489+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f72fc06dc80 con 0x7f72e80039c0 2023-11-24T08:10:14.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.489+0000 7f72f1ffb640 1 -- 172.21.15.106:0/4031499879 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f72f807a5c0 con 0x7f72e80039c0 2023-11-24T08:10:14.491 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:10:14.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72e006aed0 msgr2=0x7f72e006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72e006aed0 0x7f72e006d390 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f72fc0c7ab0 tx=0x7f72ec001570 comp rx=0 tx=0).stop 2023-11-24T08:10:14.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 msgr2=0x7f72fc0ad780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72fc0ad780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f72f800bb50 tx=0x7f72f8004730 comp rx=0 tx=0).stop 2023-11-24T08:10:14.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 shutdown_connections 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72fc0c8270 0x7f72fc1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72e006aed0 0x7f72e006d390 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72fc1f0ea0 0x7f72fc0c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 --2- 172.21.15.106:0/4031499879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72e80039c0 0x7f72fc0ad780 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.492+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 >> 172.21.15.106:0/4031499879 conn(0x7f72fc0b0c50 msgr2=0x7f72fc0ac5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.493+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 shutdown_connections 2023-11-24T08:10:14.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.493+0000 7f73038ee640 1 -- 172.21.15.106:0/4031499879 wait complete. 2023-11-24T08:10:14.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:10:14.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.719+0000 7fe30bfff640 1 Processor -- start 2023-11-24T08:10:14.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.719+0000 7fe30bfff640 1 -- start start 2023-11-24T08:10:14.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe30bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe30c0b0b40 0x7fe30c0b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe30bfff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c06d430 con 0x7fe30c15a1c0 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe30bfff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe30c0b14e0 con 0x7fe30c15ded0 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe30bfff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe30c15ea70 con 0x7fe30c0b0b40 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe3027fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe30c0b0b40 0x7fe30c0b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe3027fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe30c0b0b40 0x7fe30c0b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42158/0 (socket says 172.21.15.106:42158) 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.721+0000 7fe3027fc640 1 -- 172.21.15.106:0/3236339465 learned_addr learned my addr 172.21.15.106:0/3236339465 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:14.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.722+0000 7fe3027fc640 1 -- 172.21.15.106:0/3236339465 >> v1:172.21.15.156:6789/0 conn(0x7fe30c15ded0 legacy=0x7fe30c15e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:14.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.722+0000 7fe3027fc640 1 -- 172.21.15.106:0/3236339465 >> v1:172.21.15.106:6789/0 conn(0x7fe30c15a1c0 legacy=0x7fe30c06d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:14.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.722+0000 7fe3027fc640 1 -- 172.21.15.106:0/3236339465 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe30c15ec50 con 0x7fe30c0b0b40 2023-11-24T08:10:14.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.723+0000 7fe3027fc640 1 --2- 172.21.15.106:0/3236339465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c0b0b40 0x7fe30c0b0fa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009e50 tx=0x7fe2f402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c533ce8383aa5748 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:14.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.723+0000 7fe30a7fc640 1 -- 172.21.15.106:0/3236339465 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f402ec90 con 0x7fe30c0b0b40 2023-11-24T08:10:14.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.723+0000 7fe30a7fc640 1 -- 172.21.15.106:0/3236339465 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f402edf0 con 0x7fe30c0b0b40 2023-11-24T08:10:14.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.723+0000 7fe30a7fc640 1 -- 172.21.15.106:0/3236339465 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4036610 con 0x7fe30c0b0b40 2023-11-24T08:10:14.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 -- 172.21.15.106:0/3236339465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c0b0b40 msgr2=0x7fe30c0b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 --2- 172.21.15.106:0/3236339465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c0b0b40 0x7fe30c0b0fa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009e50 tx=0x7fe2f402e0f0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 -- 172.21.15.106:0/3236339465 shutdown_connections 2023-11-24T08:10:14.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 --2- 172.21.15.106:0/3236339465 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c0b0b40 0x7fe30c0b0fa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 -- 172.21.15.106:0/3236339465 >> 172.21.15.106:0/3236339465 conn(0x7fe30c0cbb40 msgr2=0x7fe30c0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:14.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 -- 172.21.15.106:0/3236339465 shutdown_connections 2023-11-24T08:10:14.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.724+0000 7fe30bfff640 1 -- 172.21.15.106:0/3236339465 wait complete. 2023-11-24T08:10:14.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.726+0000 7fe30bfff640 1 Processor -- start 2023-11-24T08:10:14.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.726+0000 7fe30bfff640 1 -- start start 2023-11-24T08:10:14.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30bfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe30c0b0b40 0x7fe30c09c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 0x7fe30c1558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe30c15ded0 0x7fe30c0c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30bfff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c84c0 con 0x7fe30c15a1c0 2023-11-24T08:10:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30bfff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c09b7e0 con 0x7fe30c15ded0 2023-11-24T08:10:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30bfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe30c0c81e0 con 0x7fe30c0b0b40 2023-11-24T08:10:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 0x7fe30c1558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 0x7fe30c1558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42172/0 (socket says 172.21.15.106:42172) 2023-11-24T08:10:14.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 -- 172.21.15.106:0/2349650563 learned_addr learned my addr 172.21.15.106:0/2349650563 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30b7fe640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe30c15ded0 0x7fe30c0c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 -- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe30c0b0b40 msgr2=0x7fe30c09c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe30c0b0b40 0x7fe30c09c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 -- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe30c15ded0 msgr2=0x7fe30c0c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe30c15ded0 0x7fe30c0c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe3027fc640 1 -- 172.21.15.106:0/2349650563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2f4009b30 con 0x7fe30c15a1c0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.727+0000 7fe30b7fe640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe30c15ded0 0x7fe30c0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.728+0000 7fe3027fc640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 0x7fe30c1558c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009e50 tx=0x7fe2f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.728+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f4043070 con 0x7fe30c15a1c0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.728+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f400e140 con 0x7fe30c15a1c0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.728+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f403f670 con 0x7fe30c15a1c0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.728+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe30c09ba30 con 0x7fe30c15a1c0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.728+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe30c09bff0 con 0x7fe30c15a1c0 2023-11-24T08:10:14.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.729+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe30c04e840 con 0x7fe30c15a1c0 2023-11-24T08:10:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.731+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe2f4048410 con 0x7fe30c15a1c0 2023-11-24T08:10:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.731+0000 7fe308ff9640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe2ec06aed0 0x7fe2ec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.732+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f40ab770 con 0x7fe30c15a1c0 2023-11-24T08:10:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.732+0000 7fe30affd640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe2ec06aed0 0x7fe2ec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.732+0000 7fe30affd640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe2ec06aed0 0x7fe2ec06d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc003cc0 tx=0x7fe2fc0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:14.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.736+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2f4076730 con 0x7fe30c15a1c0 2023-11-24T08:10:14.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.925+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe30c04e840 con 0x7fe30c15a1c0 2023-11-24T08:10:14.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.926+0000 7fe308ff9640 1 -- 172.21.15.106:0/2349650563 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2f4079d50 con 0x7fe30c15a1c0 2023-11-24T08:10:14.928 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:10:14.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe2ec06aed0 msgr2=0x7fe2ec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe2ec06aed0 0x7fe2ec06d390 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe2fc003cc0 tx=0x7fe2fc0079b0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 msgr2=0x7fe30c1558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 0x7fe30c1558c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fe2f4009e50 tx=0x7fe2f4004a40 comp rx=0 tx=0).stop 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 shutdown_connections 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe30c15ded0 0x7fe30c0c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe30c15a1c0 0x7fe30c1558c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe2ec06aed0 0x7fe2ec06d390 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 --2- 172.21.15.106:0/2349650563 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe30c0b0b40 0x7fe30c09c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 >> 172.21.15.106:0/2349650563 conn(0x7fe30c0cbb40 msgr2=0x7fe30c0ce6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 shutdown_connections 2023-11-24T08:10:14.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:14.929+0000 7fe30bfff640 1 -- 172.21.15.106:0/2349650563 wait complete. 2023-11-24T08:10:14.950 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:10:15.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.152+0000 7f8b88cc2640 1 Processor -- start 2023-11-24T08:10:15.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.152+0000 7f8b88cc2640 1 -- start start 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b88cc2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b8415b440 0x7f8b840cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b88cc2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b840cbf20 0x7f8b840c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b88cc2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b840c57e0 0x7f8b840c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b88cc2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b8415b820 con 0x7f8b840cbf20 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b88cc2640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b840cc3a0 con 0x7f8b840c57e0 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b88cc2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8415eea0 con 0x7f8b8415b440 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b840cbf20 0x7f8b840c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b840cbf20 0x7f8b840c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42184/0 (socket says 172.21.15.106:42184) 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 -- 172.21.15.106:0/2448494201 learned_addr learned my addr 172.21.15.106:0/2448494201 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b82d9d640 1 --2- 172.21.15.106:0/2448494201 >> v2:172.21.15.156:3300/0 conn(0x7f8b840c57e0 0x7f8b840c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b8259c640 1 --2- 172.21.15.106:0/2448494201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b8415b440 0x7f8b840cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 -- 172.21.15.106:0/2448494201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b8415b440 msgr2=0x7f8b840cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 --2- 172.21.15.106:0/2448494201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b8415b440 0x7f8b840cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 -- 172.21.15.106:0/2448494201 >> v2:172.21.15.156:3300/0 conn(0x7f8b840c57e0 msgr2=0x7f8b840c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 --2- 172.21.15.106:0/2448494201 >> v2:172.21.15.156:3300/0 conn(0x7f8b840c57e0 0x7f8b840c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.154+0000 7f8b81d9b640 1 -- 172.21.15.106:0/2448494201 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b840abf30 con 0x7f8b840cbf20 2023-11-24T08:10:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.156+0000 7f8b81d9b640 1 --2- 172.21.15.106:0/2448494201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840cbf20 0x7f8b840c52a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8b840cb910 tx=0x7f8b7802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d77a4ae45db00111 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.156+0000 7f8b8159a640 1 -- 172.21.15.106:0/2448494201 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b780317d0 con 0x7f8b840cbf20 2023-11-24T08:10:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.156+0000 7f8b8159a640 1 -- 172.21.15.106:0/2448494201 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b78031930 con 0x7f8b840cbf20 2023-11-24T08:10:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.156+0000 7f8b8159a640 1 -- 172.21.15.106:0/2448494201 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b78031cc0 con 0x7f8b840cbf20 2023-11-24T08:10:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.156+0000 7f8b88cc2640 1 -- 172.21.15.106:0/2448494201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840cbf20 msgr2=0x7f8b840c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.156+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/2448494201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840cbf20 0x7f8b840c52a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8b840cb910 tx=0x7f8b7802b870 comp rx=0 tx=0).stop 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 -- 172.21.15.106:0/2448494201 shutdown_connections 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/2448494201 >> v2:172.21.15.156:3300/0 conn(0x7f8b840c57e0 0x7f8b840c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/2448494201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840cbf20 0x7f8b840c52a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/2448494201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b8415b440 0x7f8b840cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 -- 172.21.15.106:0/2448494201 >> 172.21.15.106:0/2448494201 conn(0x7f8b840b0c50 msgr2=0x7f8b840b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 -- 172.21.15.106:0/2448494201 shutdown_connections 2023-11-24T08:10:15.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.157+0000 7f8b88cc2640 1 -- 172.21.15.106:0/2448494201 wait complete. 2023-11-24T08:10:15.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.158+0000 7f8b88cc2640 1 Processor -- start 2023-11-24T08:10:15.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.158+0000 7f8b88cc2640 1 -- start start 2023-11-24T08:10:15.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.159+0000 7f8b88cc2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 0x7f8b840cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.159+0000 7f8b88cc2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b840cbf20 0x7f8b8414c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.159+0000 7f8b88cc2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8415b440 0x7f8b8414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.159+0000 7f8b88cc2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8409a5f0 con 0x7f8b840c57e0 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.159+0000 7f8b88cc2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84151430 con 0x7f8b8415b440 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.159+0000 7f8b88cc2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84151710 con 0x7f8b840cbf20 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 0x7f8b840cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 0x7f8b840cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42190/0 (socket says 172.21.15.106:42190) 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 -- 172.21.15.106:0/710420296 learned_addr learned my addr 172.21.15.106:0/710420296 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b82d9d640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8415b440 0x7f8b8414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b81d9b640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b840cbf20 0x7f8b8414c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 -- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b840cbf20 msgr2=0x7f8b8414c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b840cbf20 0x7f8b8414c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 -- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8415b440 msgr2=0x7f8b8414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8415b440 0x7f8b8414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b8259c640 1 -- 172.21.15.106:0/710420296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780094d0 con 0x7f8b840c57e0 2023-11-24T08:10:15.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.160+0000 7f8b82d9d640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8415b440 0x7f8b8414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.161+0000 7f8b8259c640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 0x7f8b840cde30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400c7f0 tx=0x7f8b7400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.161+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74004220 con 0x7f8b840c57e0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.161+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74004380 con 0x7f8b840c57e0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.161+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74007840 con 0x7f8b840c57e0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.161+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b841519f0 con 0x7f8b840c57e0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.161+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b840c7a00 con 0x7f8b840c57e0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.162+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8404e8b0 con 0x7f8b840c57e0 2023-11-24T08:10:15.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.163+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8b74007bb0 con 0x7f8b840c57e0 2023-11-24T08:10:15.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.164+0000 7f8b72ffd640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b6406aed0 0x7f8b6406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.164+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b740895b0 con 0x7f8b840c57e0 2023-11-24T08:10:15.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.164+0000 7f8b81d9b640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b6406aed0 0x7f8b6406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.165+0000 7f8b81d9b640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b6406aed0 0x7f8b6406d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b78036000 tx=0x7f8b7803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:15.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.168+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b740545f0 con 0x7f8b840c57e0 2023-11-24T08:10:15.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.360+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b8406dc80 con 0x7f8b840c57e0 2023-11-24T08:10:15.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.361+0000 7f8b72ffd640 1 -- 172.21.15.106:0/710420296 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8b74007e60 con 0x7f8b840c57e0 2023-11-24T08:10:15.363 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b6406aed0 msgr2=0x7f8b6406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b6406aed0 0x7f8b6406d390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8b78036000 tx=0x7f8b7803d040 comp rx=0 tx=0).stop 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 msgr2=0x7f8b840cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 0x7f8b840cde30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8b7400c7f0 tx=0x7f8b7400ccb0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 shutdown_connections 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b8415b440 0x7f8b8414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b6406aed0 0x7f8b6406d390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b840cbf20 0x7f8b8414c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 --2- 172.21.15.106:0/710420296 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b840c57e0 0x7f8b840cde30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 >> 172.21.15.106:0/710420296 conn(0x7f8b840b0c50 msgr2=0x7f8b8406c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 shutdown_connections 2023-11-24T08:10:15.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.365+0000 7f8b88cc2640 1 -- 172.21.15.106:0/710420296 wait complete. 2023-11-24T08:10:15.385 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:10:15.562 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:10:15.572 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:10:15.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.579+0000 7f00154c2640 1 Processor -- start 2023-11-24T08:10:15.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.579+0000 7f00154c2640 1 -- start start 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f00154c2640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f00100b63d0 0x7f00100d3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f00154c2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f00100ce270 0x7f00100d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f00154c2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00100cea30 con 0x7f001015d490 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f00154c2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00100d0450 con 0x7f00100ce270 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f00154c2640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00100d0bf0 con 0x7f00100b63d0 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f000ffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f001015d490 0x7f00100cd8f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43820/0 (socket says 172.21.15.106:43820) 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f000ffff640 1 -- 172.21.15.106:0/2579361471 learned_addr learned my addr 172.21.15.106:0/2579361471 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f0014cc1640 1 --2- 172.21.15.106:0/2579361471 >> v2:172.21.15.106:3300/0 conn(0x7f00100ce270 0x7f00100d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.581+0000 7f0007fff640 1 --2- 172.21.15.106:0/2579361471 >> v2:172.21.15.156:3300/0 conn(0x7f00100b63d0 0x7f00100d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.582+0000 7f0014cc1640 1 -- 172.21.15.106:0/2579361471 >> v2:172.21.15.156:3300/0 conn(0x7f00100b63d0 msgr2=0x7f00100d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:15.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.582+0000 7f0014cc1640 1 --2- 172.21.15.106:0/2579361471 >> v2:172.21.15.156:3300/0 conn(0x7f00100b63d0 0x7f00100d3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.582+0000 7f0014cc1640 1 -- 172.21.15.106:0/2579361471 >> v1:172.21.15.106:6789/0 conn(0x7f001015d490 legacy=0x7f00100cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.582+0000 7f0014cc1640 1 -- 172.21.15.106:0/2579361471 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00100d2260 con 0x7f00100ce270 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.583+0000 7f0014cc1640 1 --2- 172.21.15.106:0/2579361471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00100ce270 0x7f00100d3770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f00100b5e70 tx=0x7f000802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c91fb3d4dc33595d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.583+0000 7f000f7fe640 1 -- 172.21.15.106:0/2579361471 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000802cc80 con 0x7f00100ce270 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.583+0000 7f000f7fe640 1 -- 172.21.15.106:0/2579361471 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000802cde0 con 0x7f00100ce270 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.583+0000 7f000f7fe640 1 -- 172.21.15.106:0/2579361471 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008034660 con 0x7f00100ce270 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 -- 172.21.15.106:0/2579361471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00100ce270 msgr2=0x7f00100d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 --2- 172.21.15.106:0/2579361471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00100ce270 0x7f00100d3770 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f00100b5e70 tx=0x7f000802bed0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 -- 172.21.15.106:0/2579361471 shutdown_connections 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 --2- 172.21.15.106:0/2579361471 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f00100ce270 0x7f00100d3770 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 --2- 172.21.15.106:0/2579361471 >> v2:172.21.15.156:3300/0 conn(0x7f00100b63d0 0x7f00100d3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 -- 172.21.15.106:0/2579361471 >> 172.21.15.106:0/2579361471 conn(0x7f0010061340 msgr2=0x7f0010061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:15.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 -- 172.21.15.106:0/2579361471 shutdown_connections 2023-11-24T08:10:15.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.584+0000 7f00154c2640 1 -- 172.21.15.106:0/2579361471 wait complete. 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.586+0000 7f00154c2640 1 Processor -- start 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.586+0000 7f00154c2640 1 -- start start 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f00154c2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00100ce270 0x7f00101540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f00154c2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 0x7f00100cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f00154c2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00100c6fa0 0x7f00100b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f00154c2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001015d870 con 0x7f001015d490 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f00154c2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0010154610 con 0x7f00100ce270 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f00154c2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001015dac0 con 0x7f00100c6fa0 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.587+0000 7f0007fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 0x7f00100cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 0x7f00100cbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42214/0 (socket says 172.21.15.106:42214) 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 -- 172.21.15.106:0/1806461556 learned_addr learned my addr 172.21.15.106:0/1806461556 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f000ffff640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00100ce270 0x7f00101540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0014cc1640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00100c6fa0 0x7f00100b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 -- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00100c6fa0 msgr2=0x7f00100b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00100c6fa0 0x7f00100b80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 -- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00100ce270 msgr2=0x7f00101540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00100ce270 0x7f00101540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 -- 172.21.15.106:0/1806461556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0008009a70 con 0x7f001015d490 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.588+0000 7f0007fff640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 0x7f00100cbd30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efff800d830 tx=0x7efff800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.589+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800b6c0 con 0x7f001015d490 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.589+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efff800b820 con 0x7f001015d490 2023-11-24T08:10:15.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.589+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001015dda0 con 0x7f001015d490 2023-11-24T08:10:15.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.589+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efff800bbb0 con 0x7f001015d490 2023-11-24T08:10:15.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.589+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001015e360 con 0x7f001015d490 2023-11-24T08:10:15.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.590+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001004e840 con 0x7f001015d490 2023-11-24T08:10:15.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.591+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7efff801e460 con 0x7f001015d490 2023-11-24T08:10:15.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.592+0000 7f000d7fa640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efff006aed0 0x7efff006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:15.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.592+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efff8089e20 con 0x7f001015d490 2023-11-24T08:10:15.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.593+0000 7f000ffff640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efff006aed0 0x7efff006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:15.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.593+0000 7f000ffff640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efff006aed0 0x7efff006d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0000005880 tx=0x7f0000009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:15.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.597+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efff8054de0 con 0x7f001015d490 2023-11-24T08:10:15.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.790+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f00100b2380 con 0x7f001015d490 2023-11-24T08:10:15.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.791+0000 7f000d7fa640 1 -- 172.21.15.106:0/1806461556 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v36) v1 ==== 80+0+0 (secure 0 0 0) 0x7efff8058400 con 0x7f001015d490 2023-11-24T08:10:15.792 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:10:15.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.793+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efff006aed0 msgr2=0x7efff006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.793+0000 7f00154c2640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efff006aed0 0x7efff006d390 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0000005880 tx=0x7f0000009040 comp rx=0 tx=0).stop 2023-11-24T08:10:15.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.793+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 msgr2=0x7f00100cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:15.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.793+0000 7f00154c2640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 0x7f00100cbd30 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efff800d830 tx=0x7efff800dcf0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 shutdown_connections 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efff006aed0 0x7efff006d390 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f00100c6fa0 0x7f00100b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f001015d490 0x7f00100cbd30 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 --2- 172.21.15.106:0/1806461556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00100ce270 0x7f00101540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 >> 172.21.15.106:0/1806461556 conn(0x7f0010061340 msgr2=0x7f00100b38e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 shutdown_connections 2023-11-24T08:10:15.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:15.794+0000 7f00154c2640 1 -- 172.21.15.106:0/1806461556 wait complete. 2023-11-24T08:10:15.812 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:10:15.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:15.812 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:15.931 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:10:15.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:15.931 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:10:15.982 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:15.983 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:16.041 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:10:16.041 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:16.041 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:16.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:10:16.100 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:16.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.319+0000 7fb34103d640 1 Processor -- start 2023-11-24T08:10:16.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.319+0000 7fb34103d640 1 -- start start 2023-11-24T08:10:16.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.320+0000 7fb34103d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.320+0000 7fb34103d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb33c0b2630 0x7fb33c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb34103d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c15ab80 con 0x7fb33c0cd470 2023-11-24T08:10:16.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb34103d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c15a990 con 0x7fb33c0b2630 2023-11-24T08:10:16.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb34103d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c15e680 con 0x7fb33c0b2250 2023-11-24T08:10:16.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb33a59c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb33c0b2630 0x7fb33c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb33ad9d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb33a59c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb33c0b2630 0x7fb33c0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39078/0 (socket says 172.21.15.106:39078) 2023-11-24T08:10:16.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.321+0000 7fb33a59c640 1 -- 172.21.15.106:0/3576805139 learned_addr learned my addr 172.21.15.106:0/3576805139 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:16.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.322+0000 7fb33a59c640 1 -- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 msgr2=0x7fb33c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:16.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.322+0000 7fb33a59c640 1 --2- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.322+0000 7fb33a59c640 1 -- 172.21.15.106:0/3576805139 >> v1:172.21.15.156:6789/0 conn(0x7fb33c0cd470 legacy=0x7fb33c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.322+0000 7fb33a59c640 1 -- 172.21.15.106:0/3576805139 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c15e960 con 0x7fb33c0b2630 2023-11-24T08:10:16.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.322+0000 7fb33ad9d640 1 --2- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:10:16.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.323+0000 7fb33a59c640 1 --2- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c0ccf30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df51397f1dea52ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:16.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.324+0000 7fb339d9b640 1 -- 172.21.15.106:0/3576805139 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330030840 con 0x7fb33c0b2630 2023-11-24T08:10:16.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.324+0000 7fb339d9b640 1 -- 172.21.15.106:0/3576805139 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3300309a0 con 0x7fb33c0b2630 2023-11-24T08:10:16.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.324+0000 7fb339d9b640 1 -- 172.21.15.106:0/3576805139 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330030d30 con 0x7fb33c0b2630 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 -- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 msgr2=0x7fb33c0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 --2- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c0ccf30 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002b980 comp rx=0 tx=0).stop 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 -- 172.21.15.106:0/3576805139 shutdown_connections 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 --2- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c0ccf30 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 --2- 172.21.15.106:0/3576805139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 -- 172.21.15.106:0/3576805139 >> 172.21.15.106:0/3576805139 conn(0x7fb33c06a920 msgr2=0x7fb33c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 -- 172.21.15.106:0/3576805139 shutdown_connections 2023-11-24T08:10:16.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.325+0000 7fb34103d640 1 -- 172.21.15.106:0/3576805139 wait complete. 2023-11-24T08:10:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.327+0000 7fb34103d640 1 Processor -- start 2023-11-24T08:10:16.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.327+0000 7fb34103d640 1 -- start start 2023-11-24T08:10:16.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb34103d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb34103d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c1522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb34103d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 0x7fb33c098480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb34103d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c13cbc0 con 0x7fb33c0cd470 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb34103d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c152d50 con 0x7fb33c0b2630 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb34103d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c153030 con 0x7fb33c0b2250 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.328+0000 7fb33b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 0x7fb33c098480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 0x7fb33c098480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42228/0 (socket says 172.21.15.106:42228) 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 -- 172.21.15.106:0/844520403 learned_addr learned my addr 172.21.15.106:0/844520403 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33ad9d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33a59c640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c1522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 -- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 msgr2=0x7fb33c153be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c153be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 -- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 msgr2=0x7fb33c1522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c1522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 -- 172.21.15.106:0/844520403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c009400 con 0x7fb33c0cd470 2023-11-24T08:10:16.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33a59c640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:16.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.329+0000 7fb33b59e640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 0x7fb33c098480 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c730 tx=0x7fb33400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:16.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.330+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334003b70 con 0x7fb33c0cd470 2023-11-24T08:10:16.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.330+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb334003cf0 con 0x7fb33c0cd470 2023-11-24T08:10:16.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.330+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334003ea0 con 0x7fb33c0cd470 2023-11-24T08:10:16.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.330+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3300094d0 con 0x7fb33c0cd470 2023-11-24T08:10:16.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.330+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c13d0f0 con 0x7fb33c0cd470 2023-11-24T08:10:16.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.331+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c067470 con 0x7fb33c0cd470 2023-11-24T08:10:16.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.332+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb334005310 con 0x7fb33c0cd470 2023-11-24T08:10:16.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.333+0000 7fb3237fe640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb32806af00 0x7fb32806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.333+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb334089330 con 0x7fb33c0cd470 2023-11-24T08:10:16.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.333+0000 7fb33ad9d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb32806af00 0x7fb32806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.334+0000 7fb33ad9d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb32806af00 0x7fb32806d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009c60 tx=0x7fb32c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:16.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.337+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb334054480 con 0x7fb33c0cd470 2023-11-24T08:10:16.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.568+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb33c072880 con 0x7fb33c0cd470 2023-11-24T08:10:16.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.569+0000 7fb3237fe640 1 -- 172.21.15.106:0/844520403 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fb334057aa0 con 0x7fb33c0cd470 2023-11-24T08:10:16.571 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:16.571 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:09:56.761029+0000","modified":"2023-11-24T08:09:59.823336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6836/3854927347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3854927347},{"type":"v1","addr":"172.21.15.106:6836","nonce":3854927347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:10:16.572 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 15 2023-11-24T08:10:16.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb32806af00 msgr2=0x7fb32806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb32806af00 0x7fb32806d3c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fb32c009c60 tx=0x7fb32c008040 comp rx=0 tx=0).stop 2023-11-24T08:10:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 msgr2=0x7fb33c098480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 0x7fb33c098480 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c730 tx=0x7fb33400cbf0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 shutdown_connections 2023-11-24T08:10:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0cd470 0x7fb33c098480 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c0b2630 0x7fb33c1522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb32806af00 0x7fb32806d3c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 --2- 172.21.15.106:0/844520403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b2250 0x7fb33c153be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.572+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 >> 172.21.15.106:0/844520403 conn(0x7fb33c06a920 msgr2=0x7fb33c15ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.573+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 shutdown_connections 2023-11-24T08:10:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.573+0000 7fb34103d640 1 -- 172.21.15.106:0/844520403 wait complete. 2023-11-24T08:10:16.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:16.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.802+0000 7f56bf6da640 1 Processor -- start 2023-11-24T08:10:16.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.802+0000 7f56bf6da640 1 -- start start 2023-11-24T08:10:16.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bf6da640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56b815c7f0 0x7f56b806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bf6da640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f56b80ce6c0 0x7f56b80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bf6da640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56b80cece0 con 0x7f56b80cf450 2023-11-24T08:10:16.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bf6da640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b80cfe40 con 0x7f56b815c7f0 2023-11-24T08:10:16.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bf6da640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56b80ceaf0 con 0x7f56b80ce6c0 2023-11-24T08:10:16.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bd476640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56b815c7f0 0x7f56b806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bd476640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f56b815c7f0 0x7f56b806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42244/0 (socket says 172.21.15.106:42244) 2023-11-24T08:10:16.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bd476640 1 -- 172.21.15.106:0/4171594574 learned_addr learned my addr 172.21.15.106:0/4171594574 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:16.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.804+0000 7f56bdc77640 1 --2- 172.21.15.106:0/4171594574 >> v2:172.21.15.156:3300/0 conn(0x7f56b80ce6c0 0x7f56b80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.805+0000 7f56bd476640 1 -- 172.21.15.106:0/4171594574 >> v2:172.21.15.156:3300/0 conn(0x7f56b80ce6c0 msgr2=0x7f56b80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:16.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.805+0000 7f56bd476640 1 --2- 172.21.15.106:0/4171594574 >> v2:172.21.15.156:3300/0 conn(0x7f56b80ce6c0 0x7f56b80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.805+0000 7f56bd476640 1 -- 172.21.15.106:0/4171594574 >> v1:172.21.15.106:6789/0 conn(0x7f56b80cf450 legacy=0x7f56b80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.805+0000 7f56bd476640 1 -- 172.21.15.106:0/4171594574 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b806d040 con 0x7f56b815c7f0 2023-11-24T08:10:16.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.806+0000 7f56bd476640 1 --2- 172.21.15.106:0/4171594574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b806db20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f56b80d00e0 tx=0x7f56b402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=947af31dd5f50485 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:16.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.806+0000 7f56affff640 1 -- 172.21.15.106:0/4171594574 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4030860 con 0x7f56b815c7f0 2023-11-24T08:10:16.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.806+0000 7f56affff640 1 -- 172.21.15.106:0/4171594574 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b40309c0 con 0x7f56b815c7f0 2023-11-24T08:10:16.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.806+0000 7f56affff640 1 -- 172.21.15.106:0/4171594574 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b4030d50 con 0x7f56b815c7f0 2023-11-24T08:10:16.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 -- 172.21.15.106:0/4171594574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 msgr2=0x7f56b806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 --2- 172.21.15.106:0/4171594574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b806db20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f56b80d00e0 tx=0x7f56b402b9a0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 -- 172.21.15.106:0/4171594574 shutdown_connections 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 --2- 172.21.15.106:0/4171594574 >> v2:172.21.15.156:3300/0 conn(0x7f56b80ce6c0 0x7f56b80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 --2- 172.21.15.106:0/4171594574 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b806db20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 -- 172.21.15.106:0/4171594574 >> 172.21.15.106:0/4171594574 conn(0x7f56b806a920 msgr2=0x7f56b806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 -- 172.21.15.106:0/4171594574 shutdown_connections 2023-11-24T08:10:16.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.807+0000 7f56bf6da640 1 -- 172.21.15.106:0/4171594574 wait complete. 2023-11-24T08:10:16.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.809+0000 7f56bf6da640 1 Processor -- start 2023-11-24T08:10:16.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.809+0000 7f56bf6da640 1 -- start start 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bf6da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b80ce6c0 0x7f56b80cba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bf6da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56b80cf450 0x7f56b8157190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bf6da640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b815a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bf6da640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b81576d0 con 0x7f56b815c7f0 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bf6da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8157970 con 0x7f56b80ce6c0 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bf6da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56b8157c50 con 0x7f56b80cf450 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bdc77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b815a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bdc77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b815a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42250/0 (socket says 172.21.15.106:42250) 2023-11-24T08:10:16.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bdc77640 1 -- 172.21.15.106:0/3957090462 learned_addr learned my addr 172.21.15.106:0/3957090462 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bd476640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b80ce6c0 0x7f56b80cba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bcc75640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56b80cf450 0x7f56b8157190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.810+0000 7f56bdc77640 1 -- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56b80cf450 msgr2=0x7f56b8157190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bdc77640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56b80cf450 0x7f56b8157190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bdc77640 1 -- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b80ce6c0 msgr2=0x7f56b80cba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bdc77640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b80ce6c0 0x7f56b80cba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bdc77640 1 -- 172.21.15.106:0/3957090462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b40094d0 con 0x7f56b815c7f0 2023-11-24T08:10:16.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bd476640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b80ce6c0 0x7f56b80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:16.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bdc77640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b815a990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007d90 tx=0x7f56b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:16.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0014420 con 0x7f56b815c7f0 2023-11-24T08:10:16.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b00145a0 con 0x7f56b815c7f0 2023-11-24T08:10:16.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b0014950 con 0x7f56b815c7f0 2023-11-24T08:10:16.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b8157f30 con 0x7f56b815c7f0 2023-11-24T08:10:16.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.811+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b81584f0 con 0x7f56b815c7f0 2023-11-24T08:10:16.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.812+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b81586e0 con 0x7f56b815c7f0 2023-11-24T08:10:16.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.813+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f56b0014ab0 con 0x7f56b815c7f0 2023-11-24T08:10:16.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.814+0000 7f56adffb640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f569c06aed0 0x7f569c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:16.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.814+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f56b0089fe0 con 0x7f56b815c7f0 2023-11-24T08:10:16.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.815+0000 7f56bd476640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f569c06aed0 0x7f569c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:16.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.815+0000 7f56bd476640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f569c06aed0 0x7f569c06d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56b4036000 tx=0x7f56b403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:16.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:16.819+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f56b0055030 con 0x7f56b815c7f0 2023-11-24T08:10:17.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.009+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56b8067470 con 0x7f56b815c7f0 2023-11-24T08:10:17.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.012+0000 7f56adffb640 1 -- 172.21.15.106:0/3957090462 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16617 (secure 0 0 0) 0x7f56b0058650 con 0x7f56b815c7f0 2023-11-24T08:10:17.014 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:17.014 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:58.320664+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:17.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f569c06aed0 msgr2=0x7f569c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f569c06aed0 0x7f569c06d390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f56b4036000 tx=0x7f56b403b040 comp rx=0 tx=0).stop 2023-11-24T08:10:17.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 msgr2=0x7f56b815a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b815a990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007d90 tx=0x7f56b000e4b0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 shutdown_connections 2023-11-24T08:10:17.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f56b815c7f0 0x7f56b815a990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f569c06aed0 0x7f569c06d390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f56b80cf450 0x7f56b8157190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 --2- 172.21.15.106:0/3957090462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56b80ce6c0 0x7f56b80cba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 >> 172.21.15.106:0/3957090462 conn(0x7f56b806a920 msgr2=0x7f56b815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:17.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 shutdown_connections 2023-11-24T08:10:17.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.015+0000 7f56bf6da640 1 -- 172.21.15.106:0/3957090462 wait complete. 2023-11-24T08:10:17.036 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:09:54.590990+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '35', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:09:55.792078+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '36', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:10:17.036 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.251+0000 7f54d881c640 1 Processor -- start 2023-11-24T08:10:17.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.252+0000 7f54d881c640 1 -- start start 2023-11-24T08:10:17.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d881c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f54d415b440 0x7f54d40cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d881c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54d40c59b0 0x7f54d406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d881c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d40c5dc0 con 0x7f54d40cbf20 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d881c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d40cc3d0 con 0x7f54d40c59b0 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d881c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d415b820 con 0x7f54d415b440 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54d40c59b0 0x7f54d406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54d40c59b0 0x7f54d406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42254/0 (socket says 172.21.15.106:42254) 2023-11-24T08:10:17.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.253+0000 7f54d2d9d640 1 -- 172.21.15.106:0/3284639466 learned_addr learned my addr 172.21.15.106:0/3284639466 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.254+0000 7f54d259c640 1 --2- 172.21.15.106:0/3284639466 >> v2:172.21.15.156:3300/0 conn(0x7f54d415b440 0x7f54d40cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.254+0000 7f54d2d9d640 1 -- 172.21.15.106:0/3284639466 >> v2:172.21.15.156:3300/0 conn(0x7f54d415b440 msgr2=0x7f54d40cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.254+0000 7f54d2d9d640 1 --2- 172.21.15.106:0/3284639466 >> v2:172.21.15.156:3300/0 conn(0x7f54d415b440 0x7f54d40cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.254+0000 7f54d2d9d640 1 -- 172.21.15.106:0/3284639466 >> v1:172.21.15.106:6789/0 conn(0x7f54d40cbf20 legacy=0x7f54d40c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.254+0000 7f54d2d9d640 1 -- 172.21.15.106:0/3284639466 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d415eea0 con 0x7f54d40c59b0 2023-11-24T08:10:17.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.255+0000 7f54d2d9d640 1 --2- 172.21.15.106:0/3284639466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d406c5f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f54d406e770 tx=0x7f54cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415f35a79dfb8292 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.255+0000 7f54d159a640 1 -- 172.21.15.106:0/3284639466 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc031870 con 0x7f54d40c59b0 2023-11-24T08:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.255+0000 7f54d159a640 1 -- 172.21.15.106:0/3284639466 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc0319d0 con 0x7f54d40c59b0 2023-11-24T08:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.255+0000 7f54d159a640 1 -- 172.21.15.106:0/3284639466 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc031d60 con 0x7f54d40c59b0 2023-11-24T08:10:17.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 -- 172.21.15.106:0/3284639466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 msgr2=0x7f54d406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 --2- 172.21.15.106:0/3284639466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d406c5f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f54d406e770 tx=0x7f54cc02b910 comp rx=0 tx=0).stop 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 -- 172.21.15.106:0/3284639466 shutdown_connections 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 --2- 172.21.15.106:0/3284639466 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d406c5f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 --2- 172.21.15.106:0/3284639466 >> v2:172.21.15.156:3300/0 conn(0x7f54d415b440 0x7f54d40cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 -- 172.21.15.106:0/3284639466 >> 172.21.15.106:0/3284639466 conn(0x7f54d40b0c50 msgr2=0x7f54d40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 -- 172.21.15.106:0/3284639466 shutdown_connections 2023-11-24T08:10:17.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.256+0000 7f54d881c640 1 -- 172.21.15.106:0/3284639466 wait complete. 2023-11-24T08:10:17.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.258+0000 7f54d881c640 1 Processor -- start 2023-11-24T08:10:17.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.258+0000 7f54d881c640 1 -- start start 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d881c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d40c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d881c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54d40cbf20 0x7f54d40adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d881c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d415b440 0x7f54d414cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d881c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40c90f0 con 0x7f54d40c59b0 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d881c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40c8470 con 0x7f54d415b440 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d881c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d40c8750 con 0x7f54d40cbf20 2023-11-24T08:10:17.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d40c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d40c7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42268/0 (socket says 172.21.15.106:42268) 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 -- 172.21.15.106:0/1108937484 learned_addr learned my addr 172.21.15.106:0/1108937484 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 -- 172.21.15.106:0/1108937484 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54d40cbf20 msgr2=0x7f54d40adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54d40cbf20 0x7f54d40adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 -- 172.21.15.106:0/1108937484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d415b440 msgr2=0x7f54d414cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d415b440 0x7f54d414cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.259+0000 7f54d259c640 1 -- 172.21.15.106:0/1108937484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0094d0 con 0x7f54d40c59b0 2023-11-24T08:10:17.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.260+0000 7f54d259c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d40c7e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f54c400ccd0 tx=0x7f54c400a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:17.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.260+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4003990 con 0x7f54d40c59b0 2023-11-24T08:10:17.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.260+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c4003af0 con 0x7f54d40c59b0 2023-11-24T08:10:17.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.260+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4004020 con 0x7f54d40c59b0 2023-11-24T08:10:17.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.260+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d40c8a30 con 0x7f54d40c59b0 2023-11-24T08:10:17.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.260+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d40ae170 con 0x7f54d40c59b0 2023-11-24T08:10:17.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.261+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d415ec80 con 0x7f54d40c59b0 2023-11-24T08:10:17.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.262+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f54c4004180 con 0x7f54d40c59b0 2023-11-24T08:10:17.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.263+0000 7f54caffd640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54b406aed0 0x7f54b406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.264+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f54c4089a90 con 0x7f54d40c59b0 2023-11-24T08:10:17.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.264+0000 7f54d1d9b640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54b406aed0 0x7f54b406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.264+0000 7f54d1d9b640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54b406aed0 0x7f54b406d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f54bc000f60 tx=0x7f54bc0079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:17.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.268+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54c4054ad0 con 0x7f54d40c59b0 2023-11-24T08:10:17.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.462+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f54d406dc80 con 0x7f54d40c59b0 2023-11-24T08:10:17.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.463+0000 7f54caffd640 1 -- 172.21.15.106:0/1108937484 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7f54c401d070 con 0x7f54d40c59b0 2023-11-24T08:10:17.465 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:10:17.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54b406aed0 msgr2=0x7f54b406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54b406aed0 0x7f54b406d390 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f54bc000f60 tx=0x7f54bc0079b0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 msgr2=0x7f54d40c7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d40c7e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f54c400ccd0 tx=0x7f54c400a5c0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 shutdown_connections 2023-11-24T08:10:17.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d415b440 0x7f54d414cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54b406aed0 0x7f54b406d390 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54d40cbf20 0x7f54d40adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 --2- 172.21.15.106:0/1108937484 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54d40c59b0 0x7f54d40c7e40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 >> 172.21.15.106:0/1108937484 conn(0x7f54d40b0c50 msgr2=0x7f54d4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:17.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 shutdown_connections 2023-11-24T08:10:17.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.467+0000 7f54d881c640 1 -- 172.21.15.106:0/1108937484 wait complete. 2023-11-24T08:10:17.488 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:10:17.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.695+0000 7f071dd8a640 1 Processor -- start 2023-11-24T08:10:17.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.695+0000 7f071dd8a640 1 -- start start 2023-11-24T08:10:17.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.696+0000 7f071dd8a640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f071806de10 0x7f071815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.696+0000 7f071dd8a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07180c49f0 0x7f07180705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.696+0000 7f071dd8a640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071815ec90 con 0x7f0718072160 2023-11-24T08:10:17.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.696+0000 7f071dd8a640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07180cd530 con 0x7f071806de10 2023-11-24T08:10:17.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.696+0000 7f071dd8a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c4ed0 con 0x7f07180c49f0 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.697+0000 7f070ffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f071806de10 0x7f071815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.697+0000 7f0717fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07180c49f0 0x7f07180705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.697+0000 7f070ffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f071806de10 0x7f071815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39134/0 (socket says 172.21.15.106:39134) 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.697+0000 7f070ffff640 1 -- 172.21.15.106:0/1648017522 learned_addr learned my addr 172.21.15.106:0/1648017522 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.697+0000 7f0717fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07180c49f0 0x7f07180705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:46984/0 (socket says 172.21.15.106:46984) 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.698+0000 7f070ffff640 1 -- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07180c49f0 msgr2=0x7f07180705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.698+0000 7f070ffff640 1 --2- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07180c49f0 0x7f07180705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.698+0000 7f070ffff640 1 -- 172.21.15.106:0/1648017522 >> v1:172.21.15.156:6789/0 conn(0x7f0718072160 legacy=0x7f071806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.698+0000 7f070ffff640 1 -- 172.21.15.106:0/1648017522 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07180c51b0 con 0x7f071806de10 2023-11-24T08:10:17.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.699+0000 7f070ffff640 1 --2- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f071806de10 0x7f071815e750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0700009870 tx=0x7f070002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1644eb4f34560bfa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:17.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f0716ffd640 1 -- 172.21.15.106:0/1648017522 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700030840 con 0x7f071806de10 2023-11-24T08:10:17.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f0716ffd640 1 -- 172.21.15.106:0/1648017522 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07000309a0 con 0x7f071806de10 2023-11-24T08:10:17.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f0716ffd640 1 -- 172.21.15.106:0/1648017522 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700030d30 con 0x7f071806de10 2023-11-24T08:10:17.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 -- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f071806de10 msgr2=0x7f071815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 --2- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f071806de10 0x7f071815e750 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f0700009870 tx=0x7f070002b980 comp rx=0 tx=0).stop 2023-11-24T08:10:17.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 -- 172.21.15.106:0/1648017522 shutdown_connections 2023-11-24T08:10:17.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 --2- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07180c49f0 0x7f07180705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 --2- 172.21.15.106:0/1648017522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f071806de10 0x7f071815e750 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 -- 172.21.15.106:0/1648017522 >> 172.21.15.106:0/1648017522 conn(0x7f07180b0c50 msgr2=0x7f07180b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:17.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 -- 172.21.15.106:0/1648017522 shutdown_connections 2023-11-24T08:10:17.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.700+0000 7f071dd8a640 1 -- 172.21.15.106:0/1648017522 wait complete. 2023-11-24T08:10:17.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.702+0000 7f071dd8a640 1 Processor -- start 2023-11-24T08:10:17.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.702+0000 7f071dd8a640 1 -- start start 2023-11-24T08:10:17.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.703+0000 7f071dd8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 0x7f07180cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.703+0000 7f071dd8a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0718072160 0x7f07180b3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.703+0000 7f071dd8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07180c49f0 0x7f07180c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.703+0000 7f071dd8a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c8c80 con 0x7f071806de10 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.703+0000 7f071dd8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c7be0 con 0x7f07180c49f0 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.703+0000 7f071dd8a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c7ec0 con 0x7f0718072160 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 0x7f07180cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 0x7f07180cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42280/0 (socket says 172.21.15.106:42280) 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 -- 172.21.15.106:0/2539742187 learned_addr learned my addr 172.21.15.106:0/2539742187 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f0717fff640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07180c49f0 0x7f07180c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f070ffff640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0718072160 0x7f07180b3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 -- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0718072160 msgr2=0x7f07180b3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0718072160 0x7f07180b3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 -- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07180c49f0 msgr2=0x7f07180c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07180c49f0 0x7f07180c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 -- 172.21.15.106:0/2539742187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07000094d0 con 0x7f071806de10 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f070ffff640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0718072160 0x7f07180b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.704+0000 7f07177fe640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 0x7f07180cac90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f070800c450 tx=0x7f070800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.705+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07080052d0 con 0x7f071806de10 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.705+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0708005430 con 0x7f071806de10 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.705+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708012870 con 0x7f071806de10 2023-11-24T08:10:17.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.705+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07180c81a0 con 0x7f071806de10 2023-11-24T08:10:17.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.705+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07180c8760 con 0x7f071806de10 2023-11-24T08:10:17.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.706+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071804e8b0 con 0x7f071806de10 2023-11-24T08:10:17.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.707+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f07080129d0 con 0x7f071806de10 2023-11-24T08:10:17.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.708+0000 7f0714ff9640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f06f806af00 0x7f06f806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:17.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.708+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0708089130 con 0x7f071806de10 2023-11-24T08:10:17.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.708+0000 7f070ffff640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f06f806af00 0x7f06f806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:17.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.709+0000 7f070ffff640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f06f806af00 0x7f06f806d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0700036000 tx=0x7f070003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:17.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.712+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0708054060 con 0x7f071806de10 2023-11-24T08:10:17.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.904+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f071815ec90 con 0x7f071806de10 2023-11-24T08:10:17.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.905+0000 7f0714ff9640 1 -- 172.21.15.106:0/2539742187 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v36) v1 ==== 104+0+11 (secure 0 0 0) 0x7f071815ec90 con 0x7f071806de10 2023-11-24T08:10:17.907 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:10:17.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f06f806af00 msgr2=0x7f06f806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f06f806af00 0x7f06f806d3c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0700036000 tx=0x7f070003b040 comp rx=0 tx=0).stop 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 msgr2=0x7f07180cac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 0x7f07180cac90 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f070800c450 tx=0x7f070800c910 comp rx=0 tx=0).stop 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 shutdown_connections 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07180c49f0 0x7f07180c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f06f806af00 0x7f06f806d3c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0718072160 0x7f07180b3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 --2- 172.21.15.106:0/2539742187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f071806de10 0x7f07180cac90 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 >> 172.21.15.106:0/2539742187 conn(0x7f07180b0c50 msgr2=0x7f07180cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 shutdown_connections 2023-11-24T08:10:17.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:17.908+0000 7f071dd8a640 1 -- 172.21.15.106:0/2539742187 wait complete. 2023-11-24T08:10:17.928 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:10:18.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.139+0000 7fb616f06640 1 Processor -- start 2023-11-24T08:10:18.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.140+0000 7fb616f06640 1 -- start start 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb616f06640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb6101608f0 0x7fb6100d1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb616f06640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb6100ce260 0x7fb6100d3180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb616f06640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb610160cd0 con 0x7fb6100ce710 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb616f06640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100b8f60 con 0x7fb6101608f0 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb616f06640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6100cdcb0 con 0x7fb6100ce260 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb615f04640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb6101608f0 0x7fb6100d1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb615f04640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb6101608f0 0x7fb6100d1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42294/0 (socket says 172.21.15.106:42294) 2023-11-24T08:10:18.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.141+0000 7fb615f04640 1 -- 172.21.15.106:0/3159183957 learned_addr learned my addr 172.21.15.106:0/3159183957 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:18.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.142+0000 7fb615703640 1 --2- 172.21.15.106:0/3159183957 >> v2:172.21.15.156:3300/0 conn(0x7fb6100ce260 0x7fb6100d3180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.142+0000 7fb615f04640 1 -- 172.21.15.106:0/3159183957 >> v2:172.21.15.156:3300/0 conn(0x7fb6100ce260 msgr2=0x7fb6100d3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:18.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.142+0000 7fb615f04640 1 --2- 172.21.15.106:0/3159183957 >> v2:172.21.15.156:3300/0 conn(0x7fb6100ce260 0x7fb6100d3180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.142+0000 7fb615f04640 1 -- 172.21.15.106:0/3159183957 >> v1:172.21.15.156:6789/0 conn(0x7fb6100ce710 legacy=0x7fb6100b63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:18.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.142+0000 7fb615f04640 1 -- 172.21.15.106:0/3159183957 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6100cdf90 con 0x7fb6101608f0 2023-11-24T08:10:18.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.143+0000 7fb615f04640 1 --2- 172.21.15.106:0/3159183957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb6100d1ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b90a0 tx=0x7fb60002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969549c76bead02c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:18.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.143+0000 7fb614f02640 1 -- 172.21.15.106:0/3159183957 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60002ca80 con 0x7fb6101608f0 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.143+0000 7fb614f02640 1 -- 172.21.15.106:0/3159183957 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb60002cbe0 con 0x7fb6101608f0 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.143+0000 7fb614f02640 1 -- 172.21.15.106:0/3159183957 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6000343f0 con 0x7fb6101608f0 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 -- 172.21.15.106:0/3159183957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 msgr2=0x7fb6100d1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 --2- 172.21.15.106:0/3159183957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb6100d1ac0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb6100b90a0 tx=0x7fb60002bee0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 -- 172.21.15.106:0/3159183957 shutdown_connections 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 --2- 172.21.15.106:0/3159183957 >> v2:172.21.15.156:3300/0 conn(0x7fb6100ce260 0x7fb6100d3180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 --2- 172.21.15.106:0/3159183957 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb6100d1ac0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 -- 172.21.15.106:0/3159183957 >> 172.21.15.106:0/3159183957 conn(0x7fb610061340 msgr2=0x7fb610061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 -- 172.21.15.106:0/3159183957 shutdown_connections 2023-11-24T08:10:18.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.144+0000 7fb616f06640 1 -- 172.21.15.106:0/3159183957 wait complete. 2023-11-24T08:10:18.147 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.146+0000 7fb616f06640 1 Processor -- start 2023-11-24T08:10:18.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.146+0000 7fb616f06640 1 -- start start 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.147+0000 7fb616f06640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6100ce260 0x7fb6100c5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.147+0000 7fb616f06640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6100ce710 0x7fb61015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.147+0000 7fb616f06640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb61015f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.147+0000 7fb616f06640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100ca060 con 0x7fb6101608f0 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.147+0000 7fb616f06640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6101f6d60 con 0x7fb6100ce260 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.147+0000 7fb616f06640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6100caaf0 con 0x7fb6100ce710 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb61015f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb61015f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42302/0 (socket says 172.21.15.106:42302) 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 -- 172.21.15.106:0/3034668718 learned_addr learned my addr 172.21.15.106:0/3034668718 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:18.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb615703640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6100ce710 0x7fb61015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb615f04640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6100ce260 0x7fb6100c5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 -- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6100ce710 msgr2=0x7fb61015f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6100ce710 0x7fb61015f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 -- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6100ce260 msgr2=0x7fb6100c5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6100ce260 0x7fb6100c5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 -- 172.21.15.106:0/3034668718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb600009b30 con 0x7fb6101608f0 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb615f04640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6100ce260 0x7fb6100c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:18.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.148+0000 7fb616705640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb61015f940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb60800c390 tx=0x7fb60800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:18.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.149+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6080035f0 con 0x7fb6101608f0 2023-11-24T08:10:18.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.149+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb608003750 con 0x7fb6101608f0 2023-11-24T08:10:18.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.149+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb608003b00 con 0x7fb6101608f0 2023-11-24T08:10:18.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.149+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6100c6350 con 0x7fb6101608f0 2023-11-24T08:10:18.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.149+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6100ca4a0 con 0x7fb6101608f0 2023-11-24T08:10:18.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.150+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6100cac80 con 0x7fb6101608f0 2023-11-24T08:10:18.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.151+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb608003c60 con 0x7fb6101608f0 2023-11-24T08:10:18.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.152+0000 7fb60e7fc640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb5fc06aed0 0x7fb5fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.152+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb608088820 con 0x7fb6101608f0 2023-11-24T08:10:18.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.152+0000 7fb615f04640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb5fc06aed0 0x7fb5fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.153+0000 7fb615f04640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb5fc06aed0 0x7fb5fc06d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb600035000 tx=0x7fb600030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:18.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.156+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb608053870 con 0x7fb6101608f0 2023-11-24T08:10:18.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.346+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb61004e840 con 0x7fb6101608f0 2023-11-24T08:10:18.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.347+0000 7fb60e7fc640 1 -- 172.21.15.106:0/3034668718 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v36) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb608056e90 con 0x7fb6101608f0 2023-11-24T08:10:18.349 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.349+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb5fc06aed0 msgr2=0x7fb5fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.349+0000 7fb616f06640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb5fc06aed0 0x7fb5fc06d390 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb600035000 tx=0x7fb600030000 comp rx=0 tx=0).stop 2023-11-24T08:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.349+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 msgr2=0x7fb61015f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.349+0000 7fb616f06640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb61015f940 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb60800c390 tx=0x7fb60800c850 comp rx=0 tx=0).stop 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 shutdown_connections 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb6101608f0 0x7fb61015f940 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb5fc06aed0 0x7fb5fc06d390 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb6100ce710 0x7fb61015f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 --2- 172.21.15.106:0/3034668718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6100ce260 0x7fb6100c5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 >> 172.21.15.106:0/3034668718 conn(0x7fb610061340 msgr2=0x7fb61015da80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 shutdown_connections 2023-11-24T08:10:18.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.350+0000 7fb616f06640 1 -- 172.21.15.106:0/3034668718 wait complete. 2023-11-24T08:10:18.370 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:18.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.585+0000 7fb33b59e640 1 Processor -- start 2023-11-24T08:10:18.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.585+0000 7fb33b59e640 1 -- start start 2023-11-24T08:10:18.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb33b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb33c15c410 0x7fb33c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb33b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b1de0 0x7fb33c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb33b59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c15bf80 con 0x7fb33c144b30 2023-11-24T08:10:18.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb33b59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0b27b0 con 0x7fb33c15c410 2023-11-24T08:10:18.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb33b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0b1560 con 0x7fb33c0b1de0 2023-11-24T08:10:18.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb33c15c410 0x7fb33c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb33c15c410 0x7fb33c15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42306/0 (socket says 172.21.15.106:42306) 2023-11-24T08:10:18.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 -- 172.21.15.106:0/56476805 learned_addr learned my addr 172.21.15.106:0/56476805 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:18.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb33ad9d640 1 --2- 172.21.15.106:0/56476805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b1de0 0x7fb33c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 -- 172.21.15.106:0/56476805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b1de0 msgr2=0x7fb33c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:18.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 --2- 172.21.15.106:0/56476805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b1de0 0x7fb33c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 -- 172.21.15.106:0/56476805 >> v1:172.21.15.156:6789/0 conn(0x7fb33c144b30 legacy=0x7fb33c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:18.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.587+0000 7fb339d9b640 1 -- 172.21.15.106:0/56476805 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0b1280 con 0x7fb33c15c410 2023-11-24T08:10:18.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.589+0000 7fb339d9b640 1 --2- 172.21.15.106:0/56476805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c15c410 0x7fb33c15f920 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2396ba2f6ad88663 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:18.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.589+0000 7fb33959a640 1 -- 172.21.15.106:0/56476805 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330031640 con 0x7fb33c15c410 2023-11-24T08:10:18.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.589+0000 7fb33959a640 1 -- 172.21.15.106:0/56476805 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3300317a0 con 0x7fb33c15c410 2023-11-24T08:10:18.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.589+0000 7fb33959a640 1 -- 172.21.15.106:0/56476805 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330031b30 con 0x7fb33c15c410 2023-11-24T08:10:18.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.589+0000 7fb33b59e640 1 -- 172.21.15.106:0/56476805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c15c410 msgr2=0x7fb33c15f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.589+0000 7fb33b59e640 1 --2- 172.21.15.106:0/56476805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c15c410 0x7fb33c15f920 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fb330009870 tx=0x7fb33002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.590+0000 7fb33b59e640 1 -- 172.21.15.106:0/56476805 shutdown_connections 2023-11-24T08:10:18.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.590+0000 7fb33b59e640 1 --2- 172.21.15.106:0/56476805 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c0b1de0 0x7fb33c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.590+0000 7fb33b59e640 1 --2- 172.21.15.106:0/56476805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c15c410 0x7fb33c15f920 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.590+0000 7fb33b59e640 1 -- 172.21.15.106:0/56476805 >> 172.21.15.106:0/56476805 conn(0x7fb33c0cb830 msgr2=0x7fb33c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:18.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.590+0000 7fb33b59e640 1 -- 172.21.15.106:0/56476805 shutdown_connections 2023-11-24T08:10:18.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.590+0000 7fb33b59e640 1 -- 172.21.15.106:0/56476805 wait complete. 2023-11-24T08:10:18.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.591+0000 7fb33b59e640 1 Processor -- start 2023-11-24T08:10:18.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 -- start start 2023-11-24T08:10:18.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 0x7fb33c13fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c144b30 0x7fb33c0c9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15c410 0x7fb33c142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1e97b0 con 0x7fb33c0b1de0 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1e8ba0 con 0x7fb33c144b30 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.592+0000 7fb33b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c1e8e80 con 0x7fb33c15c410 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 0x7fb33c13fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 0x7fb33c13fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42314/0 (socket says 172.21.15.106:42314) 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 -- 172.21.15.106:0/3517842713 learned_addr learned my addr 172.21.15.106:0/3517842713 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb339d9b640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c144b30 0x7fb33c0c9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 -- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15c410 msgr2=0x7fb33c142f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15c410 0x7fb33c142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 -- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c144b30 msgr2=0x7fb33c0c9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c144b30 0x7fb33c0c9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 -- 172.21.15.106:0/3517842713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3300094d0 con 0x7fb33c0b1de0 2023-11-24T08:10:18.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.593+0000 7fb33a59c640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 0x7fb33c13fb50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c430 tx=0x7fb32c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:18.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.594+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011790 con 0x7fb33c0b1de0 2023-11-24T08:10:18.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.594+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32c0118f0 con 0x7fb33c0b1de0 2023-11-24T08:10:18.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.594+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32c011ce0 con 0x7fb33c0b1de0 2023-11-24T08:10:18.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.594+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c1e9160 con 0x7fb33c0b1de0 2023-11-24T08:10:18.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.594+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c140090 con 0x7fb33c0b1de0 2023-11-24T08:10:18.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.595+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c1e9920 con 0x7fb33c0b1de0 2023-11-24T08:10:18.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.596+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb32c01c460 con 0x7fb33c0b1de0 2023-11-24T08:10:18.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.597+0000 7fb32affd640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31c06aed0 0x7fb31c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:18.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.597+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb32c089700 con 0x7fb33c0b1de0 2023-11-24T08:10:18.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.597+0000 7fb339d9b640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31c06aed0 0x7fb31c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:18.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.598+0000 7fb339d9b640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31c06aed0 0x7fb31c06d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb330036000 tx=0x7fb33003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:18.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.601+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb32c0546c0 con 0x7fb33c0b1de0 2023-11-24T08:10:18.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.829+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb33c04e840 con 0x7fb33c0b1de0 2023-11-24T08:10:18.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.830+0000 7fb32affd640 1 -- 172.21.15.106:0/3517842713 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4922 (secure 0 0 0) 0x7fb32c057ce0 con 0x7fb33c0b1de0 2023-11-24T08:10:18.832 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:18.832 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:09:56.761029+0000","modified":"2023-11-24T08:09:59.823336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6836/3854927347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3854927347},{"type":"v1","addr":"172.21.15.106:6836","nonce":3854927347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:10:18.832 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 15 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31c06aed0 msgr2=0x7fb31c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31c06aed0 0x7fb31c06d390 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb330036000 tx=0x7fb33003c040 comp rx=0 tx=0).stop 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 msgr2=0x7fb33c13fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 0x7fb33c13fb50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00c430 tx=0x7fb32c00c8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 shutdown_connections 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31c06aed0 0x7fb31c06d390 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb33c15c410 0x7fb33c142f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb33c144b30 0x7fb33c0c9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 --2- 172.21.15.106:0/3517842713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb33c0b1de0 0x7fb33c13fb50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 >> 172.21.15.106:0/3517842713 conn(0x7fb33c0cb830 msgr2=0x7fb33c15e380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 shutdown_connections 2023-11-24T08:10:18.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:18.833+0000 7fb33b59e640 1 -- 172.21.15.106:0/3517842713 wait complete. 2023-11-24T08:10:18.853 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.064+0000 7fd75f10f640 1 Processor -- start 2023-11-24T08:10:19.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.064+0000 7fd75f10f640 1 -- start start 2023-11-24T08:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75f10f640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd7580b2250 0x7fd75815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75f10f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580b2aa0 0x7fd75815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75f10f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd75815ab80 con 0x7fd7580b2630 2023-11-24T08:10:19.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75f10f640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75815a990 con 0x7fd7580b2250 2023-11-24T08:10:19.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75f10f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75815e680 con 0x7fd7580b2aa0 2023-11-24T08:10:19.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd757fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd7580b2630 0x7fd7580ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43850/0 (socket says 172.21.15.106:43850) 2023-11-24T08:10:19.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd757fff640 1 -- 172.21.15.106:0/3656064821 learned_addr learned my addr 172.21.15.106:0/3656064821 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:19.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75ceab640 1 --2- 172.21.15.106:0/3656064821 >> v2:172.21.15.156:3300/0 conn(0x7fd7580b2250 0x7fd75815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376316663 0 0) 0x7fd75815ab80 con 0x7fd7580b2630 2023-11-24T08:10:19.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd75d6ac640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580b2aa0 0x7fd75815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.066+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd748005730 con 0x7fd7580b2630 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 131848955 0 0) 0x7fd748005730 con 0x7fd7580b2630 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580b2aa0 msgr2=0x7fd75815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580b2aa0 0x7fd75815b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 >> v2:172.21.15.156:3300/0 conn(0x7fd7580b2250 msgr2=0x7fd75815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 --2- 172.21.15.106:0/3656064821 >> v2:172.21.15.156:3300/0 conn(0x7fd7580b2250 0x7fd75815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75815e960 con 0x7fd7580b2630 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fd744001670 con 0x7fd7580b2630 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd75d6ac640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580b2aa0 0x7fd75815b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:10:19.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 >> v1:172.21.15.106:6789/0 conn(0x7fd7580b2630 legacy=0x7fd7580ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd748005b80 0x7fd748005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd748006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd748007390 0x7fd74800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75815e680 con 0x7fd7480069d0 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74800e0d0 con 0x7fd748007390 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74800e290 con 0x7fd748005b80 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.067+0000 7fd757fff640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd748006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd75ceab640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd748005b80 0x7fd748005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd75d6ac640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd748007390 0x7fd74800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd757fff640 1 -- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd748005b80 msgr2=0x7fd748005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd757fff640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd748005b80 0x7fd748005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd757fff640 1 -- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd748007390 msgr2=0x7fd74800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd757fff640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd748007390 0x7fd74800db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.068+0000 7fd757fff640 1 -- 172.21.15.106:0/3656064821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd748005730 con 0x7fd7480069d0 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.069+0000 7fd757fff640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd748006e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd74400c110 tx=0x7fd748007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecf7a8533ad335a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.069+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744005ad0 con 0x7fd7480069d0 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.069+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744005c50 con 0x7fd7480069d0 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.069+0000 7fd7577fe640 1 -- 172.21.15.106:0/3656064821 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74402c830 con 0x7fd7480069d0 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75f10f640 1 -- 172.21.15.106:0/3656064821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 msgr2=0x7fd748006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75f10f640 1 --2- 172.21.15.106:0/3656064821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd748006e50 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd74400c110 tx=0x7fd748007a00 comp rx=0 tx=0).stop 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75d6ac640 1 -- 172.21.15.106:0/3656064821 reap_dead start 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75f10f640 1 -- 172.21.15.106:0/3656064821 shutdown_connections 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75f10f640 1 -- 172.21.15.106:0/3656064821 >> 172.21.15.106:0/3656064821 conn(0x7fd75806a920 msgr2=0x7fd75806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75f10f640 1 -- 172.21.15.106:0/3656064821 shutdown_connections 2023-11-24T08:10:19.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.070+0000 7fd75f10f640 1 -- 172.21.15.106:0/3656064821 wait complete. 2023-11-24T08:10:19.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.072+0000 7fd75f10f640 1 Processor -- start 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.072+0000 7fd75f10f640 1 -- start start 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75f10f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd7580acad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75f10f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd758152750 0x7fd7580ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75f10f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580ad550 0x7fd7580b02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75f10f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758157f20 con 0x7fd7480069d0 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75f10f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758152fd0 con 0x7fd758152750 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75f10f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd758157960 con 0x7fd7580ad550 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd7580acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd7580acad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42336/0 (socket says 172.21.15.106:42336) 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 -- 172.21.15.106:0/1510865666 learned_addr learned my addr 172.21.15.106:0/1510865666 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 -- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580ad550 msgr2=0x7fd7580b02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580ad550 0x7fd7580b02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 -- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd758152750 msgr2=0x7fd7580ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd757fff640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd758152750 0x7fd7580ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd758152750 0x7fd7580ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.073+0000 7fd75ceab640 1 -- 172.21.15.106:0/1510865666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7580cd7c0 con 0x7fd7480069d0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.074+0000 7fd75ceab640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd7580acad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd74c010890 tx=0x7fd74c010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.074+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c009690 con 0x7fd7480069d0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.074+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd74c00eb60 con 0x7fd7480069d0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.074+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74c01c400 con 0x7fd7480069d0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.074+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758157c40 con 0x7fd7480069d0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.074+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7580b1880 con 0x7fd7480069d0 2023-11-24T08:10:19.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.075+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd758073f60 con 0x7fd7480069d0 2023-11-24T08:10:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.076+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fd74c01c560 con 0x7fd7480069d0 2023-11-24T08:10:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.077+0000 7fd7557fa640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd73c06af00 0x7fd73c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.077+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd74c089bb0 con 0x7fd7480069d0 2023-11-24T08:10:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.077+0000 7fd757fff640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd73c06af00 0x7fd73c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.078+0000 7fd757fff640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd73c06af00 0x7fd73c06d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd74400c110 tx=0x7fd744007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.082+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd74c054cb0 con 0x7fd7480069d0 2023-11-24T08:10:19.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.271+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd758067470 con 0x7fd7480069d0 2023-11-24T08:10:19.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.273+0000 7fd7557fa640 1 -- 172.21.15.106:0/1510865666 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fd74c0582d0 con 0x7fd7480069d0 2023-11-24T08:10:19.275 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:19.275 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:58.320664+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:19.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd73c06af00 msgr2=0x7fd73c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd73c06af00 0x7fd73c06d3c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd74400c110 tx=0x7fd744007000 comp rx=0 tx=0).stop 2023-11-24T08:10:19.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 msgr2=0x7fd7580acad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd7580acad0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd74c010890 tx=0x7fd74c010d50 comp rx=0 tx=0).stop 2023-11-24T08:10:19.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 shutdown_connections 2023-11-24T08:10:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd73c06af00 0x7fd73c06d3c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd7580ad550 0x7fd7580b02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd758152750 0x7fd7580ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 --2- 172.21.15.106:0/1510865666 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd7480069d0 0x7fd7580acad0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.275+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 >> 172.21.15.106:0/1510865666 conn(0x7fd75806a920 msgr2=0x7fd758159320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:19.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.276+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 shutdown_connections 2023-11-24T08:10:19.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.276+0000 7fd75f10f640 1 -- 172.21.15.106:0/1510865666 wait complete. 2023-11-24T08:10:19.297 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:10:19.297 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:19.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.507+0000 7f2c63abb640 1 Processor -- start 2023-11-24T08:10:19.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.508+0000 7f2c63abb640 1 -- start start 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.509+0000 7f2c63abb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2c5c0cf450 0x7f2c5c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.509+0000 7f2c63abb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c5c0cdfb0 0x7f2c5c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.509+0000 7f2c63abb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c5c0cece0 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.509+0000 7f2c63abb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c0cfe40 con 0x7f2c5c0cdfb0 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.509+0000 7f2c63abb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5c156010 con 0x7f2c5c0cf450 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.509+0000 7f2c62058640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c5c0cdfb0 0x7f2c5c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c62058640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c5c0cdfb0 0x7f2c5c0ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42348/0 (socket says 172.21.15.106:42348) 2023-11-24T08:10:19.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c62058640 1 -- 172.21.15.106:0/1944426980 learned_addr learned my addr 172.21.15.106:0/1944426980 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:19.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c62058640 1 -- 172.21.15.106:0/1944426980 >> v2:172.21.15.156:3300/0 conn(0x7f2c5c0cf450 msgr2=0x7f2c5c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c61056640 1 --2- 172.21.15.106:0/1944426980 >> v2:172.21.15.156:3300/0 conn(0x7f2c5c0cf450 0x7f2c5c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c62058640 1 --2- 172.21.15.106:0/1944426980 >> v2:172.21.15.156:3300/0 conn(0x7f2c5c0cf450 0x7f2c5c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c62058640 1 -- 172.21.15.106:0/1944426980 >> v1:172.21.15.106:6789/0 conn(0x7f2c5c15c7f0 legacy=0x7f2c5c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c62058640 1 -- 172.21.15.106:0/1944426980 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c06d040 con 0x7f2c5c0cdfb0 2023-11-24T08:10:19.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.510+0000 7f2c61056640 1 --2- 172.21.15.106:0/1944426980 >> v2:172.21.15.156:3300/0 conn(0x7f2c5c0cf450 0x7f2c5c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.511+0000 7f2c62058640 1 --2- 172.21.15.106:0/1944426980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c0cdfb0 0x7f2c5c0ce3e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c070410 tx=0x7f2c5402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1935ad44412e7ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.511+0000 7f2c60855640 1 -- 172.21.15.106:0/1944426980 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5402cb10 con 0x7f2c5c0cdfb0 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c60855640 1 -- 172.21.15.106:0/1944426980 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5402cc70 con 0x7f2c5c0cdfb0 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c60855640 1 -- 172.21.15.106:0/1944426980 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c540343f0 con 0x7f2c5c0cdfb0 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 -- 172.21.15.106:0/1944426980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c0cdfb0 msgr2=0x7f2c5c0ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 --2- 172.21.15.106:0/1944426980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c0cdfb0 0x7f2c5c0ce3e0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c070410 tx=0x7f2c5402bf70 comp rx=0 tx=0).stop 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 -- 172.21.15.106:0/1944426980 shutdown_connections 2023-11-24T08:10:19.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 --2- 172.21.15.106:0/1944426980 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c0cdfb0 0x7f2c5c0ce3e0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 --2- 172.21.15.106:0/1944426980 >> v2:172.21.15.156:3300/0 conn(0x7f2c5c0cf450 0x7f2c5c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 -- 172.21.15.106:0/1944426980 >> 172.21.15.106:0/1944426980 conn(0x7f2c5c06a920 msgr2=0x7f2c5c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:19.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 -- 172.21.15.106:0/1944426980 shutdown_connections 2023-11-24T08:10:19.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.512+0000 7f2c63abb640 1 -- 172.21.15.106:0/1944426980 wait complete. 2023-11-24T08:10:19.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.514+0000 7f2c63abb640 1 Processor -- start 2023-11-24T08:10:19.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.514+0000 7f2c63abb640 1 -- start start 2023-11-24T08:10:19.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.515+0000 7f2c63abb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5c0cdfb0 0x7f2c5c156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.515+0000 7f2c63abb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5c0cf450 0x7f2c5c0ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.515+0000 7f2c63abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 0x7f2c5c0ad700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.515+0000 7f2c63abb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c15eb20 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.515+0000 7f2c63abb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c158f60 con 0x7f2c5c0cf450 2023-11-24T08:10:19.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.515+0000 7f2c63abb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5c15e560 con 0x7f2c5c0cdfb0 2023-11-24T08:10:19.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 0x7f2c5c0ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 0x7f2c5c0ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42352/0 (socket says 172.21.15.106:42352) 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 -- 172.21.15.106:0/325477372 learned_addr learned my addr 172.21.15.106:0/325477372 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c61056640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5c0cf450 0x7f2c5c0ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c61857640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5c0cdfb0 0x7f2c5c156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 -- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5c0cdfb0 msgr2=0x7f2c5c156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5c0cdfb0 0x7f2c5c156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 -- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5c0cf450 msgr2=0x7f2c5c0ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5c0cf450 0x7f2c5c0ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c62058640 1 -- 172.21.15.106:0/325477372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c54009b30 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.516+0000 7f2c61056640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5c0cf450 0x7f2c5c0ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.517+0000 7f2c62058640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 0x7f2c5c0ad700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2c54036040 tx=0x7f2c54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.517+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c54009510 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.517+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5403e450 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.517+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5c15e840 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.517+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5403e800 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.517+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5c158a40 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.518+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5c1585c0 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.519+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2c5403e960 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.520+0000 7f2c5a7fc640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c4806aed0 0x7f2c4806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.520+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2c540ac510 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.521+0000 7f2c61857640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c4806aed0 0x7f2c4806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.521+0000 7f2c61857640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c4806aed0 0x7f2c4806d390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000f60 tx=0x7f2c4c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.525+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c5c1585c0 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.751+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c5c1585c0 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.752+0000 7f2c5a7fc640 1 -- 172.21.15.106:0/325477372 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f2c5c1585c0 con 0x7f2c5c15c7f0 2023-11-24T08:10:19.754 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:19.754 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:09:56.761029+0000","modified":"2023-11-24T08:09:59.823336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6836/3854927347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3854927347},{"type":"v1","addr":"172.21.15.106:6836","nonce":3854927347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:10:19.754 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 15 2023-11-24T08:10:19.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.754+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c4806aed0 msgr2=0x7f2c4806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.754+0000 7f2c63abb640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c4806aed0 0x7f2c4806d390 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c000f60 tx=0x7f2c4c009040 comp rx=0 tx=0).stop 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.754+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 msgr2=0x7f2c5c0ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.754+0000 7f2c63abb640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 0x7f2c5c0ad700 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f2c54036040 tx=0x7f2c54004a40 comp rx=0 tx=0).stop 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 shutdown_connections 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c5c15c7f0 0x7f2c5c0ad700 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5c0cf450 0x7f2c5c0ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c4806aed0 0x7f2c4806d390 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 --2- 172.21.15.106:0/325477372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5c0cdfb0 0x7f2c5c156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 >> 172.21.15.106:0/325477372 conn(0x7f2c5c06a920 msgr2=0x7f2c5c1554b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 shutdown_connections 2023-11-24T08:10:19.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.755+0000 7f2c63abb640 1 -- 172.21.15.106:0/325477372 wait complete. 2023-11-24T08:10:19.775 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:19.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.980+0000 7f9a96abb640 1 Processor -- start 2023-11-24T08:10:19.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.981+0000 7f9a96abb640 1 -- start start 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.982+0000 7f9a96abb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a9015d4e0 0x7f9a900d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.982+0000 7f9a96abb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a900d0240 0x7f9a900b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.982+0000 7f9a96abb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a900ce990 con 0x7f9a900d38e0 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.982+0000 7f9a96abb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a900d4110 con 0x7f9a900d0240 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.982+0000 7f9a96abb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900cec70 con 0x7f9a9015d4e0 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.982+0000 7f9a962ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a900d0240 0x7f9a900b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.983+0000 7f9a962ba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a900d0240 0x7f9a900b2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42356/0 (socket says 172.21.15.106:42356) 2023-11-24T08:10:19.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.983+0000 7f9a962ba640 1 -- 172.21.15.106:0/2647899457 learned_addr learned my addr 172.21.15.106:0/2647899457 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:19.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.983+0000 7f9a962ba640 1 -- 172.21.15.106:0/2647899457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a9015d4e0 msgr2=0x7f9a900d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.983+0000 7f9a962ba640 1 --2- 172.21.15.106:0/2647899457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a9015d4e0 0x7f9a900d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.983+0000 7f9a962ba640 1 -- 172.21.15.106:0/2647899457 >> v1:172.21.15.156:6789/0 conn(0x7f9a900d38e0 legacy=0x7f9a900cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:19.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.983+0000 7f9a962ba640 1 -- 172.21.15.106:0/2647899457 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a900cef50 con 0x7f9a900d0240 2023-11-24T08:10:19.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.984+0000 7f9a962ba640 1 --2- 172.21.15.106:0/2647899457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a900b2e50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9a900b2a00 tx=0x7f9a8802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e4c9ee8778149c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a94ab7640 1 -- 172.21.15.106:0/2647899457 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8802ca50 con 0x7f9a900d0240 2023-11-24T08:10:19.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a94ab7640 1 -- 172.21.15.106:0/2647899457 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8802cbb0 con 0x7f9a900d0240 2023-11-24T08:10:19.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a94ab7640 1 -- 172.21.15.106:0/2647899457 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a880343f0 con 0x7f9a900d0240 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 -- 172.21.15.106:0/2647899457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 msgr2=0x7f9a900b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 --2- 172.21.15.106:0/2647899457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a900b2e50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9a900b2a00 tx=0x7f9a8802beb0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 -- 172.21.15.106:0/2647899457 shutdown_connections 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 --2- 172.21.15.106:0/2647899457 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a900b2e50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 --2- 172.21.15.106:0/2647899457 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a9015d4e0 0x7f9a900d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 -- 172.21.15.106:0/2647899457 >> 172.21.15.106:0/2647899457 conn(0x7f9a9009c510 msgr2=0x7f9a900d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:19.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 -- 172.21.15.106:0/2647899457 shutdown_connections 2023-11-24T08:10:19.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.985+0000 7f9a96abb640 1 -- 172.21.15.106:0/2647899457 wait complete. 2023-11-24T08:10:19.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.987+0000 7f9a96abb640 1 Processor -- start 2023-11-24T08:10:19.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.988+0000 7f9a96abb640 1 -- start start 2023-11-24T08:10:19.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a96abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a9015b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a96abb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a900d38e0 0x7f9a900cb840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a96abb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a9015d4e0 0x7f9a9015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a96abb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a90160330 con 0x7f9a900d0240 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a96abb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a900cb420 con 0x7f9a9015d4e0 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a96abb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9015b270 con 0x7f9a900d38e0 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a95ab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a9015b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a95ab9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a9015b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42362/0 (socket says 172.21.15.106:42362) 2023-11-24T08:10:19.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.989+0000 7f9a95ab9640 1 -- 172.21.15.106:0/26754862 learned_addr learned my addr 172.21.15.106:0/26754862 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:19.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a95ab9640 1 -- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a900d38e0 msgr2=0x7f9a900cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a962ba640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a9015d4e0 0x7f9a9015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a952b8640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a900d38e0 0x7f9a900cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a95ab9640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a900d38e0 0x7f9a900cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a95ab9640 1 -- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a9015d4e0 msgr2=0x7f9a9015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a95ab9640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a9015d4e0 0x7f9a9015ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a95ab9640 1 -- 172.21.15.106:0/26754862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a88009b30 con 0x7f9a900d0240 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a95ab9640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a9015b460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b440 tx=0x7f9a8c00b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c011890 con 0x7f9a900d0240 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c0119f0 con 0x7f9a900d0240 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c011da0 con 0x7f9a900d0240 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a900ca7a0 con 0x7f9a900d0240 2023-11-24T08:10:19.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.990+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a900cad60 con 0x7f9a900d0240 2023-11-24T08:10:19.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.991+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a900cb690 con 0x7f9a900d0240 2023-11-24T08:10:19.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.992+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9a8c01a400 con 0x7f9a900d0240 2023-11-24T08:10:19.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.994+0000 7f9a827fc640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a7406aed0 0x7f9a7406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:19.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.994+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a8c089a60 con 0x7f9a900d0240 2023-11-24T08:10:19.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.994+0000 7f9a952b8640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a7406aed0 0x7f9a7406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:19.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:19.995+0000 7f9a952b8640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a7406aed0 0x7f9a7406d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a900b5560 tx=0x7f9a84009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.000+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a8c054ad0 con 0x7f9a900d0240 2023-11-24T08:10:20.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.226+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a9004e840 con 0x7f9a900d0240 2023-11-24T08:10:20.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.228+0000 7f9a827fc640 1 -- 172.21.15.106:0/26754862 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4922 (secure 0 0 0) 0x7f9a8c015020 con 0x7f9a900d0240 2023-11-24T08:10:20.230 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:20.230 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:09:56.761029+0000","modified":"2023-11-24T08:09:59.823336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5082},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5082":{"gid":5082,"name":"c","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.106:6836/3854927347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3854927347},{"type":"v1","addr":"172.21.15.106:6836","nonce":3854927347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-24T08:10:20.230 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 15 2023-11-24T08:10:20.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a7406aed0 msgr2=0x7f9a7406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a7406aed0 0x7f9a7406d390 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9a900b5560 tx=0x7f9a84009040 comp rx=0 tx=0).stop 2023-11-24T08:10:20.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 msgr2=0x7f9a9015b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a9015b460 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c00b440 tx=0x7f9a8c00b900 comp rx=0 tx=0).stop 2023-11-24T08:10:20.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 shutdown_connections 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a9015d4e0 0x7f9a9015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a7406aed0 0x7f9a7406d390 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a900d38e0 0x7f9a900cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 --2- 172.21.15.106:0/26754862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a900d0240 0x7f9a9015b460 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.230+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 >> 172.21.15.106:0/26754862 conn(0x7f9a9009c510 msgr2=0x7f9a900b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.231+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 shutdown_connections 2023-11-24T08:10:20.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.231+0000 7f9a96abb640 1 -- 172.21.15.106:0/26754862 wait complete. 2023-11-24T08:10:20.252 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:20.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.460+0000 7fe3460bb640 1 Processor -- start 2023-11-24T08:10:20.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.460+0000 7fe3460bb640 1 -- start start 2023-11-24T08:10:20.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe3460bb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe34015b2b0 0x7fe340161c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe3460bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 0x7fe34006a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe3460bb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400d0000 con 0x7fe34006f290 2023-11-24T08:10:20.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe3460bb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe34006aeb0 con 0x7fe34015b2b0 2023-11-24T08:10:20.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe3460bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015ec80 con 0x7fe34006a500 2023-11-24T08:10:20.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe33effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe34006f290 0x7fe34015dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43864/0 (socket says 172.21.15.106:43864) 2023-11-24T08:10:20.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.462+0000 7fe33effd640 1 -- 172.21.15.106:0/636228630 learned_addr learned my addr 172.21.15.106:0/636228630 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:20.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33ffff640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 0x7fe34006a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583585319 0 0) 0x7fe3400d0000 con 0x7fe34006f290 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/636228630 >> v2:172.21.15.156:3300/0 conn(0x7fe34015b2b0 0x7fe340161c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe328003520 con 0x7fe34006f290 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2367163183 0 0) 0x7fe328003520 con 0x7fe34006f290 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 msgr2=0x7fe34006a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 0x7fe34006a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 >> v2:172.21.15.156:3300/0 conn(0x7fe34015b2b0 msgr2=0x7fe340161c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 --2- 172.21.15.106:0/636228630 >> v2:172.21.15.156:3300/0 conn(0x7fe34015b2b0 0x7fe340161c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34015e3f0 con 0x7fe34006f290 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fe334001670 con 0x7fe34006f290 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 >> v1:172.21.15.106:6789/0 conn(0x7fe34006f290 legacy=0x7fe34015dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe328003970 0x7fe328003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 0x7fe3280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328004d00 0x7fe328005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34006aeb0 con 0x7fe328003970 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015ec80 con 0x7fe328004d00 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32800b9b0 con 0x7fe34006a500 2023-11-24T08:10:20.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.463+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe328003970 0x7fe328003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.464+0000 7fe33f7fe640 1 -- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 msgr2=0x7fe3280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.464+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe34006a500 0x7fe3280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.464+0000 7fe33f7fe640 1 -- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328004d00 msgr2=0x7fe328005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.464+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328004d00 0x7fe328005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.464+0000 7fe33f7fe640 1 -- 172.21.15.106:0/636228630 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe328003520 con 0x7fe328003970 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.465+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe328003970 0x7fe328003d90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe330009ac0 tx=0x7fe3280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d7d9bc62d22a30 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.465+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33002c710 con 0x7fe328003970 2023-11-24T08:10:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.465+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33002c870 con 0x7fe328003970 2023-11-24T08:10:20.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.465+0000 7fe33e7fc640 1 -- 172.21.15.106:0/636228630 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33002cc00 con 0x7fe328003970 2023-11-24T08:10:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe3460bb640 1 -- 172.21.15.106:0/636228630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe328003970 msgr2=0x7fe328003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe3460bb640 1 --2- 172.21.15.106:0/636228630 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe328003970 0x7fe328003d90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fe330009ac0 tx=0x7fe3280057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe33ffff640 1 -- 172.21.15.106:0/636228630 reap_dead start 2023-11-24T08:10:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe3460bb640 1 -- 172.21.15.106:0/636228630 shutdown_connections 2023-11-24T08:10:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe3460bb640 1 -- 172.21.15.106:0/636228630 >> 172.21.15.106:0/636228630 conn(0x7fe3400d01b0 msgr2=0x7fe3400d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe3460bb640 1 -- 172.21.15.106:0/636228630 shutdown_connections 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.466+0000 7fe3460bb640 1 -- 172.21.15.106:0/636228630 wait complete. 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.468+0000 7fe3460bb640 1 Processor -- start 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.468+0000 7fe3460bb640 1 -- start start 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe3460bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328003970 0x7fe34015a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe3460bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 0x7fe3400af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe3460bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b3f80 0x7fe3400b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe3460bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c6840 con 0x7fe3400aee10 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe3460bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400af7d0 con 0x7fe328003970 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe3460bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400b5270 con 0x7fe3400b3f80 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 0x7fe3400af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 0x7fe3400af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42382/0 (socket says 172.21.15.106:42382) 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 -- 172.21.15.106:0/2968638606 learned_addr learned my addr 172.21.15.106:0/2968638606 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 -- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b3f80 msgr2=0x7fe3400b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33ffff640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b3f80 0x7fe3400b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328003970 0x7fe34015a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b3f80 0x7fe3400b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 -- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328003970 msgr2=0x7fe34015a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328003970 0x7fe34015a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33effd640 1 -- 172.21.15.106:0/2968638606 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400c4180 con 0x7fe3400aee10 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.469+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328003970 0x7fe34015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.470+0000 7fe33effd640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 0x7fe3400af290 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c730 tx=0x7fe33400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.470+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3340074e0 con 0x7fe3400aee10 2023-11-24T08:10:20.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.470+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe334007640 con 0x7fe3400aee10 2023-11-24T08:10:20.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.470+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe334004a00 con 0x7fe3400aee10 2023-11-24T08:10:20.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.470+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3400b4cb0 con 0x7fe3400aee10 2023-11-24T08:10:20.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.470+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3400c3e10 con 0x7fe3400aee10 2023-11-24T08:10:20.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.471+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3400c4310 con 0x7fe3400aee10 2023-11-24T08:10:20.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.472+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe334004b60 con 0x7fe3400aee10 2023-11-24T08:10:20.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.473+0000 7fe327fff640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32c06af00 0x7fe32c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.473+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe334088ac0 con 0x7fe3400aee10 2023-11-24T08:10:20.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.473+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32c06af00 0x7fe32c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.474+0000 7fe33f7fe640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32c06af00 0x7fe32c06d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe330036000 tx=0x7fe330030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.476+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3340575a0 con 0x7fe3400aee10 2023-11-24T08:10:20.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.665+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe340067b80 con 0x7fe3400aee10 2023-11-24T08:10:20.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.668+0000 7fe327fff640 1 -- 172.21.15.106:0/2968638606 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+16617 (secure 0 0 0) 0x7fe334018020 con 0x7fe3400aee10 2023-11-24T08:10:20.671 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:20.671 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:09:58.320664+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:09:54.590990+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-11-24T08:09:55.792078+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"36","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:20.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32c06af00 msgr2=0x7fe32c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32c06af00 0x7fe32c06d3c0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe330036000 tx=0x7fe330030000 comp rx=0 tx=0).stop 2023-11-24T08:10:20.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 msgr2=0x7fe3400af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 0x7fe3400af290 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c730 tx=0x7fe33400cbf0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 shutdown_connections 2023-11-24T08:10:20.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32c06af00 0x7fe32c06d3c0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b3f80 0x7fe3400b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400aee10 0x7fe3400af290 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 --2- 172.21.15.106:0/2968638606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe328003970 0x7fe34015a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.670+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 >> 172.21.15.106:0/2968638606 conn(0x7fe3400d01b0 msgr2=0x7fe3400ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:20.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.671+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 shutdown_connections 2023-11-24T08:10:20.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.671+0000 7fe3460bb640 1 -- 172.21.15.106:0/2968638606 wait complete. 2023-11-24T08:10:20.692 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:10:20.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.899+0000 7f1eb5d27640 1 Processor -- start 2023-11-24T08:10:20.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.899+0000 7f1eb5d27640 1 -- start start 2023-11-24T08:10:20.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.900+0000 7f1eb5d27640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1eb00cb690 0x7f1eb00b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eb5d27640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb00ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eb5d27640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb006c530 con 0x7f1eb0160890 2023-11-24T08:10:20.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eb5d27640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1eb006d170 con 0x7f1eb00cb690 2023-11-24T08:10:20.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eb5d27640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb00b1510 con 0x7f1eb00b1080 2023-11-24T08:10:20.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eaffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1eb00cb690 0x7f1eb00b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eaffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1eb00cb690 0x7f1eb00b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42392/0 (socket says 172.21.15.106:42392) 2023-11-24T08:10:20.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eaffff640 1 -- 172.21.15.106:0/3587108945 learned_addr learned my addr 172.21.15.106:0/3587108945 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:20.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.901+0000 7f1eb5526640 1 --2- 172.21.15.106:0/3587108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb00ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.902+0000 7f1eaffff640 1 -- 172.21.15.106:0/3587108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 msgr2=0x7f1eb00ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:20.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.902+0000 7f1eaffff640 1 --2- 172.21.15.106:0/3587108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb00ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.902+0000 7f1eaffff640 1 -- 172.21.15.106:0/3587108945 >> v1:172.21.15.156:6789/0 conn(0x7f1eb0160890 legacy=0x7f1eb00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.902+0000 7f1eaffff640 1 -- 172.21.15.106:0/3587108945 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb006c7d0 con 0x7f1eb00cb690 2023-11-24T08:10:20.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.903+0000 7f1eaffff640 1 --2- 172.21.15.106:0/3587108945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b0b40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009870 tx=0x7f1ea002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78bd93a1695a1547 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.903+0000 7f1eaf7fe640 1 -- 172.21.15.106:0/3587108945 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0030840 con 0x7f1eb00cb690 2023-11-24T08:10:20.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.903+0000 7f1eaf7fe640 1 -- 172.21.15.106:0/3587108945 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea00309a0 con 0x7f1eb00cb690 2023-11-24T08:10:20.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.903+0000 7f1eaf7fe640 1 -- 172.21.15.106:0/3587108945 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea0030d30 con 0x7f1eb00cb690 2023-11-24T08:10:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 -- 172.21.15.106:0/3587108945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb00cb690 msgr2=0x7f1eb00b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/3587108945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b0b40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009870 tx=0x7f1ea002b980 comp rx=0 tx=0).stop 2023-11-24T08:10:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 -- 172.21.15.106:0/3587108945 shutdown_connections 2023-11-24T08:10:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/3587108945 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb00ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/3587108945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b0b40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 -- 172.21.15.106:0/3587108945 >> 172.21.15.106:0/3587108945 conn(0x7f1eb00cbb40 msgr2=0x7f1eb00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 -- 172.21.15.106:0/3587108945 shutdown_connections 2023-11-24T08:10:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.904+0000 7f1eb5d27640 1 -- 172.21.15.106:0/3587108945 wait complete. 2023-11-24T08:10:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.906+0000 7f1eb5d27640 1 Processor -- start 2023-11-24T08:10:20.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.906+0000 7f1eb5d27640 1 -- start start 2023-11-24T08:10:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5d27640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb009ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5d27640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5d27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 0x7f1eb0152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5d27640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb009ca60 con 0x7f1eb0160890 2023-11-24T08:10:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5d27640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb009c4a0 con 0x7f1eb00cb690 2023-11-24T08:10:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5d27640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb009c780 con 0x7f1eb00b1080 2023-11-24T08:10:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 0x7f1eb0152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 0x7f1eb0152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42408/0 (socket says 172.21.15.106:42408) 2023-11-24T08:10:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 -- 172.21.15.106:0/2245678565 learned_addr learned my addr 172.21.15.106:0/2245678565 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eaffff640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb4d25640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb009ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 -- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 msgr2=0x7f1eb009ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb009ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 -- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb00cb690 msgr2=0x7f1eb00b3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb5526640 1 -- 172.21.15.106:0/2245678565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00094d0 con 0x7f1eb0160890 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eb4d25640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb009ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.907+0000 7f1eaffff640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.908+0000 7f1eb5526640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 0x7f1eb0152160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800efc0 tx=0x7f1ea800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.908+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8009ba0 con 0x7f1eb0160890 2023-11-24T08:10:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.908+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea800f040 con 0x7f1eb0160890 2023-11-24T08:10:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.908+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea80146c0 con 0x7f1eb0160890 2023-11-24T08:10:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.908+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb009a480 con 0x7f1eb0160890 2023-11-24T08:10:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.908+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb009b1c0 con 0x7f1eb0160890 2023-11-24T08:10:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.909+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb009a9a0 con 0x7f1eb0160890 2023-11-24T08:10:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.910+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1ea8014820 con 0x7f1eb0160890 2023-11-24T08:10:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.911+0000 7f1ead7fa640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1e9c06aed0 0x7f1e9c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.911+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ea808a210 con 0x7f1eb0160890 2023-11-24T08:10:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.912+0000 7f1eb4d25640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1e9c06aed0 0x7f1e9c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.912+0000 7f1eb4d25640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1e9c06aed0 0x7f1e9c06d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4000f60 tx=0x7f1ea4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:20.916+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1eb009a9a0 con 0x7f1eb0160890 2023-11-24T08:10:21.147 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.144+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1eb009a9a0 con 0x7f1eb0160890 2023-11-24T08:10:21.148 INFO:tasks.ceph.mon.a.smithi106.stderr:2023-11-24T08:10:21.145+0000 7fb1127dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:10:21.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.149+0000 7f1ead7fa640 1 -- 172.21.15.106:0/2245678565 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1eb009a9a0 con 0x7f1eb0160890 2023-11-24T08:10:21.152 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:10:21.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.152+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1e9c06aed0 msgr2=0x7f1e9c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:21.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.152+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1e9c06aed0 0x7f1e9c06d390 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1ea4000f60 tx=0x7f1ea4009040 comp rx=0 tx=0).stop 2023-11-24T08:10:21.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.152+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 msgr2=0x7f1eb0152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:21.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.152+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 0x7f1eb0152160 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f1ea800efc0 tx=0x7f1ea800c490 comp rx=0 tx=0).stop 2023-11-24T08:10:21.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.153+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 shutdown_connections 2023-11-24T08:10:21.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.153+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1eb0160890 0x7f1eb0152160 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.153+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1eb00cb690 0x7f1eb00b3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.153+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1e9c06aed0 0x7f1e9c06d390 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.153+0000 7f1eb5d27640 1 --2- 172.21.15.106:0/2245678565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1eb00b1080 0x7f1eb009ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.153+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 >> 172.21.15.106:0/2245678565 conn(0x7f1eb00cbb40 msgr2=0x7f1eb0071700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:21.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.154+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 shutdown_connections 2023-11-24T08:10:21.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.154+0000 7f1eb5d27640 1 -- 172.21.15.106:0/2245678565 wait complete. 2023-11-24T08:10:21.177 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:10:21.190 INFO:tasks.ceph.mds.c.smithi106.stderr: -4370> 2023-11-24T08:09:58.535+0000 7f420ede4640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:10:21.274 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:10:21.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.391+0000 7f6dc7fff640 1 Processor -- start 2023-11-24T08:10:21.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.391+0000 7f6dc7fff640 1 -- start start 2023-11-24T08:10:21.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.392+0000 7f6dc7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6dc815e620 0x7f6dc80ae130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:21.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.392+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80ae670 0x7f6dc80b1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:21.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.392+0000 7f6dc7fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc8072960 con 0x7f6dc80b0b50 2023-11-24T08:10:21.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.392+0000 7f6dc7fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc815dc10 con 0x7f6dc815e620 2023-11-24T08:10:21.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.392+0000 7f6dc7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc80cb0b0 con 0x7f6dc80ae670 2023-11-24T08:10:21.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dc6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6dc80b0b50 0x7f6dc80b0f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:43880/0 (socket says 172.21.15.106:43880) 2023-11-24T08:10:21.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dc6ffd640 1 -- 172.21.15.106:0/226516996 learned_addr learned my addr 172.21.15.106:0/226516996 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:21.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dbe7fc640 1 --2- 172.21.15.106:0/226516996 >> v2:172.21.15.106:3300/0 conn(0x7f6dc815e620 0x7f6dc80ae130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dc77fe640 1 --2- 172.21.15.106:0/226516996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80ae670 0x7f6dc80b1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dc67fc640 1 -- 172.21.15.106:0/226516996 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842853906 0 0) 0x7f6dc8072960 con 0x7f6dc80b0b50 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dbe7fc640 1 -- 172.21.15.106:0/226516996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80ae670 msgr2=0x7f6dc80b1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dbe7fc640 1 --2- 172.21.15.106:0/226516996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80ae670 0x7f6dc80b1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dbe7fc640 1 -- 172.21.15.106:0/226516996 >> v1:172.21.15.106:6789/0 conn(0x7f6dc80b0b50 legacy=0x7f6dc80b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.393+0000 7f6dbe7fc640 1 -- 172.21.15.106:0/226516996 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc80cb390 con 0x7f6dc815e620 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.394+0000 7f6dbe7fc640 1 --2- 172.21.15.106:0/226516996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc80ae130 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6db00097a0 tx=0x7f6db002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90640265ea254323 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:21.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc67fc640 1 -- 172.21.15.106:0/226516996 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0031830 con 0x7f6dc815e620 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc67fc640 1 -- 172.21.15.106:0/226516996 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db0031990 con 0x7f6dc815e620 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc67fc640 1 -- 172.21.15.106:0/226516996 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db0031d20 con 0x7f6dc815e620 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 -- 172.21.15.106:0/226516996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 msgr2=0x7f6dc80ae130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/226516996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc80ae130 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f6db00097a0 tx=0x7f6db002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 -- 172.21.15.106:0/226516996 shutdown_connections 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/226516996 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80ae670 0x7f6dc80b1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/226516996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc80ae130 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 -- 172.21.15.106:0/226516996 >> 172.21.15.106:0/226516996 conn(0x7f6dc806a500 msgr2=0x7f6dc806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 -- 172.21.15.106:0/226516996 shutdown_connections 2023-11-24T08:10:21.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.395+0000 7f6dc7fff640 1 -- 172.21.15.106:0/226516996 wait complete. 2023-11-24T08:10:21.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.397+0000 7f6dc7fff640 1 Processor -- start 2023-11-24T08:10:21.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.397+0000 7f6dc7fff640 1 -- start start 2023-11-24T08:10:21.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.398+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc80ae670 0x7f6dc815af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.398+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80b0b50 0x7f6dc814ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.398+0000 7f6dc7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc81f3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.398+0000 7f6dc7fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc8155fa0 con 0x7f6dc815e620 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.398+0000 7f6dc7fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc815b490 con 0x7f6dc80ae670 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.398+0000 7f6dc7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc81561f0 con 0x7f6dc80b0b50 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc81f3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc81f3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42440/0 (socket says 172.21.15.106:42440) 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 -- 172.21.15.106:0/454405652 learned_addr learned my addr 172.21.15.106:0/454405652 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:21.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc6ffd640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc80ae670 0x7f6dc815af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dbe7fc640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80b0b50 0x7f6dc814ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 -- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80b0b50 msgr2=0x7f6dc814ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80b0b50 0x7f6dc814ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 -- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc80ae670 msgr2=0x7f6dc815af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc80ae670 0x7f6dc815af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 -- 172.21.15.106:0/454405652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0009400 con 0x7f6dc815e620 2023-11-24T08:10:21.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.399+0000 7f6dc77fe640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc81f3000 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000ccb0 tx=0x7f6dc000a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:21.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.400+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc000ae80 con 0x7f6dc815e620 2023-11-24T08:10:21.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.400+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0003990 con 0x7f6dc815e620 2023-11-24T08:10:21.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.400+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0003d40 con 0x7f6dc815e620 2023-11-24T08:10:21.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.400+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc81564d0 con 0x7f6dc815e620 2023-11-24T08:10:21.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.400+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc8156a90 con 0x7f6dc815e620 2023-11-24T08:10:21.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.401+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc8156cb0 con 0x7f6dc815e620 2023-11-24T08:10:21.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.402+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6dc0003ea0 con 0x7f6dc815e620 2023-11-24T08:10:21.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.403+0000 7f6dbffff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6da806aed0 0x7f6da806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:21.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.403+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6dc0089c30 con 0x7f6dc815e620 2023-11-24T08:10:21.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.403+0000 7f6dc6ffd640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6da806aed0 0x7f6da806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:21.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.404+0000 7f6dc6ffd640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6da806aed0 0x7f6da806d390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6db8000f60 tx=0x7f6db8007b50 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:21.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.407+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6dc0054a80 con 0x7f6dc815e620 2023-11-24T08:10:21.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:21.636+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6dc804e840 con 0x7f6dc815e620 2023-11-24T08:10:22.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.153+0000 7f6dbffff640 1 -- 172.21.15.106:0/454405652 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6dc001d020 con 0x7f6dc815e620 2023-11-24T08:10:22.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6da806aed0 msgr2=0x7f6da806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:22.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6da806aed0 0x7f6da806d390 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f6db8000f60 tx=0x7f6db8007b50 comp rx=0 tx=0).stop 2023-11-24T08:10:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 msgr2=0x7f6dc81f3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc81f3000 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f6dc000ccb0 tx=0x7f6dc000a5c0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 shutdown_connections 2023-11-24T08:10:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6dc815e620 0x7f6dc81f3000 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6da806aed0 0x7f6da806d390 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6dc80b0b50 0x7f6dc814ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 --2- 172.21.15.106:0/454405652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6dc80ae670 0x7f6dc815af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.156+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 >> 172.21.15.106:0/454405652 conn(0x7f6dc806a500 msgr2=0x7f6dc80cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.157+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 shutdown_connections 2023-11-24T08:10:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.157+0000 7f6dc7fff640 1 -- 172.21.15.106:0/454405652 wait complete. 2023-11-24T08:10:22.180 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:10:22.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.382+0000 7f7933a56640 1 Processor -- start 2023-11-24T08:10:22.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.383+0000 7f7933a56640 1 -- start start 2023-11-24T08:10:22.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.384+0000 7f7933a56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c15c7f0 0x7f792c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:22.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.384+0000 7f7933a56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f792c0ce6c0 0x7f792c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:22.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.384+0000 7f7933a56640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f792c0cece0 con 0x7f792c0cf450 2023-11-24T08:10:22.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.384+0000 7f7933a56640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f792c156010 con 0x7f792c0ce6c0 2023-11-24T08:10:22.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.384+0000 7f7933a56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c06ce30 con 0x7f792c15c7f0 2023-11-24T08:10:22.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f792c0ce6c0 0x7f792c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:22.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f792c0ce6c0 0x7f792c0cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42452/0 (socket says 172.21.15.106:42452) 2023-11-24T08:10:22.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 -- 172.21.15.106:0/460704791 learned_addr learned my addr 172.21.15.106:0/460704791 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:22.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f79317f2640 1 --2- 172.21.15.106:0/460704791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c15c7f0 0x7f792c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:22.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 -- 172.21.15.106:0/460704791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c15c7f0 msgr2=0x7f792c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:22.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 --2- 172.21.15.106:0/460704791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c15c7f0 0x7f792c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 -- 172.21.15.106:0/460704791 >> v1:172.21.15.156:6789/0 conn(0x7f792c0cf450 legacy=0x7f792c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:22.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.385+0000 7f7931ff3640 1 -- 172.21.15.106:0/460704791 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f792c06d040 con 0x7f792c0ce6c0 2023-11-24T08:10:22.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.386+0000 7f7931ff3640 1 --2- 172.21.15.106:0/460704791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0ce6c0 0x7f792c0cf8d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7924009870 tx=0x7f792402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c21905a9e924b714 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.386+0000 7f792bfff640 1 -- 172.21.15.106:0/460704791 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924031640 con 0x7f792c0ce6c0 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f792bfff640 1 -- 172.21.15.106:0/460704791 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79240317a0 con 0x7f792c0ce6c0 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f792bfff640 1 -- 172.21.15.106:0/460704791 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7924031b30 con 0x7f792c0ce6c0 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 -- 172.21.15.106:0/460704791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0ce6c0 msgr2=0x7f792c0cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 --2- 172.21.15.106:0/460704791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0ce6c0 0x7f792c0cf8d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f7924009870 tx=0x7f792402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 -- 172.21.15.106:0/460704791 shutdown_connections 2023-11-24T08:10:22.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 --2- 172.21.15.106:0/460704791 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0ce6c0 0x7f792c0cf8d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 --2- 172.21.15.106:0/460704791 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c15c7f0 0x7f792c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 -- 172.21.15.106:0/460704791 >> 172.21.15.106:0/460704791 conn(0x7f792c06a920 msgr2=0x7f792c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:22.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 -- 172.21.15.106:0/460704791 shutdown_connections 2023-11-24T08:10:22.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.387+0000 7f7933a56640 1 -- 172.21.15.106:0/460704791 wait complete. 2023-11-24T08:10:22.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.389+0000 7f7933a56640 1 Processor -- start 2023-11-24T08:10:22.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.389+0000 7f7933a56640 1 -- start start 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7933a56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c0ce6c0 0x7f792c154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7933a56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 0x7f792c0c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7933a56640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f792c15c7f0 0x7f792c0c9de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7933a56640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0c8d40 con 0x7f792c0cf450 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7933a56640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0c9020 con 0x7f792c15c7f0 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7933a56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792c0c9300 con 0x7f792c0ce6c0 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7930ff1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 0x7f792c0c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7930ff1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 0x7f792c0c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42454/0 (socket says 172.21.15.106:42454) 2023-11-24T08:10:22.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.390+0000 7f7930ff1640 1 -- 172.21.15.106:0/1613003305 learned_addr learned my addr 172.21.15.106:0/1613003305 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:22.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7930ff1640 1 -- 172.21.15.106:0/1613003305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c0ce6c0 msgr2=0x7f792c154e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:22.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7930ff1640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c0ce6c0 0x7f792c154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7930ff1640 1 -- 172.21.15.106:0/1613003305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f792c15c7f0 msgr2=0x7f792c0c9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7930ff1640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f792c15c7f0 0x7f792c0c9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7930ff1640 1 -- 172.21.15.106:0/1613003305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79240094d0 con 0x7f792c0cf450 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7930ff1640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 0x7f792c0c8360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f791800c430 tx=0x7f791800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918011790 con 0x7f792c0cf450 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79180118f0 con 0x7f792c0cf450 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918011ce0 con 0x7f792c0cf450 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.391+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f792c0c95e0 con 0x7f792c0cf450 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.392+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f792c155340 con 0x7f792c0cf450 2023-11-24T08:10:22.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.393+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f792c0c9b00 con 0x7f792c0cf450 2023-11-24T08:10:22.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.394+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f791801c460 con 0x7f792c0cf450 2023-11-24T08:10:22.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.395+0000 7f7929ffb640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f791006aed0 0x7f791006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:22.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.395+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f7918089660 con 0x7f792c0cf450 2023-11-24T08:10:22.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.395+0000 7f79317f2640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f791006aed0 0x7f791006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:22.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.396+0000 7f79317f2640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f791006aed0 0x7f791006d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f79200070b0 tx=0x7f7920007040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:22.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.399+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f792c0c9b00 con 0x7f792c0cf450 2023-11-24T08:10:22.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:22.592+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f792c0c9b00 con 0x7f792c0cf450 2023-11-24T08:10:23.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.477+0000 7f7929ffb640 1 -- 172.21.15.106:0/1613003305 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v38) v1 ==== 181+0+0 (secure 0 0 0) 0x7f792c0c9b00 con 0x7f792c0cf450 2023-11-24T08:10:23.481 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:10:23.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f791006aed0 msgr2=0x7f791006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:23.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f791006aed0 0x7f791006d390 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f79200070b0 tx=0x7f7920007040 comp rx=0 tx=0).stop 2023-11-24T08:10:23.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 msgr2=0x7f792c0c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:23.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 0x7f792c0c8360 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f791800c430 tx=0x7f791800c8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 shutdown_connections 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f792c15c7f0 0x7f792c0c9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f792c0cf450 0x7f792c0c8360 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f791006aed0 0x7f791006d390 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 --2- 172.21.15.106:0/1613003305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f792c0ce6c0 0x7f792c154e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 >> 172.21.15.106:0/1613003305 conn(0x7f792c06a920 msgr2=0x7f792c0cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.481+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 shutdown_connections 2023-11-24T08:10:23.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.482+0000 7f7933a56640 1 -- 172.21.15.106:0/1613003305 wait complete. 2023-11-24T08:10:23.505 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:10:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.710+0000 7f01e7fff640 1 Processor -- start 2023-11-24T08:10:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.710+0000 7f01e7fff640 1 -- start start 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.712+0000 7f01e7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01e8146660 0x7f01e8160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.712+0000 7f01e7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01e80b1560 0x7f01e815e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.712+0000 7f01e7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.712+0000 7f01e7fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e80ad210 con 0x7f01e8146660 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.712+0000 7f01e7fff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e815de90 con 0x7f01e80b1560 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.712+0000 7f01e7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8146a40 con 0x7f01e80ac840 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01e8146660 0x7f01e8160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01e8146660 0x7f01e8160fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42462/0 (socket says 172.21.15.106:42462) 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 -- 172.21.15.106:0/1664724855 learned_addr learned my addr 172.21.15.106:0/1664724855 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e77fe640 1 --2- 172.21.15.106:0/1664724855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01de7fc640 1 --2- 172.21.15.106:0/1664724855 >> v2:172.21.15.156:3300/0 conn(0x7f01e80b1560 0x7f01e815e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 -- 172.21.15.106:0/1664724855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 msgr2=0x7f01e80accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 --2- 172.21.15.106:0/1664724855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e80accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 -- 172.21.15.106:0/1664724855 >> v2:172.21.15.156:3300/0 conn(0x7f01e80b1560 msgr2=0x7f01e815e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 --2- 172.21.15.106:0/1664724855 >> v2:172.21.15.156:3300/0 conn(0x7f01e80b1560 0x7f01e815e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01e6ffd640 1 -- 172.21.15.106:0/1664724855 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e8161830 con 0x7f01e8146660 2023-11-24T08:10:23.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.713+0000 7f01de7fc640 1 --2- 172.21.15.106:0/1664724855 >> v2:172.21.15.156:3300/0 conn(0x7f01e80b1560 0x7f01e815e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.714+0000 7f01e6ffd640 1 --2- 172.21.15.106:0/1664724855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e8146660 0x7f01e8160fe0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01e80b1bd0 tx=0x7f01d802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40efdfe584eab3c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.715+0000 7f01e67fc640 1 -- 172.21.15.106:0/1664724855 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d802cb00 con 0x7f01e8146660 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.715+0000 7f01e67fc640 1 -- 172.21.15.106:0/1664724855 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d802cc60 con 0x7f01e8146660 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.715+0000 7f01e67fc640 1 -- 172.21.15.106:0/1664724855 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80343f0 con 0x7f01e8146660 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 -- 172.21.15.106:0/1664724855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e8146660 msgr2=0x7f01e8160fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1664724855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e8146660 0x7f01e8160fe0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f01e80b1bd0 tx=0x7f01d802bf60 comp rx=0 tx=0).stop 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 -- 172.21.15.106:0/1664724855 shutdown_connections 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1664724855 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e80accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1664724855 >> v2:172.21.15.156:3300/0 conn(0x7f01e80b1560 0x7f01e815e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1664724855 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e8146660 0x7f01e8160fe0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 -- 172.21.15.106:0/1664724855 >> 172.21.15.106:0/1664724855 conn(0x7f01e80cb8a0 msgr2=0x7f01e80cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 -- 172.21.15.106:0/1664724855 shutdown_connections 2023-11-24T08:10:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.716+0000 7f01e7fff640 1 -- 172.21.15.106:0/1664724855 wait complete. 2023-11-24T08:10:23.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.718+0000 7f01e7fff640 1 Processor -- start 2023-11-24T08:10:23.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.718+0000 7f01e7fff640 1 -- start start 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01e7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e8137a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01e7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 0x7f01e80cab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01e7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e8146660 0x7f01e815d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01e7fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80ca6a0 con 0x7f01e80b1560 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01e7fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e8137f80 con 0x7f01e8146660 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01e7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e80c9a90 con 0x7f01e80ac840 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01de7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 0x7f01e80cab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01de7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 0x7f01e80cab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42478/0 (socket says 172.21.15.106:42478) 2023-11-24T08:10:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.719+0000 7f01de7fc640 1 -- 172.21.15.106:0/1921403095 learned_addr learned my addr 172.21.15.106:0/1921403095 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01de7fc640 1 -- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 msgr2=0x7f01e8137a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01de7fc640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e8137a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01e77fe640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e8146660 0x7f01e815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01de7fc640 1 -- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e8146660 msgr2=0x7f01e815d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01de7fc640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e8146660 0x7f01e815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01de7fc640 1 -- 172.21.15.106:0/1921403095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8009b30 con 0x7f01e80b1560 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01de7fc640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 0x7f01e80cab30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c820 tx=0x7f01d000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0005b90 con 0x7f01e80b1560 2023-11-24T08:10:23.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d0005cf0 con 0x7f01e80b1560 2023-11-24T08:10:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0007a30 con 0x7f01e80b1560 2023-11-24T08:10:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e80c9d70 con 0x7f01e80b1560 2023-11-24T08:10:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.720+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e80ca330 con 0x7f01e80b1560 2023-11-24T08:10:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.721+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e804e840 con 0x7f01e80b1560 2023-11-24T08:10:23.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.722+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f01d0007b90 con 0x7f01e80b1560 2023-11-24T08:10:23.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.723+0000 7f01dffff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01c806aed0 0x7f01c806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:23.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.723+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f01d00895d0 con 0x7f01e80b1560 2023-11-24T08:10:23.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.724+0000 7f01e6ffd640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01c806aed0 0x7f01c806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:23.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.724+0000 7f01e6ffd640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01c806aed0 0x7f01c806d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01d8035000 tx=0x7f01d802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:23.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.728+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01d0054670 con 0x7f01e80b1560 2023-11-24T08:10:23.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:23.917+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01e80ca850 con 0x7f01e80b1560 2023-11-24T08:10:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.481+0000 7f01dffff640 1 -- 172.21.15.106:0/1921403095 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v39) v1 ==== 169+0+0 (secure 0 0 0) 0x7f01d0057c90 con 0x7f01e80b1560 2023-11-24T08:10:24.484 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2023-11-24T08:10:24.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01c806aed0 msgr2=0x7f01c806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01c806aed0 0x7f01c806d390 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f01d8035000 tx=0x7f01d802e000 comp rx=0 tx=0).stop 2023-11-24T08:10:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 msgr2=0x7f01e80cab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 0x7f01e80cab30 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f01d000c820 tx=0x7f01d000cc20 comp rx=0 tx=0).stop 2023-11-24T08:10:24.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 shutdown_connections 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01e8146660 0x7f01e815d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01e80b1560 0x7f01e80cab30 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01c806aed0 0x7f01c806d390 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 --2- 172.21.15.106:0/1921403095 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01e80ac840 0x7f01e8137a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 >> 172.21.15.106:0/1921403095 conn(0x7f01e80cb8a0 msgr2=0x7f01e80b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 shutdown_connections 2023-11-24T08:10:24.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.485+0000 7f01e7fff640 1 -- 172.21.15.106:0/1921403095 wait complete. 2023-11-24T08:10:24.511 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:10:24.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.706+0000 7f3d83f6f640 1 Processor -- start 2023-11-24T08:10:24.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.706+0000 7f3d83f6f640 1 -- start start 2023-11-24T08:10:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.708+0000 7f3d83f6f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 0x7f3d7c15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.708+0000 7f3d83f6f640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c15ec90 con 0x7f3d7c06de10 2023-11-24T08:10:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.708+0000 7f3d83f6f640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d7c0cd530 con 0x7f3d7c0c49f0 2023-11-24T08:10:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.708+0000 7f3d83f6f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c4ed0 con 0x7f3d7c072160 2023-11-24T08:10:24.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.708+0000 7f3d8250c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3d7c06de10 0x7f3d7c0705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36278/0 (socket says 172.21.15.106:36278) 2023-11-24T08:10:24.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.708+0000 7f3d8250c640 1 -- 172.21.15.106:0/3738252283 learned_addr learned my addr 172.21.15.106:0/3738252283 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:24.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695616985 0 0) 0x7f3d74001490 con 0x7f3d7c06de10 2023-11-24T08:10:24.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 0x7f3d7c15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003520 con 0x7f3d7c06de10 2023-11-24T08:10:24.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1703445977 0 0) 0x7f3d68003520 con 0x7f3d7c06de10 2023-11-24T08:10:24.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 msgr2=0x7f3d7c15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 0x7f3d7c15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 >> v1:172.21.15.156:6789/0 conn(0x7f3d7c0c49f0 legacy=0x7f3d7c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c0c51b0 con 0x7f3d7c06de10 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f3d74001b80 con 0x7f3d7c06de10 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 >> v1:172.21.15.106:6789/0 conn(0x7f3d7c06de10 legacy=0x7f3d7c0705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d680039a0 0x7f3d68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 0x7f3d680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.709+0000 7f3d80d09640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0c4ed0 con 0x7f3d680039a0 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800bb10 con 0x7f3d68004d00 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6800bca0 con 0x7f3d7c072160 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d680039a0 0x7f3d68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d8150a640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 0x7f3d680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d8250c640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d81d0b640 1 -- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 msgr2=0x7f3d680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d7c072160 0x7f3d680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d81d0b640 1 -- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d68004d00 msgr2=0x7f3d68005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d81d0b640 1 -- 172.21.15.106:0/3738252283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d68003520 con 0x7f3d680039a0 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.710+0000 7f3d8250c640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d68004d00 0x7f3d68005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.711+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d680039a0 0x7f3d68003dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ba60 tx=0x7f3d680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3bddc001c8bdae server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:24.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78032630 con 0x7f3d680039a0 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78032790 con 0x7f3d680039a0 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d80d09640 1 -- 172.21.15.106:0/3738252283 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78032b20 con 0x7f3d680039a0 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3738252283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d680039a0 msgr2=0x7f3d68003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3738252283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d680039a0 0x7f3d68003dc0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ba60 tx=0x7f3d680057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d8250c640 1 -- 172.21.15.106:0/3738252283 reap_dead start 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3738252283 shutdown_connections 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3738252283 >> 172.21.15.106:0/3738252283 conn(0x7f3d7c0b0c50 msgr2=0x7f3d7c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:24.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3738252283 shutdown_connections 2023-11-24T08:10:24.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.712+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3738252283 wait complete. 2023-11-24T08:10:24.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.714+0000 7f3d83f6f640 1 Processor -- start 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.714+0000 7f3d83f6f640 1 -- start start 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d83f6f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d680039a0 0x7f3d7c13eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d83f6f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 0x7f3d7c0af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d83f6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d7c09b6c0 0x7f3d7c09bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d83f6f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0afb20 con 0x7f3d7c0af160 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d83f6f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c0b2120 con 0x7f3d7c09b6c0 2023-11-24T08:10:24.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d83f6f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7c09c9f0 con 0x7f3d680039a0 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d8150a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 0x7f3d7c0af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d8150a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 0x7f3d7c0af5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42854/0 (socket says 172.21.15.106:42854) 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.715+0000 7f3d8150a640 1 -- 172.21.15.106:0/3645763575 learned_addr learned my addr 172.21.15.106:0/3645763575 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8250c640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d7c09b6c0 0x7f3d7c09bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8150a640 1 -- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d680039a0 msgr2=0x7f3d7c13eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d680039a0 0x7f3d7c13eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8150a640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d680039a0 0x7f3d7c13eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8150a640 1 -- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d7c09b6c0 msgr2=0x7f3d7c09bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8150a640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d7c09b6c0 0x7f3d7c09bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8150a640 1 -- 172.21.15.106:0/3645763575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d7c09c430 con 0x7f3d7c0af160 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8250c640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d7c09b6c0 0x7f3d7c09bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d8150a640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 0x7f3d7c0af5e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00e580 tx=0x7f3d6c00ea40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c0052e0 con 0x7f3d7c0af160 2023-11-24T08:10:24.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.716+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6c005440 con 0x7f3d7c0af160 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.717+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6c003810 con 0x7f3d7c0af160 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.717+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7c09c710 con 0x7f3d7c0af160 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.717+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7c09c080 con 0x7f3d7c0af160 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.718+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d7c04e8b0 con 0x7f3d7c0af160 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.719+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3d6c003970 con 0x7f3d7c0af160 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.720+0000 7f3d727fc640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3d6006af00 0x7f3d6006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:24.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.720+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3d6c089070 con 0x7f3d7c0af160 2023-11-24T08:10:24.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.720+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3d6006af00 0x7f3d6006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:24.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.721+0000 7f3d81d0b640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3d6006af00 0x7f3d6006d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ba30 tx=0x7f3d7803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:24.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.724+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3d6c054440 con 0x7f3d7c0af160 2023-11-24T08:10:24.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.913+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3d7c15dab0 con 0x7f3d7c0af160 2023-11-24T08:10:24.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.919+0000 7f3d727fc640 1 -- 172.21.15.106:0/3645763575 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3d6c057a60 con 0x7f3d7c0af160 2023-11-24T08:10:24.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3d6006af00 msgr2=0x7f3d6006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3d6006af00 0x7f3d6006d3c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800ba30 tx=0x7f3d7803d040 comp rx=0 tx=0).stop 2023-11-24T08:10:24.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 msgr2=0x7f3d7c0af5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:24.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 0x7f3d7c0af5e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c00e580 tx=0x7f3d6c00ea40 comp rx=0 tx=0).stop 2023-11-24T08:10:24.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 shutdown_connections 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3d7c09b6c0 0x7f3d7c09bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3d7c0af160 0x7f3d7c0af5e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3d6006af00 0x7f3d6006d3c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 --2- 172.21.15.106:0/3645763575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3d680039a0 0x7f3d7c13eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 >> 172.21.15.106:0/3645763575 conn(0x7f3d7c0b0c50 msgr2=0x7f3d7c0cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 shutdown_connections 2023-11-24T08:10:24.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:24.922+0000 7f3d83f6f640 1 -- 172.21.15.106:0/3645763575 wait complete. 2023-11-24T08:10:24.942 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status' 2023-11-24T08:10:25.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.153+0000 7f1f3359e640 1 Processor -- start 2023-11-24T08:10:25.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.153+0000 7f1f3359e640 1 -- start start 2023-11-24T08:10:25.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f3359e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340703f0 0x7f1f34162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f3359e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f340cd160 0x7f1f340cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f3359e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f34162eb0 con 0x7f1f3414b100 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f3359e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f341654a0 con 0x7f1f340cd160 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f3359e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340ccf00 con 0x7f1f340703f0 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f32d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f340cd160 0x7f1f340cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f32d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f340cd160 0x7f1f340cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42856/0 (socket says 172.21.15.106:42856) 2023-11-24T08:10:25.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f32d9d640 1 -- 172.21.15.106:0/2496621678 learned_addr learned my addr 172.21.15.106:0/2496621678 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:25.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.155+0000 7f1f2bfff640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340703f0 0x7f1f34162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.156+0000 7f1f32d9d640 1 -- 172.21.15.106:0/2496621678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340703f0 msgr2=0x7f1f34162970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:25.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.156+0000 7f1f32d9d640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340703f0 0x7f1f34162970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.156+0000 7f1f32d9d640 1 -- 172.21.15.106:0/2496621678 >> v1:172.21.15.156:6789/0 conn(0x7f1f3414b100 legacy=0x7f1f34165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.156+0000 7f1f32d9d640 1 -- 172.21.15.106:0/2496621678 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f34166320 con 0x7f1f340cd160 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.156+0000 7f1f2bfff640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340703f0 0x7f1f34162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.157+0000 7f1f32d9d640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f340cd160 0x7f1f340cd5f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009e50 tx=0x7f1f2c02be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c084190ebf04ffa7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.157+0000 7f1f31d9b640 1 -- 172.21.15.106:0/2496621678 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c02c9e0 con 0x7f1f340cd160 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.157+0000 7f1f31d9b640 1 -- 172.21.15.106:0/2496621678 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c02cb40 con 0x7f1f340cd160 2023-11-24T08:10:25.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.157+0000 7f1f31d9b640 1 -- 172.21.15.106:0/2496621678 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0343f0 con 0x7f1f340cd160 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 -- 172.21.15.106:0/2496621678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f340cd160 msgr2=0x7f1f340cd5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f340cd160 0x7f1f340cd5f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c009e50 tx=0x7f1f2c02be40 comp rx=0 tx=0).stop 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 -- 172.21.15.106:0/2496621678 shutdown_connections 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f340cd160 0x7f1f340cd5f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 --2- 172.21.15.106:0/2496621678 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340703f0 0x7f1f34162970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 -- 172.21.15.106:0/2496621678 >> 172.21.15.106:0/2496621678 conn(0x7f1f340cb830 msgr2=0x7f1f340cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 -- 172.21.15.106:0/2496621678 shutdown_connections 2023-11-24T08:10:25.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.158+0000 7f1f3359e640 1 -- 172.21.15.106:0/2496621678 wait complete. 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.160+0000 7f1f3359e640 1 Processor -- start 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.160+0000 7f1f3359e640 1 -- start start 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3359e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340703f0 0x7f1f34149200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3359e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340cd160 0x7f1f340aa140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 0x7f1f340aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340b2760 con 0x7f1f3414b100 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3359e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340b4f00 con 0x7f1f340703f0 2023-11-24T08:10:25.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3359e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f340b2970 con 0x7f1f340cd160 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 0x7f1f340aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 0x7f1f340aa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42860/0 (socket says 172.21.15.106:42860) 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 -- 172.21.15.106:0/150784849 learned_addr learned my addr 172.21.15.106:0/150784849 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f3259c640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340703f0 0x7f1f34149200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f2bfff640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340cd160 0x7f1f340aa140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 -- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340cd160 msgr2=0x7f1f340aa140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340cd160 0x7f1f340aa140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 -- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340703f0 msgr2=0x7f1f34149200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.161+0000 7f1f32d9d640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340703f0 0x7f1f34149200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f32d9d640 1 -- 172.21.15.106:0/150784849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c009b30 con 0x7f1f3414b100 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f3259c640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340703f0 0x7f1f34149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f32d9d640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 0x7f1f340aa680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c036040 tx=0x7f1f2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c03e450 con 0x7f1f3414b100 2023-11-24T08:10:25.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f2c03e5d0 con 0x7f1f3414b100 2023-11-24T08:10:25.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c03e980 con 0x7f1f3414b100 2023-11-24T08:10:25.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f340b2c50 con 0x7f1f3414b100 2023-11-24T08:10:25.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.162+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f34149740 con 0x7f1f3414b100 2023-11-24T08:10:25.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.163+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f340b3110 con 0x7f1f3414b100 2023-11-24T08:10:25.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.164+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1f2c03eae0 con 0x7f1f3414b100 2023-11-24T08:10:25.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.165+0000 7f1f2affd640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f1406aed0 0x7f1f1406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.165+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1f2c0ac1a0 con 0x7f1f3414b100 2023-11-24T08:10:25.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.165+0000 7f1f3259c640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f1406aed0 0x7f1f1406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.166+0000 7f1f3259c640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f1406aed0 0x7f1f1406d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f24003eb0 tx=0x7f1f24009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:25.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.169+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f2c077580 con 0x7f1f3414b100 2023-11-24T08:10:25.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.450+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]} v 0) v1 -- 0x7f1f34162eb0 con 0x7f1f3414b100 2023-11-24T08:10:25.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.496+0000 7f1f2affd640 1 -- 172.21.15.106:0/150784849 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots.test_subvolume_group_snapshot_unsupported_status"]}]=0 v108) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1f34162eb0 con 0x7f1f3414b100 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f1406aed0 msgr2=0x7f1f1406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f1406aed0 0x7f1f1406d390 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1f24003eb0 tx=0x7f1f24009040 comp rx=0 tx=0).stop 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 msgr2=0x7f1f340aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 0x7f1f340aa680 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c036040 tx=0x7f1f2c004a40 comp rx=0 tx=0).stop 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 shutdown_connections 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f3414b100 0x7f1f340aa680 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f1406aed0 0x7f1f1406d390 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f340cd160 0x7f1f340aa140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 --2- 172.21.15.106:0/150784849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f340703f0 0x7f1f34149200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.499+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 >> 172.21.15.106:0/150784849 conn(0x7f1f340cb830 msgr2=0x7f1f34163050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:25.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.500+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 shutdown_connections 2023-11-24T08:10:25.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.500+0000 7f1f3359e640 1 -- 172.21.15.106:0/150784849 wait complete. 2023-11-24T08:10:25.520 INFO:tasks.cephfs_test_runner:test_subvolume_group_snapshot_unsupported_status (tasks.cephfs.test_volumes.TestSubvolumeGroupSnapshots) ... ok 2023-11-24T08:10:25.520 INFO:tasks.cephfs_test_runner:Starting test: test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) Validate cleaning of stale subvolume snapshot metadata. 2023-11-24T08:10:25.520 INFO:tasks.cephfs_test_runner:test_clean_stale_subvolume_snapshot_metadata (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:10:25.521 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:25.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.728+0000 7f726e2d1640 1 Processor -- start 2023-11-24T08:10:25.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.728+0000 7f726e2d1640 1 -- start start 2023-11-24T08:10:25.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726e2d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f726815af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726e2d1640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f726806db20 0x7f72680b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726e2d1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72680cec00 con 0x7f726806d6c0 2023-11-24T08:10:25.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726e2d1640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726813f9e0 con 0x7f726806db20 2023-11-24T08:10:25.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726e2d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726813fcc0 con 0x7f726813d120 2023-11-24T08:10:25.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726cace640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f726806d6c0 0x7f72680b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36292/0 (socket says 172.21.15.106:36292) 2023-11-24T08:10:25.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.730+0000 7f726cace640 1 -- 172.21.15.106:0/2201368677 learned_addr learned my addr 172.21.15.106:0/2201368677 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:25.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f726d2cf640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f726815af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f726dad0640 1 --2- 172.21.15.106:0/2201368677 >> v2:172.21.15.156:3300/0 conn(0x7f726806db20 0x7f72680b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026577187 0 0) 0x7f72680cec00 con 0x7f726806d6c0 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7254005730 con 0x7f726806d6c0 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 904913856 0 0) 0x7f7254005730 con 0x7f726806d6c0 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 msgr2=0x7f726815af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f726815af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 >> v2:172.21.15.156:3300/0 conn(0x7f726806db20 msgr2=0x7f72680b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 --2- 172.21.15.106:0/2201368677 >> v2:172.21.15.156:3300/0 conn(0x7f726806db20 0x7f72680b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726813ffa0 con 0x7f726806d6c0 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f7258001670 con 0x7f726806d6c0 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 >> v1:172.21.15.106:6789/0 conn(0x7f726806d6c0 legacy=0x7f72680b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f7254005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7254006100 0x7f72540069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254006f10 0x7f7254007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726813f9e0 con 0x7f7254006100 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726813fcc0 con 0x7f7254006f10 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.731+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725400dbe0 con 0x7f726813d120 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726cace640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7254006100 0x7f72540069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726dad0640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254006f10 0x7f7254007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726d2cf640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f7254005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726cace640 1 -- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 msgr2=0x7f7254005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726cace640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f7254005b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726cace640 1 -- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254006f10 msgr2=0x7f7254007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:25.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726cace640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7254006f10 0x7f7254007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726cace640 1 -- 172.21.15.106:0/2201368677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254005730 con 0x7f7254006100 2023-11-24T08:10:25.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.732+0000 7f726d2cf640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f726813d120 0x7f7254005b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:25.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.733+0000 7f726cace640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7254006100 0x7f72540069d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f725800c2c0 tx=0x7f7254007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cc37338d4809de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:25.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258005ad0 con 0x7f7254006100 2023-11-24T08:10:25.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7258005c30 con 0x7f7254006100 2023-11-24T08:10:25.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f725ffff640 1 -- 172.21.15.106:0/2201368677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725802c8b0 con 0x7f7254006100 2023-11-24T08:10:25.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726e2d1640 1 -- 172.21.15.106:0/2201368677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7254006100 msgr2=0x7f72540069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2201368677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7254006100 0x7f72540069d0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f725800c2c0 tx=0x7f7254007a00 comp rx=0 tx=0).stop 2023-11-24T08:10:25.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726dad0640 1 -- 172.21.15.106:0/2201368677 reap_dead start 2023-11-24T08:10:25.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726e2d1640 1 -- 172.21.15.106:0/2201368677 shutdown_connections 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726e2d1640 1 -- 172.21.15.106:0/2201368677 >> 172.21.15.106:0/2201368677 conn(0x7f72680cb830 msgr2=0x7f72680cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726e2d1640 1 -- 172.21.15.106:0/2201368677 shutdown_connections 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.734+0000 7f726e2d1640 1 -- 172.21.15.106:0/2201368677 wait complete. 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.736+0000 7f726e2d1640 1 Processor -- start 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.736+0000 7f726e2d1640 1 -- start start 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726e2d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7254006100 0x7f7268138970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726e2d1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72680b1340 0x7f72680ac840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726e2d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 0x7f72680cee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726e2d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726815bc20 con 0x7f72680acd80 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726e2d1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726815bf00 con 0x7f72680b1340 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726e2d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726815c1e0 con 0x7f7254006100 2023-11-24T08:10:25.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726dad0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 0x7f72680cee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726dad0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 0x7f72680cee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42886/0 (socket says 172.21.15.106:42886) 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.737+0000 7f726dad0640 1 -- 172.21.15.106:0/2669965326 learned_addr learned my addr 172.21.15.106:0/2669965326 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726d2cf640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7254006100 0x7f7268138970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726cace640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72680b1340 0x7f72680ac840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726dad0640 1 -- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7254006100 msgr2=0x7f7268138970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726dad0640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7254006100 0x7f7268138970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726dad0640 1 -- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72680b1340 msgr2=0x7f72680ac840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726dad0640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72680b1340 0x7f72680ac840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726dad0640 1 -- 172.21.15.106:0/2669965326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72680cda90 con 0x7f72680acd80 2023-11-24T08:10:25.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f726dad0640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 0x7f72680cee60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7268158170 tx=0x7f72600115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:25.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726000e070 con 0x7f72680acd80 2023-11-24T08:10:25.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260011e10 con 0x7f72680acd80 2023-11-24T08:10:25.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.738+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7260003990 con 0x7f72680acd80 2023-11-24T08:10:25.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.739+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72680cdd70 con 0x7f72680acd80 2023-11-24T08:10:25.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.739+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72680b19a0 con 0x7f72680acd80 2023-11-24T08:10:25.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.740+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72680cdf60 con 0x7f72680acd80 2023-11-24T08:10:25.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.741+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7260003af0 con 0x7f72680acd80 2023-11-24T08:10:25.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.742+0000 7f725dffb640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f724c06aed0 0x7f724c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:25.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.742+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7260089b40 con 0x7f72680acd80 2023-11-24T08:10:25.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.742+0000 7f726d2cf640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f724c06aed0 0x7f724c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:25.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.742+0000 7f726d2cf640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f724c06aed0 0x7f724c06d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f72640098f0 tx=0x7f7264008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:25.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.746+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7260054cf0 con 0x7f72680acd80 2023-11-24T08:10:25.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.936+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f726804e840 con 0x7f72680acd80 2023-11-24T08:10:25.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.938+0000 7f725dffb640 1 -- 172.21.15.106:0/2669965326 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+13036 (secure 0 0 0) 0x7f7260058310 con 0x7f72680acd80 2023-11-24T08:10:25.940 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:25.941 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":39,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:10:24.478867+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6836/3854927347":"2023-11-25T08:10:21.147066+0000","172.21.15.106:6834/3854927347":"2023-11-25T08:10:21.147066+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.940+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f724c06aed0 msgr2=0x7f724c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.940+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f724c06aed0 0x7f724c06d390 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f72640098f0 tx=0x7f7264008040 comp rx=0 tx=0).stop 2023-11-24T08:10:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.940+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 msgr2=0x7f72680cee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.940+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 0x7f72680cee60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7268158170 tx=0x7f72600115a0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 shutdown_connections 2023-11-24T08:10:25.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72680acd80 0x7f72680cee60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72680b1340 0x7f72680ac840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f724c06aed0 0x7f724c06d390 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 --2- 172.21.15.106:0/2669965326 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7254006100 0x7f7268138970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:25.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 >> 172.21.15.106:0/2669965326 conn(0x7f72680cb830 msgr2=0x7f726806cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:25.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 shutdown_connections 2023-11-24T08:10:25.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:25.941+0000 7f726e2d1640 1 -- 172.21.15.106:0/2669965326 wait complete. 2023-11-24T08:10:25.963 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:10:25.963 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:10:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.178+0000 7f3e17213640 1 Processor -- start 2023-11-24T08:10:26.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.178+0000 7f3e17213640 1 -- start start 2023-11-24T08:10:26.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.179+0000 7f3e17213640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3e100b0b40 0x7f3e100b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e17213640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3e100cce40 0x7f3e100cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e17213640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e100b14d0 con 0x7f3e1015d310 2023-11-24T08:10:26.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e17213640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e1006cd10 con 0x7f3e100cce40 2023-11-24T08:10:26.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e17213640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e100ae5d0 con 0x7f3e100b0b40 2023-11-24T08:10:26.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e16211640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3e1015d310 0x7f3e100ca120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36294/0 (socket says 172.21.15.106:36294) 2023-11-24T08:10:26.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e16211640 1 -- 172.21.15.106:0/3745336962 learned_addr learned my addr 172.21.15.106:0/3745336962 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:26.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e16a12640 1 --2- 172.21.15.106:0/3745336962 >> v2:172.21.15.106:3300/0 conn(0x7f3e100cce40 0x7f3e100cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.180+0000 7f3e15a10640 1 --2- 172.21.15.106:0/3745336962 >> v2:172.21.15.156:3300/0 conn(0x7f3e100b0b40 0x7f3e100b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.181+0000 7f3e16a12640 1 -- 172.21.15.106:0/3745336962 >> v2:172.21.15.156:3300/0 conn(0x7f3e100b0b40 msgr2=0x7f3e100b0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:26.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.181+0000 7f3e16a12640 1 --2- 172.21.15.106:0/3745336962 >> v2:172.21.15.156:3300/0 conn(0x7f3e100b0b40 0x7f3e100b0f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.181+0000 7f3e16a12640 1 -- 172.21.15.106:0/3745336962 >> v1:172.21.15.106:6789/0 conn(0x7f3e1015d310 legacy=0x7f3e100ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.181+0000 7f3e1520f640 1 -- 172.21.15.106:0/3745336962 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3037116917 0 0) 0x7f3e0c001110 con 0x7f3e1015d310 2023-11-24T08:10:26.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.181+0000 7f3e16a12640 1 -- 172.21.15.106:0/3745336962 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100ae8b0 con 0x7f3e100cce40 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.182+0000 7f3e16a12640 1 --2- 172.21.15.106:0/3745336962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100cce40 0x7f3e100cd2a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b1cb0 tx=0x7f3e0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd328badae198b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.182+0000 7f3e1520f640 1 -- 172.21.15.106:0/3745336962 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0802cc80 con 0x7f3e100cce40 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.182+0000 7f3e1520f640 1 -- 172.21.15.106:0/3745336962 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0802cde0 con 0x7f3e100cce40 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e1520f640 1 -- 172.21.15.106:0/3745336962 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08034660 con 0x7f3e100cce40 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 -- 172.21.15.106:0/3745336962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100cce40 msgr2=0x7f3e100cd2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 --2- 172.21.15.106:0/3745336962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100cce40 0x7f3e100cd2a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3e100b1cb0 tx=0x7f3e0802bed0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 -- 172.21.15.106:0/3745336962 shutdown_connections 2023-11-24T08:10:26.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 --2- 172.21.15.106:0/3745336962 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100cce40 0x7f3e100cd2a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 --2- 172.21.15.106:0/3745336962 >> v2:172.21.15.156:3300/0 conn(0x7f3e100b0b40 0x7f3e100b0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 -- 172.21.15.106:0/3745336962 >> 172.21.15.106:0/3745336962 conn(0x7f3e100cbb40 msgr2=0x7f3e100cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:26.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 -- 172.21.15.106:0/3745336962 shutdown_connections 2023-11-24T08:10:26.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.183+0000 7f3e17213640 1 -- 172.21.15.106:0/3745336962 wait complete. 2023-11-24T08:10:26.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.185+0000 7f3e17213640 1 Processor -- start 2023-11-24T08:10:26.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.185+0000 7f3e17213640 1 -- start start 2023-11-24T08:10:26.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.186+0000 7f3e17213640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 0x7f3e1015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.186+0000 7f3e17213640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e100cce40 0x7f3e10071230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.186+0000 7f3e17213640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e1015d310 0x7f3e101563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.186+0000 7f3e17213640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100c4c30 con 0x7f3e100b0b40 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.186+0000 7f3e17213640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e101604d0 con 0x7f3e1015d310 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.186+0000 7f3e17213640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e100c45e0 con 0x7f3e100cce40 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 0x7f3e1015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 0x7f3e1015e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42904/0 (socket says 172.21.15.106:42904) 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 -- 172.21.15.106:0/3509333932 learned_addr learned my addr 172.21.15.106:0/3509333932 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:26.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16a12640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e1015d310 0x7f3e101563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e15a10640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e100cce40 0x7f3e10071230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 -- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e100cce40 msgr2=0x7f3e10071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e100cce40 0x7f3e10071230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 -- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e1015d310 msgr2=0x7f3e101563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e1015d310 0x7f3e101563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 -- 172.21.15.106:0/3509333932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e08009a70 con 0x7f3e100b0b40 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.187+0000 7f3e16211640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 0x7f3e1015e370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00c430 tx=0x7f3e0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.188+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0074e0 con 0x7f3e100b0b40 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.188+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c007660 con 0x7f3e100b0b40 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.188+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e100c48c0 con 0x7f3e100b0b40 2023-11-24T08:10:26.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.188+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004a20 con 0x7f3e100b0b40 2023-11-24T08:10:26.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.188+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1015e8b0 con 0x7f3e100b0b40 2023-11-24T08:10:26.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.189+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1004e840 con 0x7f3e100b0b40 2023-11-24T08:10:26.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.190+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3e0c004b80 con 0x7f3e100b0b40 2023-11-24T08:10:26.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.191+0000 7f3e06ffd640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3df406aed0 0x7f3df406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.191+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e0c088f20 con 0x7f3e100b0b40 2023-11-24T08:10:26.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.191+0000 7f3e15a10640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3df406aed0 0x7f3df406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.192+0000 7f3e15a10640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3df406aed0 0x7f3df406d390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e100af5d0 tx=0x7f3e00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:26.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.195+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3e0c0bbe90 con 0x7f3e100b0b40 2023-11-24T08:10:26.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.385+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e100b14d0 con 0x7f3e100b0b40 2023-11-24T08:10:26.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.386+0000 7f3e06ffd640 1 -- 172.21.15.106:0/3509333932 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e100b14d0 con 0x7f3e100b0b40 2023-11-24T08:10:26.388 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:10:26.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3df406aed0 msgr2=0x7f3df406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3df406aed0 0x7f3df406d390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3e100af5d0 tx=0x7f3e00008040 comp rx=0 tx=0).stop 2023-11-24T08:10:26.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 msgr2=0x7f3e1015e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 0x7f3e1015e370 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c00c430 tx=0x7f3e0c00c8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 shutdown_connections 2023-11-24T08:10:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e1015d310 0x7f3e101563d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3df406aed0 0x7f3df406d390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3e100cce40 0x7f3e10071230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 --2- 172.21.15.106:0/3509333932 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3e100b0b40 0x7f3e1015e370 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.388+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 >> 172.21.15.106:0/3509333932 conn(0x7f3e100cbb40 msgr2=0x7f3e100cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.389+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 shutdown_connections 2023-11-24T08:10:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.389+0000 7f3e17213640 1 -- 172.21.15.106:0/3509333932 wait complete. 2023-11-24T08:10:26.409 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T08:10:26.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.615+0000 7f8161067640 1 Processor -- start 2023-11-24T08:10:26.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.615+0000 7f8161067640 1 -- start start 2023-11-24T08:10:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.616+0000 7f8161067640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f815c157a90 0x7f815c15e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.616+0000 7f8161067640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f815c15a620 0x7f815c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.616+0000 7f8161067640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c06c9a0 0x7f815c06ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.616+0000 7f8161067640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815c06d350 con 0x7f815c157a90 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f8161067640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f815c15dc10 con 0x7f815c15a620 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f8161067640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c15b130 con 0x7f815c06c9a0 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f815c157a90 0x7f815c15e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f815c157a90 0x7f815c15e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42918/0 (socket says 172.21.15.106:42918) 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 -- 172.21.15.106:0/3305186240 learned_addr learned my addr 172.21.15.106:0/3305186240 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f8160866640 1 --2- 172.21.15.106:0/3305186240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c06c9a0 0x7f815c06ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 -- 172.21.15.106:0/3305186240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c06c9a0 msgr2=0x7f815c06ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:26.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815b7fe640 1 --2- 172.21.15.106:0/3305186240 >> v2:172.21.15.156:3300/0 conn(0x7f815c15a620 0x7f815c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 --2- 172.21.15.106:0/3305186240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c06c9a0 0x7f815c06ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 -- 172.21.15.106:0/3305186240 >> v2:172.21.15.156:3300/0 conn(0x7f815c15a620 msgr2=0x7f815c15aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:26.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 --2- 172.21.15.106:0/3305186240 >> v2:172.21.15.156:3300/0 conn(0x7f815c15a620 0x7f815c15aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.617+0000 7f815bfff640 1 -- 172.21.15.106:0/3305186240 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c15b410 con 0x7f815c157a90 2023-11-24T08:10:26.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.618+0000 7f815bfff640 1 --2- 172.21.15.106:0/3305186240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c157a90 0x7f815c15e6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f815c0cecd0 tx=0x7f814c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2852e01e10d68f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:26.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f815affd640 1 -- 172.21.15.106:0/3305186240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c031870 con 0x7f815c157a90 2023-11-24T08:10:26.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f815affd640 1 -- 172.21.15.106:0/3305186240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c0319d0 con 0x7f815c157a90 2023-11-24T08:10:26.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f815affd640 1 -- 172.21.15.106:0/3305186240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c031d60 con 0x7f815c157a90 2023-11-24T08:10:26.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 -- 172.21.15.106:0/3305186240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c157a90 msgr2=0x7f815c15e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 --2- 172.21.15.106:0/3305186240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c157a90 0x7f815c15e6b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f815c0cecd0 tx=0x7f814c02b910 comp rx=0 tx=0).stop 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 -- 172.21.15.106:0/3305186240 shutdown_connections 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 --2- 172.21.15.106:0/3305186240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c06c9a0 0x7f815c06ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 --2- 172.21.15.106:0/3305186240 >> v2:172.21.15.156:3300/0 conn(0x7f815c15a620 0x7f815c15aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 --2- 172.21.15.106:0/3305186240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c157a90 0x7f815c15e6b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.619+0000 7f8161067640 1 -- 172.21.15.106:0/3305186240 >> 172.21.15.106:0/3305186240 conn(0x7f815c0b4630 msgr2=0x7f815c0b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.620+0000 7f8161067640 1 -- 172.21.15.106:0/3305186240 shutdown_connections 2023-11-24T08:10:26.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.620+0000 7f8161067640 1 -- 172.21.15.106:0/3305186240 wait complete. 2023-11-24T08:10:26.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.621+0000 7f8161067640 1 Processor -- start 2023-11-24T08:10:26.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.621+0000 7f8161067640 1 -- start start 2023-11-24T08:10:26.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f8161067640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 0x7f815c1f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f8161067640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c157a90 0x7f815c154190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f8161067640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c15a620 0x7f815c14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f8161067640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c151e90 con 0x7f815c06c9a0 2023-11-24T08:10:26.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f8161067640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c152170 con 0x7f815c157a90 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f8161067640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c152450 con 0x7f815c15a620 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.622+0000 7f815bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 0x7f815c1f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 0x7f815c1f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42924/0 (socket says 172.21.15.106:42924) 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 -- 172.21.15.106:0/2601281110 learned_addr learned my addr 172.21.15.106:0/2601281110 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 -- 172.21.15.106:0/2601281110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c15a620 msgr2=0x7f815c14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c15a620 0x7f815c14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 -- 172.21.15.106:0/2601281110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c157a90 msgr2=0x7f815c154190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c157a90 0x7f815c154190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:26.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 -- 172.21.15.106:0/2601281110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c0094d0 con 0x7f815c06c9a0 2023-11-24T08:10:26.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f815bfff640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 0x7f815c1f3320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f814c02b950 tx=0x7f814c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:26.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c031860 con 0x7f815c06c9a0 2023-11-24T08:10:26.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814c0443f0 con 0x7f815c06c9a0 2023-11-24T08:10:26.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814c0447a0 con 0x7f815c06c9a0 2023-11-24T08:10:26.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.623+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c152730 con 0x7f815c06c9a0 2023-11-24T08:10:26.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.624+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c1f3860 con 0x7f815c06c9a0 2023-11-24T08:10:26.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.625+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c0726c0 con 0x7f815c06c9a0 2023-11-24T08:10:26.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.626+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f814c04d450 con 0x7f815c06c9a0 2023-11-24T08:10:26.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.626+0000 7f8158ff9640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f814806aed0 0x7f814806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:26.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.627+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f814c02d040 con 0x7f815c06c9a0 2023-11-24T08:10:26.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.627+0000 7f815b7fe640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f814806aed0 0x7f814806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:26.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.627+0000 7f815b7fe640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f814806aed0 0x7f814806d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8150007130 tx=0x7f8150007310 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:26.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.631+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f814c076ed0 con 0x7f815c06c9a0 2023-11-24T08:10:26.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:26.890+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7f815c152c50 con 0x7f815c06c9a0 2023-11-24T08:10:27.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.508+0000 7f8158ff9640 1 -- 172.21.15.106:0/2601281110 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v110) v1 ==== 175+0+0 (secure 0 0 0) 0x7f815c152c50 con 0x7f815c06c9a0 2023-11-24T08:10:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f814806aed0 msgr2=0x7f814806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f814806aed0 0x7f814806d390 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8150007130 tx=0x7f8150007310 comp rx=0 tx=0).stop 2023-11-24T08:10:27.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 msgr2=0x7f815c1f3320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 0x7f815c1f3320 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f814c02b950 tx=0x7f814c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 shutdown_connections 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f814806aed0 0x7f814806d390 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f815c15a620 0x7f815c14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c157a90 0x7f815c154190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 --2- 172.21.15.106:0/2601281110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c06c9a0 0x7f815c1f3320 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.511+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 >> 172.21.15.106:0/2601281110 conn(0x7f815c0b4630 msgr2=0x7f815c071c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.512+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 shutdown_connections 2023-11-24T08:10:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.512+0000 7f8161067640 1 -- 172.21.15.106:0/2601281110 wait complete. 2023-11-24T08:10:27.535 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:10:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.748+0000 7f83f0ea2640 1 Processor -- start 2023-11-24T08:10:27.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.749+0000 7f83f0ea2640 1 -- start start 2023-11-24T08:10:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.750+0000 7f83f0ea2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f83ec15b440 0x7f83ec0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.750+0000 7f83f0ea2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0c5dc0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.750+0000 7f83f0ea2640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec15eea0 con 0x7f83ec0cbf20 2023-11-24T08:10:27.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.750+0000 7f83f0ea2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0cc3d0 con 0x7f83ec15b440 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83ead9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f83ec0c59b0 0x7f83ec06c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36306/0 (socket says 172.21.15.106:36306) 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83ead9d640 1 -- 172.21.15.106:0/3561389027 learned_addr learned my addr 172.21.15.106:0/3561389027 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83ea59c640 1 --2- 172.21.15.106:0/3561389027 >> v2:172.21.15.106:3300/0 conn(0x7f83ec15b440 0x7f83ec0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83ea59c640 1 -- 172.21.15.106:0/3561389027 >> v1:172.21.15.156:6789/0 conn(0x7f83ec0cbf20 legacy=0x7f83ec0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83e9d9b640 1 -- 172.21.15.106:0/3561389027 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3437306988 0 0) 0x7f83ec0c5dc0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83ea59c640 1 -- 172.21.15.106:0/3561389027 >> v1:172.21.15.106:6789/0 conn(0x7f83ec0c59b0 legacy=0x7f83ec06c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.751+0000 7f83ea59c640 1 -- 172.21.15.106:0/3561389027 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0abf30 con 0x7f83ec15b440 2023-11-24T08:10:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.752+0000 7f83ea59c640 1 --2- 172.21.15.106:0/3561389027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec15b440 0x7f83ec0cb9e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0097a0 tx=0x7f83dc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0c9653d09093358 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83e9d9b640 1 -- 172.21.15.106:0/3561389027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc031830 con 0x7f83ec15b440 2023-11-24T08:10:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83e9d9b640 1 -- 172.21.15.106:0/3561389027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc031990 con 0x7f83ec15b440 2023-11-24T08:10:27.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83e9d9b640 1 -- 172.21.15.106:0/3561389027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc031d20 con 0x7f83ec15b440 2023-11-24T08:10:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 -- 172.21.15.106:0/3561389027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec15b440 msgr2=0x7f83ec0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/3561389027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec15b440 0x7f83ec0cb9e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0097a0 tx=0x7f83dc02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 -- 172.21.15.106:0/3561389027 shutdown_connections 2023-11-24T08:10:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/3561389027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec15b440 0x7f83ec0cb9e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 -- 172.21.15.106:0/3561389027 >> 172.21.15.106:0/3561389027 conn(0x7f83ec0b0c50 msgr2=0x7f83ec0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 -- 172.21.15.106:0/3561389027 shutdown_connections 2023-11-24T08:10:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.753+0000 7f83f0ea2640 1 -- 172.21.15.106:0/3561389027 wait complete. 2023-11-24T08:10:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.755+0000 7f83f0ea2640 1 Processor -- start 2023-11-24T08:10:27.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.755+0000 7f83f0ea2640 1 -- start start 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83f0ea2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 0x7f83ec0ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83f0ea2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec0cbf20 0x7f83ec0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83f0ea2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83ec15b440 0x7f83ec0b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83f0ea2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec15d950 con 0x7f83ec0c59b0 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83f0ea2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1543a0 con 0x7f83ec0cbf20 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83f0ea2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec0b3070 con 0x7f83ec15b440 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83ea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 0x7f83ec0ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83ea59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 0x7f83ec0ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42938/0 (socket says 172.21.15.106:42938) 2023-11-24T08:10:27.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.756+0000 7f83ea59c640 1 -- 172.21.15.106:0/4024622849 learned_addr learned my addr 172.21.15.106:0/4024622849 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83ea59c640 1 -- 172.21.15.106:0/4024622849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83ec15b440 msgr2=0x7f83ec0b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83ea59c640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83ec15b440 0x7f83ec0b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83ea59c640 1 -- 172.21.15.106:0/4024622849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec0cbf20 msgr2=0x7f83ec0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83ea59c640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec0cbf20 0x7f83ec0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83ea59c640 1 -- 172.21.15.106:0/4024622849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc009400 con 0x7f83ec0c59b0 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83ea59c640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 0x7f83ec0ccbd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0097a0 tx=0x7f83dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc004030 con 0x7f83ec0c59b0 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc004190 con 0x7f83ec0c59b0 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc03b7e0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.757+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec15cff0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.758+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec15d570 con 0x7f83ec0c59b0 2023-11-24T08:10:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.759+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec04e8b0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.759+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f83dc043070 con 0x7f83ec0c59b0 2023-11-24T08:10:27.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.760+0000 7f83e2ffd640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f83cc06aed0 0x7f83cc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.760+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f83dc0ab330 con 0x7f83ec0c59b0 2023-11-24T08:10:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.761+0000 7f83e3fff640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f83cc06aed0 0x7f83cc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.761+0000 7f83e3fff640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f83cc06aed0 0x7f83cc06d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000f60 tx=0x7f83d40079d0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:27.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.765+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83dc0764e0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.958+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f83ec0c93c0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.968 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.966+0000 7f83e2ffd640 1 -- 172.21.15.106:0/4024622849 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f83dc0350c0 con 0x7f83ec0c59b0 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f83cc06aed0 msgr2=0x7f83cc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f83cc06aed0 0x7f83cc06d390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f83d4000f60 tx=0x7f83d40079d0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 msgr2=0x7f83ec0ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 0x7f83ec0ccbd0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0097a0 tx=0x7f83dc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 shutdown_connections 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f83cc06aed0 0x7f83cc06d390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f83ec15b440 0x7f83ec0b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec0cbf20 0x7f83ec0b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 --2- 172.21.15.106:0/4024622849 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f83ec0c59b0 0x7f83ec0ccbd0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:27.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 >> 172.21.15.106:0/4024622849 conn(0x7f83ec0b0c50 msgr2=0x7f83ec0717d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:27.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 shutdown_connections 2023-11-24T08:10:27.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:27.969+0000 7f83f0ea2640 1 -- 172.21.15.106:0/4024622849 wait complete. 2023-11-24T08:10:27.991 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:10:27.991 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:10:28.105 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:10:28.234 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:10:28.234 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:28.235 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:28.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:10:28.291 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:28.291 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:28.349 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:10:28.349 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:28.349 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:28.405 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:10:28.406 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:28.406 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:28.465 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:10:28.465 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:10:28.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:28.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.680+0000 7fb8bd075640 1 Processor -- start 2023-11-24T08:10:28.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.680+0000 7fb8bd075640 1 -- start start 2023-11-24T08:10:28.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8bd075640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 0x7fb8b80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8bd075640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b80cece0 con 0x7fb8b815c7f0 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8bd075640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8b806d040 con 0x7fb8b80cf450 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8bd075640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80cfe40 con 0x7fb8b80ce6c0 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8b6d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb8b815c7f0 0x7fb8b806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:36322/0 (socket says 172.21.15.106:36322) 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8b6d9d640 1 -- 172.21.15.106:0/4233359083 learned_addr learned my addr 172.21.15.106:0/4233359083 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1698030145 0 0) 0x7fb8b80cece0 con 0x7fb8b815c7f0 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8a4003520 con 0x7fb8b815c7f0 2023-11-24T08:10:28.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.682+0000 7fb8b759e640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 0x7fb8b80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2855908662 0 0) 0x7fb8a4003520 con 0x7fb8b815c7f0 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 msgr2=0x7fb8b80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 0x7fb8b80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 >> v1:172.21.15.156:6789/0 conn(0x7fb8b80cf450 legacy=0x7fb8b80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b806d320 con 0x7fb8b815c7f0 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fb8a8001670 con 0x7fb8b815c7f0 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 >> v1:172.21.15.106:6789/0 conn(0x7fb8b815c7f0 legacy=0x7fb8b806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a40039a0 0x7fb8a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 0x7fb8a40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a4004d00 0x7fb8a4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b80cfe40 con 0x7fb8a40039a0 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a400bb10 con 0x7fb8a4004d00 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a400bca0 con 0x7fb8b80ce6c0 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a40039a0 0x7fb8a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.683+0000 7fb8affff640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 0x7fb8a40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.684+0000 7fb8b6d9d640 1 -- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 msgr2=0x7fb8a40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.684+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8b80ce6c0 0x7fb8a40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.684+0000 7fb8b6d9d640 1 -- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a4004d00 msgr2=0x7fb8a4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:28.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.684+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8a4004d00 0x7fb8a4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.684+0000 7fb8b6d9d640 1 -- 172.21.15.106:0/4233359083 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a4003520 con 0x7fb8a40039a0 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.685+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a40039a0 0x7fb8a4003dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c270 tx=0x7fb8a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122afb3abefc2458 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.685+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a8005ad0 con 0x7fb8a40039a0 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.685+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a8005c30 con 0x7fb8a40039a0 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.685+0000 7fb8b659c640 1 -- 172.21.15.106:0/4233359083 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a802c8a0 con 0x7fb8a40039a0 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.685+0000 7fb8bd075640 1 -- 172.21.15.106:0/4233359083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a40039a0 msgr2=0x7fb8a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.686+0000 7fb8bd075640 1 --2- 172.21.15.106:0/4233359083 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8a40039a0 0x7fb8a4003dc0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fb8a800c270 tx=0x7fb8a40057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.686+0000 7fb8b759e640 1 -- 172.21.15.106:0/4233359083 reap_dead start 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.686+0000 7fb8bd075640 1 -- 172.21.15.106:0/4233359083 shutdown_connections 2023-11-24T08:10:28.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.686+0000 7fb8bd075640 1 -- 172.21.15.106:0/4233359083 >> 172.21.15.106:0/4233359083 conn(0x7fb8b806a920 msgr2=0x7fb8b806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.686+0000 7fb8bd075640 1 -- 172.21.15.106:0/4233359083 shutdown_connections 2023-11-24T08:10:28.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.686+0000 7fb8bd075640 1 -- 172.21.15.106:0/4233359083 wait complete. 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.688+0000 7fb8bd075640 1 Processor -- start 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.688+0000 7fb8bd075640 1 -- start start 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.689+0000 7fb8bd075640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40039a0 0x7fb8b8157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.689+0000 7fb8bd075640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8b80b33a0 0x7fb8b8155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.689+0000 7fb8bd075640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 0x7fb8b8147ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.689+0000 7fb8bd075640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8156860 con 0x7fb8b80b3800 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.689+0000 7fb8bd075640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8156b40 con 0x7fb8b80b33a0 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8bd075640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b8156e20 con 0x7fb8a40039a0 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 0x7fb8b8147ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 0x7fb8b8147ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42954/0 (socket says 172.21.15.106:42954) 2023-11-24T08:10:28.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 -- 172.21.15.106:0/3362587599 learned_addr learned my addr 172.21.15.106:0/3362587599 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40039a0 0x7fb8b8157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8affff640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8b80b33a0 0x7fb8b8155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 -- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40039a0 msgr2=0x7fb8b8157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40039a0 0x7fb8b8157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 -- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8b80b33a0 msgr2=0x7fb8b8155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8b80b33a0 0x7fb8b8155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b759e640 1 -- 172.21.15.106:0/3362587599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8b8157100 con 0x7fb8b80b3800 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.690+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40039a0 0x7fb8b8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.691+0000 7fb8b759e640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 0x7fb8b8147ec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80b5270 tx=0x7fb8b0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.691+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b0018070 con 0x7fb8b80b3800 2023-11-24T08:10:28.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.691+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b000ea80 con 0x7fb8b80b3800 2023-11-24T08:10:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.691+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b000ee50 con 0x7fb8b80b3800 2023-11-24T08:10:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.691+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b8155540 con 0x7fb8b80b3800 2023-11-24T08:10:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.691+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b8156500 con 0x7fb8b80b3800 2023-11-24T08:10:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.692+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b8067470 con 0x7fb8b80b3800 2023-11-24T08:10:28.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.693+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fb8b0014420 con 0x7fb8b80b3800 2023-11-24T08:10:28.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.694+0000 7fb8af7fe640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb89806af00 0x7fb89806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:28.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.694+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb8b0088f40 con 0x7fb8b80b3800 2023-11-24T08:10:28.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.694+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb89806af00 0x7fb89806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:28.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.695+0000 7fb8b6d9d640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb89806af00 0x7fb89806d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8034000 tx=0x7fb8a8009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:28.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.699+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8b0054200 con 0x7fb8b80b3800 2023-11-24T08:10:28.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.925+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8b80cc260 con 0x7fb8b80b3800 2023-11-24T08:10:28.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.926+0000 7fb8af7fe640 1 -- 172.21.15.106:0/3362587599 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3708 (secure 0 0 0) 0x7fb8b0057820 con 0x7fb8b80b3800 2023-11-24T08:10:28.929 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:28.929 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4523,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3188370300","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3188370300},{"type":"v1","addr":"172.21.15.106:6837","nonce":3188370300}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4687,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/246710320","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":246710320},{"type":"v1","addr":"172.21.15.156:6833","nonce":246710320}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":4693,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2098095436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2098095436},{"type":"v1","addr":"172.21.15.156:6835","nonce":2098095436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10},{"gid":5472,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/2039168934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":2039168934},{"type":"v1","addr":"172.21.15.106:6836","nonce":2039168934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":17}],"filesystems":[]} 2023-11-24T08:10:28.929 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 17 2023-11-24T08:10:28.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.928+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb89806af00 msgr2=0x7fb89806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.928+0000 7fb8bd075640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb89806af00 0x7fb89806d3c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb8a8034000 tx=0x7fb8a8009030 comp rx=0 tx=0).stop 2023-11-24T08:10:28.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.928+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 msgr2=0x7fb8b8147ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.928+0000 7fb8bd075640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 0x7fb8b8147ec0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb8b80b5270 tx=0x7fb8b0007f40 comp rx=0 tx=0).stop 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 shutdown_connections 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8b80b3800 0x7fb8b8147ec0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8b80b33a0 0x7fb8b8155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb89806af00 0x7fb89806d3c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 --2- 172.21.15.106:0/3362587599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8a40039a0 0x7fb8b8157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 >> 172.21.15.106:0/3362587599 conn(0x7fb8b806a920 msgr2=0x7fb8b815df30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 shutdown_connections 2023-11-24T08:10:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:28.929+0000 7fb8bd075640 1 -- 172.21.15.106:0/3362587599 wait complete. 2023-11-24T08:10:28.951 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:10:28.951 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:10:28.951 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:10:28.951 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:10:28.951 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:10:28.951 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:10:28.952 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:10:28.952 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:10:28.952 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:10:28.952 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:10:28.952 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:10:28.952 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:10:28.953 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:10:28.953 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:10:28.953 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:10:28.953 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:10:28.973 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:10:28.973 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:10:28.976 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:10:28.976 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:10:28.980 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:10:28.980 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:10:29.025 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:10:29.025 DEBUG:teuthology.parallel:result is None 2023-11-24T08:10:29.029 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:10:29.029 DEBUG:teuthology.parallel:result is None 2023-11-24T08:10:29.036 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:10:29.036 DEBUG:teuthology.parallel:result is None 2023-11-24T08:10:29.191 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:10:29.195 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-11-24T08:10:29.206 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:10:34.976 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:10:34.977 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:10:34.986 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:10:34.987 DEBUG:teuthology.parallel:result is None 2023-11-24T08:10:34.987 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:10:35.149 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:10:35.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.176+0000 7fc46b59e640 1 Processor -- start 2023-11-24T08:10:35.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.176+0000 7fc46b59e640 1 -- start start 2023-11-24T08:10:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.177+0000 7fc46b59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc46c0d3940 0x7fc46c167ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.177+0000 7fc46b59e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c0cea30 con 0x7fc46c0b63d0 2023-11-24T08:10:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.177+0000 7fc46b59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c0d2260 con 0x7fc46c15d490 2023-11-24T08:10:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.177+0000 7fc46b59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c0d0450 con 0x7fc46c0d3940 2023-11-24T08:10:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc463fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc46c0b63d0 0x7fc46c0d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55222/0 (socket says 172.21.15.106:55222) 2023-11-24T08:10:35.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc463fff640 1 -- 172.21.15.106:0/2441837988 learned_addr learned my addr 172.21.15.106:0/2441837988 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:35.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818682037 0 0) 0x7fc46c0cea30 con 0x7fc46c0b63d0 2023-11-24T08:10:35.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc458003520 con 0x7fc46c0b63d0 2023-11-24T08:10:35.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc46ad9d640 1 --2- 172.21.15.106:0/2441837988 >> v2:172.21.15.156:3300/0 conn(0x7fc46c0d3940 0x7fc46c167ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:35.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1653901633 0 0) 0x7fc458003520 con 0x7fc46c0b63d0 2023-11-24T08:10:35.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 >> v2:172.21.15.156:3300/0 conn(0x7fc46c0d3940 msgr2=0x7fc46c167ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:35.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 --2- 172.21.15.106:0/2441837988 >> v2:172.21.15.156:3300/0 conn(0x7fc46c0d3940 0x7fc46c167ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 >> v1:172.21.15.156:6789/0 conn(0x7fc46c15d490 legacy=0x7fc46c0cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:35.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.178+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c0ce750 con 0x7fc46c0b63d0 2023-11-24T08:10:35.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc454001670 con 0x7fc46c0b63d0 2023-11-24T08:10:35.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 >> v1:172.21.15.106:6789/0 conn(0x7fc46c0b63d0 legacy=0x7fc46c0d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:35.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc458003970 0x7fc458003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc458004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc458005180 0x7fc45800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0d0450 con 0x7fc4580047c0 2023-11-24T08:10:35.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45800c0b0 con 0x7fc458003970 2023-11-24T08:10:35.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc45800c270 con 0x7fc458005180 2023-11-24T08:10:35.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc463fff640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc458004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc46ad9d640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc458005180 0x7fc45800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc463fff640 1 -- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc458005180 msgr2=0x7fc45800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc463fff640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc458005180 0x7fc45800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc46a59c640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc458003970 0x7fc458003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc463fff640 1 -- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc458003970 msgr2=0x7fc458003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc463fff640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc458003970 0x7fc458003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc46ad9d640 1 -- 172.21.15.106:0/2441837988 reap_dead start 2023-11-24T08:10:35.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc463fff640 1 -- 172.21.15.106:0/2441837988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc458003520 con 0x7fc4580047c0 2023-11-24T08:10:35.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.179+0000 7fc46a59c640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc458003970 0x7fc458003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:35.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc463fff640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc458004c40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc454001eb0 tx=0x7fc4580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260285b14724e273 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454005ad0 con 0x7fc4580047c0 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454005c50 con 0x7fc4580047c0 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc469d9b640 1 -- 172.21.15.106:0/2441837988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45402c850 con 0x7fc4580047c0 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc46b59e640 1 -- 172.21.15.106:0/2441837988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 msgr2=0x7fc458004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc46b59e640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc458004c40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc454001eb0 tx=0x7fc4580057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc46b59e640 1 -- 172.21.15.106:0/2441837988 shutdown_connections 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.181+0000 7fc46b59e640 1 --2- 172.21.15.106:0/2441837988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc458004c40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.182+0000 7fc46b59e640 1 -- 172.21.15.106:0/2441837988 >> 172.21.15.106:0/2441837988 conn(0x7fc46c061340 msgr2=0x7fc46c061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.182+0000 7fc46b59e640 1 -- 172.21.15.106:0/2441837988 shutdown_connections 2023-11-24T08:10:35.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.182+0000 7fc46b59e640 1 -- 172.21.15.106:0/2441837988 wait complete. 2023-11-24T08:10:35.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.183+0000 7fc46b59e640 1 Processor -- start 2023-11-24T08:10:35.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.183+0000 7fc46b59e640 1 -- start start 2023-11-24T08:10:35.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.184+0000 7fc46b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc46c0ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.184+0000 7fc46b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c154e30 0x7fc46c0cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.184+0000 7fc46b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc46c0b7bc0 0x7fc46c0b8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.184+0000 7fc46b59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0d0bf0 con 0x7fc4580047c0 2023-11-24T08:10:35.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.184+0000 7fc46b59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0b8590 con 0x7fc46c154e30 2023-11-24T08:10:35.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.184+0000 7fc46b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0c7140 con 0x7fc46c0b7bc0 2023-11-24T08:10:35.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc46c0ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:35.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc46c0ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35172/0 (socket says 172.21.15.106:35172) 2023-11-24T08:10:35.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 -- 172.21.15.106:0/198292565 learned_addr learned my addr 172.21.15.106:0/198292565 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:35.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 -- 172.21.15.106:0/198292565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc46c0b7bc0 msgr2=0x7fc46c0b8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:35.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc46c0b7bc0 0x7fc46c0b8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 -- 172.21.15.106:0/198292565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c154e30 msgr2=0x7fc46c0cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:35.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c154e30 0x7fc46c0cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:35.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 -- 172.21.15.106:0/198292565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c15e240 con 0x7fc4580047c0 2023-11-24T08:10:35.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.185+0000 7fc46a59c640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc46c0ccd40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00d430 tx=0x7fc45c00d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:35.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.186+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c007020 con 0x7fc4580047c0 2023-11-24T08:10:35.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.186+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45c007180 con 0x7fc4580047c0 2023-11-24T08:10:35.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.186+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c15d910 con 0x7fc4580047c0 2023-11-24T08:10:35.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.186+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45c007530 con 0x7fc4580047c0 2023-11-24T08:10:35.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.186+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c15ded0 con 0x7fc4580047c0 2023-11-24T08:10:35.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.187+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c0b4b60 con 0x7fc4580047c0 2023-11-24T08:10:35.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.188+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc45c0067d0 con 0x7fc4580047c0 2023-11-24T08:10:35.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.189+0000 7fc462ffd640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc44c06af00 0x7fc44c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:35.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.189+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc45c089210 con 0x7fc4580047c0 2023-11-24T08:10:35.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.189+0000 7fc463fff640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc44c06af00 0x7fc44c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:35.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.190+0000 7fc463fff640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc44c06af00 0x7fc44c06d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc454034000 tx=0x7fc454039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:35.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.194+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc45c0544c0 con 0x7fc4580047c0 2023-11-24T08:10:35.350 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-24T08:10:35.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:35.397+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc46c04e840 con 0x7fc4580047c0 2023-11-24T08:10:36.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.324+0000 7fc462ffd640 1 -- 172.21.15.106:0/198292565 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v44) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc45c057ae0 con 0x7fc4580047c0 2023-11-24T08:10:36.326 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2023-11-24T08:10:36.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc44c06af00 msgr2=0x7fc44c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc44c06af00 0x7fc44c06d3c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc454034000 tx=0x7fc454039040 comp rx=0 tx=0).stop 2023-11-24T08:10:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 msgr2=0x7fc46c0ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc46c0ccd40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc45c00d430 tx=0x7fc45c00d8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 shutdown_connections 2023-11-24T08:10:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc44c06af00 0x7fc44c06d3c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc46c0b7bc0 0x7fc46c0b8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc46c154e30 0x7fc46c0cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 --2- 172.21.15.106:0/198292565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc4580047c0 0x7fc46c0ccd40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 >> 172.21.15.106:0/198292565 conn(0x7fc46c061340 msgr2=0x7fc46c160e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 shutdown_connections 2023-11-24T08:10:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.328+0000 7fc46b59e640 1 -- 172.21.15.106:0/198292565 wait complete. 2023-11-24T08:10:36.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:10:36.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.555+0000 7f472123d640 1 Processor -- start 2023-11-24T08:10:36.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.555+0000 7f472123d640 1 -- start start 2023-11-24T08:10:36.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.556+0000 7f472123d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f471c15b5e0 0x7f471c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:36.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f472123d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f471c0abf50 0x7f471c0b4000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:36.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f472123d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c0711e0 con 0x7f471c0ccea0 2023-11-24T08:10:36.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f472123d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471c0b4540 con 0x7f471c0abf50 2023-11-24T08:10:36.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f472123d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471c0cdc80 con 0x7f471c15b5e0 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f471c0abf50 0x7f471c0b4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f471c0abf50 0x7f471c0b4000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35178/0 (socket says 172.21.15.106:35178) 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 -- 172.21.15.106:0/3252654169 learned_addr learned my addr 172.21.15.106:0/3252654169 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 -- 172.21.15.106:0/3252654169 >> v2:172.21.15.156:3300/0 conn(0x7f471c15b5e0 msgr2=0x7f471c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 --2- 172.21.15.106:0/3252654169 >> v2:172.21.15.156:3300/0 conn(0x7f471c15b5e0 0x7f471c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 -- 172.21.15.106:0/3252654169 >> v1:172.21.15.156:6789/0 conn(0x7f471c0ccea0 legacy=0x7f471c15f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:36.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.557+0000 7f471a59c640 1 -- 172.21.15.106:0/3252654169 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c0705d0 con 0x7f471c0abf50 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.559+0000 7f471a59c640 1 --2- 172.21.15.106:0/3252654169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0abf50 0x7f471c0b4000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f471c06bcc0 tx=0x7f471002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c99153bda4511 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.559+0000 7f4719d9b640 1 -- 172.21.15.106:0/3252654169 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710031870 con 0x7f471c0abf50 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.559+0000 7f4719d9b640 1 -- 172.21.15.106:0/3252654169 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47100319d0 con 0x7f471c0abf50 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.559+0000 7f4719d9b640 1 -- 172.21.15.106:0/3252654169 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710031d60 con 0x7f471c0abf50 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 -- 172.21.15.106:0/3252654169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0abf50 msgr2=0x7f471c0b4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 --2- 172.21.15.106:0/3252654169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0abf50 0x7f471c0b4000 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f471c06bcc0 tx=0x7f471002b910 comp rx=0 tx=0).stop 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 -- 172.21.15.106:0/3252654169 shutdown_connections 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 --2- 172.21.15.106:0/3252654169 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0abf50 0x7f471c0b4000 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 --2- 172.21.15.106:0/3252654169 >> v2:172.21.15.156:3300/0 conn(0x7f471c15b5e0 0x7f471c0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 -- 172.21.15.106:0/3252654169 >> 172.21.15.106:0/3252654169 conn(0x7f471c0b0c70 msgr2=0x7f471c0b1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 -- 172.21.15.106:0/3252654169 shutdown_connections 2023-11-24T08:10:36.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.560+0000 7f472123d640 1 -- 172.21.15.106:0/3252654169 wait complete. 2023-11-24T08:10:36.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.562+0000 7f472123d640 1 Processor -- start 2023-11-24T08:10:36.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.562+0000 7f472123d640 1 -- start start 2023-11-24T08:10:36.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f472123d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c0abf50 0x7f471c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:36.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f472123d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 0x7f471c148fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:36.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f472123d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f471c15b5e0 0x7f471c1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:36.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f472123d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0cb570 con 0x7f471c0ccea0 2023-11-24T08:10:36.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f472123d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0cb850 con 0x7f471c0abf50 2023-11-24T08:10:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f472123d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0cbb30 con 0x7f471c15b5e0 2023-11-24T08:10:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 0x7f471c148fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 0x7f471c148fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35194/0 (socket says 172.21.15.106:35194) 2023-11-24T08:10:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 -- 172.21.15.106:0/668770389 learned_addr learned my addr 172.21.15.106:0/668770389 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471ad9d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c0abf50 0x7f471c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:36.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 -- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f471c15b5e0 msgr2=0x7f471c1494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f471c15b5e0 0x7f471c1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 -- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c0abf50 msgr2=0x7f471c15d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c0abf50 0x7f471c15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.563+0000 7f471a59c640 1 -- 172.21.15.106:0/668770389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47100094d0 con 0x7f471c0ccea0 2023-11-24T08:10:36.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.564+0000 7f471a59c640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 0x7f471c148fb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4710000c00 tx=0x7f47100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:36.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.564+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710031840 con 0x7f471c0ccea0 2023-11-24T08:10:36.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.564+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47100443f0 con 0x7f471c0ccea0 2023-11-24T08:10:36.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.564+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47100447a0 con 0x7f471c0ccea0 2023-11-24T08:10:36.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.564+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c0cbe10 con 0x7f471c0ccea0 2023-11-24T08:10:36.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.564+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c15d950 con 0x7f471c0ccea0 2023-11-24T08:10:36.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.565+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c04e8b0 con 0x7f471c0ccea0 2023-11-24T08:10:36.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.566+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f471004d450 con 0x7f471c0ccea0 2023-11-24T08:10:36.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.567+0000 7f47037fe640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f470806af00 0x7f470806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:36.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.567+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f471002d040 con 0x7f471c0ccea0 2023-11-24T08:10:36.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.568+0000 7f471ad9d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f470806af00 0x7f470806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:36.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.568+0000 7f471ad9d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f470806af00 0x7f470806d3c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f470c00a770 tx=0x7f470c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:36.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.572+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4710076d00 con 0x7f471c0ccea0 2023-11-24T08:10:36.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.811+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f471c15d950 con 0x7f471c0ccea0 2023-11-24T08:10:36.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.812+0000 7f47037fe640 1 -- 172.21.15.106:0/668770389 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f471c15d950 con 0x7f471c0ccea0 2023-11-24T08:10:36.814 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:36.814 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:10:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2023-11-24T08:10:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2023-11-24T08:10:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A==", 2023-11-24T08:10:36.815 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:10:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:10:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.816 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2023-11-24T08:10:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg==", 2023-11-24T08:10:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.817 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:10:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:10:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.818 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2023-11-24T08:10:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q==", 2023-11-24T08:10:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:10:36.819 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:10:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2023-11-24T08:10:36.820 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw==", 2023-11-24T08:10:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:10:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:10:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.821 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA==", 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.822 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.823 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2023-11-24T08:10:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA==", 2023-11-24T08:10:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.824 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2023-11-24T08:10:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA==", 2023-11-24T08:10:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.825 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.826 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2023-11-24T08:10:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw==", 2023-11-24T08:10:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.827 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2023-11-24T08:10:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw==", 2023-11-24T08:10:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.828 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2023-11-24T08:10:36.829 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw==", 2023-11-24T08:10:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.830 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA==", 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.831 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug==", 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.832 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ==", 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:10:36.833 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg==", 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.834 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg==", 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.835 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w==", 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.836 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBludbKAhAARxKqsuG6+yx4M/G/4IloBw==", 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2023-11-24T08:10:36.837 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ==", 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA==", 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:10:36.838 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:10:36.839 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:10:36.839 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.839 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:36.839 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2023-11-24T08:10:36.839 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:10:36.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f470806af00 msgr2=0x7f470806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f470806af00 0x7f470806d3c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f470c00a770 tx=0x7f470c008040 comp rx=0 tx=0).stop 2023-11-24T08:10:36.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 msgr2=0x7f471c148fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:36.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 0x7f471c148fb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4710000c00 tx=0x7f47100046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 shutdown_connections 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f470806af00 0x7f470806d3c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f471c15b5e0 0x7f471c1494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f471c0ccea0 0x7f471c148fb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 --2- 172.21.15.106:0/668770389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f471c0abf50 0x7f471c15d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 >> 172.21.15.106:0/668770389 conn(0x7f471c0b0c70 msgr2=0x7f471c0aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 shutdown_connections 2023-11-24T08:10:36.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:36.815+0000 7f472123d640 1 -- 172.21.15.106:0/668770389 wait complete. 2023-11-24T08:10:36.842 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.042+0000 7f5169a72640 1 Processor -- start 2023-11-24T08:10:37.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.042+0000 7f5169a72640 1 -- start start 2023-11-24T08:10:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.043+0000 7f5169a72640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5164161080 0x7f51640b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.043+0000 7f5169a72640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5164161510 con 0x7f5164143870 2023-11-24T08:10:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.043+0000 7f5169a72640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51641617f0 con 0x7f51640b0fb0 2023-11-24T08:10:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.043+0000 7f5169a72640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5164070460 con 0x7f5164161080 2023-11-24T08:10:37.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f5168a70640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5164143870 0x7f5164146970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55230/0 (socket says 172.21.15.106:55230) 2023-11-24T08:10:37.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f5168a70640 1 -- 172.21.15.106:0/3059038942 learned_addr learned my addr 172.21.15.106:0/3059038942 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37199175 0 0) 0x7f5164161510 con 0x7f5164143870 2023-11-24T08:10:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5154003520 con 0x7f5164143870 2023-11-24T08:10:37.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f5163fff640 1 --2- 172.21.15.106:0/3059038942 >> v2:172.21.15.156:3300/0 conn(0x7f5164161080 0x7f51640b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 938424576 0 0) 0x7f5154003520 con 0x7f5164143870 2023-11-24T08:10:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 >> v2:172.21.15.156:3300/0 conn(0x7f5164161080 msgr2=0x7f51640b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 --2- 172.21.15.106:0/3059038942 >> v2:172.21.15.156:3300/0 conn(0x7f5164161080 0x7f51640b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 >> v1:172.21.15.156:6789/0 conn(0x7f51640b0fb0 legacy=0x7f51640cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5164161ad0 con 0x7f5164143870 2023-11-24T08:10:37.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f5158001670 con 0x7f5164143870 2023-11-24T08:10:37.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 >> v1:172.21.15.106:6789/0 conn(0x7f5164143870 legacy=0x7f5164146970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.044+0000 7f51637fe640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540039c0 0x7f5154003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f51637fe640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540047c0 0x7f5154004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f51637fe640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51540051d0 0x7f515400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5164070460 con 0x7f51540051d0 2023-11-24T08:10:37.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515400c100 con 0x7f51540047c0 2023-11-24T08:10:37.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515400c2c0 con 0x7f51540039c0 2023-11-24T08:10:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5169271640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51540051d0 0x7f515400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5163fff640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540047c0 0x7f5154004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5168a70640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540039c0 0x7f5154003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5169271640 1 -- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540039c0 msgr2=0x7f5154003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5169271640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540039c0 0x7f5154003dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5169271640 1 -- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540047c0 msgr2=0x7f5154004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5169271640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540047c0 0x7f5154004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5169271640 1 -- 172.21.15.106:0/3059038942 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5154003520 con 0x7f51540051d0 2023-11-24T08:10:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.045+0000 7f5163fff640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540047c0 0x7f5154004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.046+0000 7f5169271640 1 -- 172.21.15.106:0/3059038942 reap_dead start 2023-11-24T08:10:37.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.046+0000 7f5169271640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51540051d0 0x7f515400b980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f515c004480 tx=0x7f51540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62184572468f15d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.046+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c006b50 con 0x7f51540051d0 2023-11-24T08:10:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.046+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c006cd0 con 0x7f51540051d0 2023-11-24T08:10:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f51637fe640 1 -- 172.21.15.106:0/3059038942 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c007260 con 0x7f51540051d0 2023-11-24T08:10:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 -- 172.21.15.106:0/3059038942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51540051d0 msgr2=0x7f515400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51540051d0 0x7f515400b980 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f515c004480 tx=0x7f51540057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 -- 172.21.15.106:0/3059038942 shutdown_connections 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 --2- 172.21.15.106:0/3059038942 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f51540051d0 0x7f515400b980 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 -- 172.21.15.106:0/3059038942 >> 172.21.15.106:0/3059038942 conn(0x7f51640cb830 msgr2=0x7f51640cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 -- 172.21.15.106:0/3059038942 shutdown_connections 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.047+0000 7f5169a72640 1 -- 172.21.15.106:0/3059038942 wait complete. 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.049+0000 7f5169a72640 1 Processor -- start 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.049+0000 7f5169a72640 1 -- start start 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169a72640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540051d0 0x7f51640af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169a72640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 0x7f516415db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169a72640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5164134680 0x7f5164146830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169a72640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516415ee60 con 0x7f516415f2f0 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169a72640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516415e250 con 0x7f5164134680 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169a72640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516415e530 con 0x7f51540051d0 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5163fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 0x7f516415db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5163fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 0x7f516415db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35204/0 (socket says 172.21.15.106:35204) 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5163fff640 1 -- 172.21.15.106:0/1151511430 learned_addr learned my addr 172.21.15.106:0/1151511430 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5163fff640 1 -- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540051d0 msgr2=0x7f51640af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5163fff640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540051d0 0x7f51640af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5169271640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5164134680 0x7f5164146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.050+0000 7f5163fff640 1 -- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5164134680 msgr2=0x7f5164146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f5163fff640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5164134680 0x7f5164146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f5163fff640 1 -- 172.21.15.106:0/1151511430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516415e810 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f5163fff640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 0x7f516415db50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f515000e7e0 tx=0x7f515000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515000c6d0 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515000c830 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515000cba0 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516415eaf0 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.051+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5164146d70 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.052+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5164067990 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.053+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f515000cd00 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.054+0000 7f51617fa640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f514806aed0 0x7f514806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.054+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5150089220 con 0x7f516415f2f0 2023-11-24T08:10:37.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.054+0000 7f5168a70640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f514806aed0 0x7f514806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.055+0000 7f5168a70640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f514806aed0 0x7f514806d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5158000fd0 tx=0x7f5158009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.058+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f51500544b0 con 0x7f516415f2f0 2023-11-24T08:10:37.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.233+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f516404e840 con 0x7f516415f2f0 2023-11-24T08:10:37.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.235+0000 7f51617fa640 1 -- 172.21.15.106:0/1151511430 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f5150057ad0 con 0x7f516415f2f0 2023-11-24T08:10:37.236 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:37.236 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:10:36.315565+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:37.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f514806aed0 msgr2=0x7f514806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f514806aed0 0x7f514806d390 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5158000fd0 tx=0x7f5158009040 comp rx=0 tx=0).stop 2023-11-24T08:10:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 msgr2=0x7f516415db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 0x7f516415db50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f515000e7e0 tx=0x7f515000eca0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 shutdown_connections 2023-11-24T08:10:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5164134680 0x7f5164146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f516415f2f0 0x7f516415db50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f514806aed0 0x7f514806d390 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 --2- 172.21.15.106:0/1151511430 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f51540051d0 0x7f51640af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.237+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 >> 172.21.15.106:0/1151511430 conn(0x7f51640cb830 msgr2=0x7f5164142ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.238+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 shutdown_connections 2023-11-24T08:10:37.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.238+0000 7f5169a72640 1 -- 172.21.15.106:0/1151511430 wait complete. 2023-11-24T08:10:37.256 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:10:37.256 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:10:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.467+0000 7f1bbe61e640 1 Processor -- start 2023-11-24T08:10:37.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.468+0000 7f1bbe61e640 1 -- start start 2023-11-24T08:10:37.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.469+0000 7f1bbe61e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bb80ccbd0 0x7f1bb81660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.469+0000 7f1bbe61e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 0x7f1bb8162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.469+0000 7f1bbe61e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1bb81655f0 con 0x7f1bb8162540 2023-11-24T08:10:37.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.469+0000 7f1bbe61e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bb80cc380 con 0x7f1bb80ccbd0 2023-11-24T08:10:37.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.469+0000 7f1bbe61e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8162b30 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.470+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 0x7f1bb8162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.470+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 0x7f1bb8162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:58946/0 (socket says 172.21.15.106:58946) 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.470+0000 7f1bb77fe640 1 -- 172.21.15.106:0/515422450 learned_addr learned my addr 172.21.15.106:0/515422450 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.470+0000 7f1bb7fff640 1 --2- 172.21.15.106:0/515422450 >> v2:172.21.15.156:3300/0 conn(0x7f1bb80ccbd0 0x7f1bb81660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.471+0000 7f1bb77fe640 1 -- 172.21.15.106:0/515422450 >> v2:172.21.15.156:3300/0 conn(0x7f1bb80ccbd0 msgr2=0x7f1bb81660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.471+0000 7f1bb77fe640 1 --2- 172.21.15.106:0/515422450 >> v2:172.21.15.156:3300/0 conn(0x7f1bb80ccbd0 0x7f1bb81660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.471+0000 7f1bb77fe640 1 -- 172.21.15.106:0/515422450 >> v1:172.21.15.156:6789/0 conn(0x7f1bb8162540 legacy=0x7f1bb80c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.471+0000 7f1bb77fe640 1 -- 172.21.15.106:0/515422450 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bb8162e10 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.471+0000 7f1bb6ffd640 1 -- 172.21.15.106:0/515422450 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 905523605 0 0) 0x7f1bb81655f0 con 0x7f1bb8162540 2023-11-24T08:10:37.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.472+0000 7f1bb77fe640 1 --2- 172.21.15.106:0/515422450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 0x7f1bb8162000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00098a0 tx=0x7f1ba002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=253de83e71bdc711 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.473+0000 7f1bb6ffd640 1 -- 172.21.15.106:0/515422450 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0030840 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.473+0000 7f1bb6ffd640 1 -- 172.21.15.106:0/515422450 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba00309a0 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.473+0000 7f1bb6ffd640 1 -- 172.21.15.106:0/515422450 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0030d30 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 -- 172.21.15.106:0/515422450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 msgr2=0x7f1bb8162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/515422450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 0x7f1bb8162000 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1ba00098a0 tx=0x7f1ba002b980 comp rx=0 tx=0).stop 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 -- 172.21.15.106:0/515422450 shutdown_connections 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/515422450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccfb0 0x7f1bb8162000 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/515422450 >> v2:172.21.15.156:3300/0 conn(0x7f1bb80ccbd0 0x7f1bb81660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 -- 172.21.15.106:0/515422450 >> 172.21.15.106:0/515422450 conn(0x7f1bb80b0c50 msgr2=0x7f1bb80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 -- 172.21.15.106:0/515422450 shutdown_connections 2023-11-24T08:10:37.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.474+0000 7f1bbe61e640 1 -- 172.21.15.106:0/515422450 wait complete. 2023-11-24T08:10:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.476+0000 7f1bbe61e640 1 Processor -- start 2023-11-24T08:10:37.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.476+0000 7f1bbe61e640 1 -- start start 2023-11-24T08:10:37.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbe61e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccbd0 0x7f1bb81ed400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbe61e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 0x7f1bb81647e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbe61e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb8162540 0x7f1bb8163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbe61e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8163b10 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbe61e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb8163df0 con 0x7f1bb8162540 2023-11-24T08:10:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbe61e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb81ec920 con 0x7f1bb80ccbd0 2023-11-24T08:10:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 0x7f1bb81647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 0x7f1bb81647e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35206/0 (socket says 172.21.15.106:35206) 2023-11-24T08:10:37.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 -- 172.21.15.106:0/1236809324 learned_addr learned my addr 172.21.15.106:0/1236809324 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bbcbbb640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb8162540 0x7f1bb8163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 -- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccbd0 msgr2=0x7f1bb81ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccbd0 0x7f1bb81ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 -- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb8162540 msgr2=0x7f1bb8163050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb8162540 0x7f1bb8163050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.477+0000 7f1bb77fe640 1 -- 172.21.15.106:0/1236809324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba00094d0 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.478+0000 7f1bb77fe640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 0x7f1bb81647e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000c00 tx=0x7f1ba0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.478+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0047070 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.478+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ba0039a90 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.478+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba0039e40 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.478+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb81ecc00 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.478+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb81587d0 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.480+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bb81ed120 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.480+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1ba004c410 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.481+0000 7f1bb4ff9640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.481+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1ba003e070 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.481+0000 7f1bb7fff640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.482+0000 7f1bb7fff640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b9806aed0 0x7f1b9806d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8006a90 tx=0x7f1ba8006a20 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.486+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1bb81ed120 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.676+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1bb81ed120 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.677+0000 7f1bb4ff9640 1 -- 172.21.15.106:0/1236809324 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v44) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bb81ed120 con 0x7f1bb80ccfb0 2023-11-24T08:10:37.679 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:10:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b9806aed0 msgr2=0x7f1b9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b9806aed0 0x7f1b9806d390 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f1ba8006a90 tx=0x7f1ba8006a20 comp rx=0 tx=0).stop 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 msgr2=0x7f1bb81647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 0x7f1bb81647e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f1ba0000c00 tx=0x7f1ba0004730 comp rx=0 tx=0).stop 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 shutdown_connections 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb8162540 0x7f1bb8163050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bb80ccfb0 0x7f1bb81647e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1b9806aed0 0x7f1b9806d390 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 --2- 172.21.15.106:0/1236809324 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bb80ccbd0 0x7f1bb81ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 >> 172.21.15.106:0/1236809324 conn(0x7f1bb80b0c50 msgr2=0x7f1bb80ad620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 shutdown_connections 2023-11-24T08:10:37.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.680+0000 7f1bbe61e640 1 -- 172.21.15.106:0/1236809324 wait complete. 2023-11-24T08:10:37.702 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:10:37.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.909+0000 7f6c10940640 1 Processor -- start 2023-11-24T08:10:37.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.909+0000 7f6c10940640 1 -- start start 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.910+0000 7f6c10940640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6c0c0703f0 0x7f6c0c162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.910+0000 7f6c10940640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c0cd160 0x7f6c0c0cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.910+0000 7f6c10940640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c162eb0 con 0x7f6c0c14b100 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.910+0000 7f6c10940640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c1654a0 con 0x7f6c0c0703f0 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c10940640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0ccf00 con 0x7f6c0c0cd160 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6c0c0703f0 0x7f6c0c162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6c0c0703f0 0x7f6c0c162970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35222/0 (socket says 172.21.15.106:35222) 2023-11-24T08:10:37.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 -- 172.21.15.106:0/2713082068 learned_addr learned my addr 172.21.15.106:0/2713082068 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:37.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0b59e640 1 --2- 172.21.15.106:0/2713082068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c0cd160 0x7f6c0c0cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 -- 172.21.15.106:0/2713082068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c0cd160 msgr2=0x7f6c0c0cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 --2- 172.21.15.106:0/2713082068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c0cd160 0x7f6c0c0cd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 -- 172.21.15.106:0/2713082068 >> v1:172.21.15.156:6789/0 conn(0x7f6c0c14b100 legacy=0x7f6c0c165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:37.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.911+0000 7f6c0a59c640 1 -- 172.21.15.106:0/2713082068 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c166320 con 0x7f6c0c0703f0 2023-11-24T08:10:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.913+0000 7f6c0a59c640 1 --2- 172.21.15.106:0/2713082068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c162970 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009870 tx=0x7f6bf402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=814c1e6cd394a108 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.913+0000 7f6c09d9b640 1 -- 172.21.15.106:0/2713082068 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf40317c0 con 0x7f6c0c0703f0 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.913+0000 7f6c09d9b640 1 -- 172.21.15.106:0/2713082068 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf4031920 con 0x7f6c0c0703f0 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.913+0000 7f6c09d9b640 1 -- 172.21.15.106:0/2713082068 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4031cb0 con 0x7f6c0c0703f0 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 -- 172.21.15.106:0/2713082068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0703f0 msgr2=0x7f6c0c162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 --2- 172.21.15.106:0/2713082068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c162970 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009870 tx=0x7f6bf402b860 comp rx=0 tx=0).stop 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 -- 172.21.15.106:0/2713082068 shutdown_connections 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 --2- 172.21.15.106:0/2713082068 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c0cd160 0x7f6c0c0cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 --2- 172.21.15.106:0/2713082068 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c162970 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 -- 172.21.15.106:0/2713082068 >> 172.21.15.106:0/2713082068 conn(0x7f6c0c0cb830 msgr2=0x7f6c0c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 -- 172.21.15.106:0/2713082068 shutdown_connections 2023-11-24T08:10:37.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.914+0000 7f6c10940640 1 -- 172.21.15.106:0/2713082068 wait complete. 2023-11-24T08:10:37.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.916+0000 7f6c10940640 1 Processor -- start 2023-11-24T08:10:37.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.916+0000 7f6c10940640 1 -- start start 2023-11-24T08:10:37.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c10940640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c0b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c10940640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 0x7f6c0c0a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c10940640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c14b100 0x7f6c0c0af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c10940640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0b4ce0 con 0x7f6c0c0cd160 2023-11-24T08:10:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c10940640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0b4fc0 con 0x7f6c0c0703f0 2023-11-24T08:10:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c10940640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0ac390 con 0x7f6c0c14b100 2023-11-24T08:10:37.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c0a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 0x7f6c0c0a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c0a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 0x7f6c0c0a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35234/0 (socket says 172.21.15.106:35234) 2023-11-24T08:10:37.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c0a59c640 1 -- 172.21.15.106:0/1763188240 learned_addr learned my addr 172.21.15.106:0/1763188240 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:37.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c0ad9d640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c0b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.917+0000 7f6c0b59e640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c14b100 0x7f6c0c0af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0a59c640 1 -- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c14b100 msgr2=0x7f6c0c0af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0a59c640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c14b100 0x7f6c0c0af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0a59c640 1 -- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c0c0703f0 msgr2=0x7f6c0c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:37.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0a59c640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c0b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:37.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0a59c640 1 -- 172.21.15.106:0/1763188240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40094d0 con 0x7f6c0c0cd160 2023-11-24T08:10:37.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0b59e640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c14b100 0x7f6c0c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:37.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c0a59c640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 0x7f6c0c0a8430 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40090b0 tx=0x7f6bf40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4032860 con 0x7f6c0c0cd160 2023-11-24T08:10:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bf40329e0 con 0x7f6c0c0cd160 2023-11-24T08:10:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c0ac670 con 0x7f6c0c0cd160 2023-11-24T08:10:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf4032d90 con 0x7f6c0c0cd160 2023-11-24T08:10:37.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.918+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c0abff0 con 0x7f6c0c0cd160 2023-11-24T08:10:37.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.919+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c0ac880 con 0x7f6c0c0cd160 2023-11-24T08:10:37.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.920+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f6bf404d3f0 con 0x7f6c0c0cd160 2023-11-24T08:10:37.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.921+0000 7f6c037fe640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bec06aed0 0x7f6bec06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:37.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.921+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6bf40aaff0 con 0x7f6c0c0cd160 2023-11-24T08:10:37.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.922+0000 7f6c0ad9d640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bec06aed0 0x7f6bec06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:37.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.922+0000 7f6c0ad9d640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bec06aed0 0x7f6bec06d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc006fd0 tx=0x7f6bfc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:37.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:37.926+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6bf40dde90 con 0x7f6c0c0cd160 2023-11-24T08:10:38.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.106+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f6c0c162eb0 con 0x7f6c0c0cd160 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.107+0000 7f6c037fe640 1 -- 172.21.15.106:0/1763188240 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v44) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6c0c162eb0 con 0x7f6c0c0cd160 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2023-11-24T08:10:38.109 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:38.110 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bec06aed0 msgr2=0x7f6bec06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bec06aed0 0x7f6bec06d390 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc006fd0 tx=0x7f6bfc008040 comp rx=0 tx=0).stop 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 msgr2=0x7f6c0c0a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 0x7f6c0c0a8430 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6bf40090b0 tx=0x7f6bf40046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 shutdown_connections 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6bec06aed0 0x7f6bec06d390 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6c0c14b100 0x7f6c0c0af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6c0c0cd160 0x7f6c0c0a8430 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 --2- 172.21.15.106:0/1763188240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c0c0703f0 0x7f6c0c0b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 >> 172.21.15.106:0/1763188240 conn(0x7f6c0c0cb830 msgr2=0x7f6c0c163960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:38.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 shutdown_connections 2023-11-24T08:10:38.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.110+0000 7f6c10940640 1 -- 172.21.15.106:0/1763188240 wait complete. 2023-11-24T08:10:38.130 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:10:38.130 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:10:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.331+0000 7f5ddab3a640 1 Processor -- start 2023-11-24T08:10:38.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.331+0000 7f5ddab3a640 1 -- start start 2023-11-24T08:10:38.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5ddab3a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd40cf450 0x7f5dd40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:38.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5ddab3a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5dd40cdfb0 0x7f5dd40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:38.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5ddab3a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd40cece0 con 0x7f5dd415c7f0 2023-11-24T08:10:38.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5ddab3a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd4156010 con 0x7f5dd40cdfb0 2023-11-24T08:10:38.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5ddab3a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd406ce30 con 0x7f5dd40cf450 2023-11-24T08:10:38.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5dd90d7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5dd40cdfb0 0x7f5dd40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:38.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5dd90d7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5dd40cdfb0 0x7f5dd40ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35248/0 (socket says 172.21.15.106:35248) 2023-11-24T08:10:38.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.333+0000 7f5dd90d7640 1 -- 172.21.15.106:0/4178142487 learned_addr learned my addr 172.21.15.106:0/4178142487 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:38.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.334+0000 7f5dd90d7640 1 -- 172.21.15.106:0/4178142487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd40cf450 msgr2=0x7f5dd40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.334+0000 7f5dd90d7640 1 --2- 172.21.15.106:0/4178142487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd40cf450 0x7f5dd40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.334+0000 7f5dd90d7640 1 -- 172.21.15.106:0/4178142487 >> v1:172.21.15.106:6789/0 conn(0x7f5dd415c7f0 legacy=0x7f5dd406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:38.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.334+0000 7f5dd90d7640 1 -- 172.21.15.106:0/4178142487 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd406d040 con 0x7f5dd40cdfb0 2023-11-24T08:10:38.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.335+0000 7f5dd90d7640 1 --2- 172.21.15.106:0/4178142487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd40ce440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009e50 tx=0x7f5dcc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f399da81254cf84c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:38.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.337+0000 7f5dd37fe640 1 -- 172.21.15.106:0/4178142487 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc02cb00 con 0x7f5dd40cdfb0 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.337+0000 7f5dd37fe640 1 -- 172.21.15.106:0/4178142487 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dcc02cc60 con 0x7f5dd40cdfb0 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.337+0000 7f5dd37fe640 1 -- 172.21.15.106:0/4178142487 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc0343f0 con 0x7f5dd40cdfb0 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 -- 172.21.15.106:0/4178142487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cdfb0 msgr2=0x7f5dd40ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/4178142487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd40ce440 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc009e50 tx=0x7f5dcc02bf60 comp rx=0 tx=0).stop 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 -- 172.21.15.106:0/4178142487 shutdown_connections 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/4178142487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd40ce440 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/4178142487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd40cf450 0x7f5dd40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 -- 172.21.15.106:0/4178142487 >> 172.21.15.106:0/4178142487 conn(0x7f5dd406a920 msgr2=0x7f5dd406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 -- 172.21.15.106:0/4178142487 shutdown_connections 2023-11-24T08:10:38.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.338+0000 7f5ddab3a640 1 -- 172.21.15.106:0/4178142487 wait complete. 2023-11-24T08:10:38.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.339+0000 7f5ddab3a640 1 Processor -- start 2023-11-24T08:10:38.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.340+0000 7f5ddab3a640 1 -- start start 2023-11-24T08:10:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.340+0000 7f5ddab3a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.340+0000 7f5ddab3a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 0x7f5dd4158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5ddab3a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd415c7f0 0x7f5dd40adc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5ddab3a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40abfc0 con 0x7f5dd40cf450 2023-11-24T08:10:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5ddab3a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40ac790 con 0x7f5dd40cdfb0 2023-11-24T08:10:38.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5ddab3a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd40ac1d0 con 0x7f5dd415c7f0 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 0x7f5dd4158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 0x7f5dd4158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35264/0 (socket says 172.21.15.106:35264) 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 -- 172.21.15.106:0/3864561027 learned_addr learned my addr 172.21.15.106:0/3864561027 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd88d6640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 -- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd415c7f0 msgr2=0x7f5dd40adc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd415c7f0 0x7f5dd40adc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 -- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd40cdfb0 msgr2=0x7f5dd415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 -- 172.21.15.106:0/3864561027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dcc009b30 con 0x7f5dd40cf450 2023-11-24T08:10:38.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd88d6640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.341+0000 7f5dd3fff640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 0x7f5dd4158b40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000b4a0 tx=0x7f5dc000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.342+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0010610 con 0x7f5dd40cf450 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.342+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dc0010770 con 0x7f5dd40cf450 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.342+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dc0010b20 con 0x7f5dd40cf450 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.342+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd40ac4b0 con 0x7f5dd40cf450 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.342+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd40ae150 con 0x7f5dd40cf450 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.343+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd40b2af0 con 0x7f5dd40cf450 2023-11-24T08:10:38.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.344+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5dc0010c80 con 0x7f5dd40cf450 2023-11-24T08:10:38.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.345+0000 7f5dd17fa640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5db806aed0 0x7f5db806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:38.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.345+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5dc0089060 con 0x7f5dd40cf450 2023-11-24T08:10:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.345+0000 7f5dd88d6640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5db806aed0 0x7f5db806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:38.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.346+0000 7f5dd88d6640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5db806aed0 0x7f5db806d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8000f60 tx=0x7f5dc8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:38.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.349+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5dc00bbe90 con 0x7f5dd40cf450 2023-11-24T08:10:38.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:38.524+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f5dd40b2ed0 con 0x7f5dd40cf450 2023-11-24T08:10:39.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.350+0000 7f5dd17fa640 1 -- 172.21.15.106:0/3864561027 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v45) v1 ==== 138+0+0 (secure 0 0 0) 0x7f5dc00140b0 con 0x7f5dd40cf450 2023-11-24T08:10:39.352 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2023-11-24T08:10:39.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5db806aed0 msgr2=0x7f5db806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:39.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5db806aed0 0x7f5db806d390 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8000f60 tx=0x7f5dc8009040 comp rx=0 tx=0).stop 2023-11-24T08:10:39.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 msgr2=0x7f5dd4158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:39.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 0x7f5dd4158b40 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f5dc000b4a0 tx=0x7f5dc000b960 comp rx=0 tx=0).stop 2023-11-24T08:10:39.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 shutdown_connections 2023-11-24T08:10:39.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5db806aed0 0x7f5db806d390 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5dd415c7f0 0x7f5dd40adc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5dd40cf450 0x7f5dd4158b40 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 --2- 172.21.15.106:0/3864561027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5dd40cdfb0 0x7f5dd415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 >> 172.21.15.106:0/3864561027 conn(0x7f5dd406a920 msgr2=0x7f5dd40722a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 shutdown_connections 2023-11-24T08:10:39.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.354+0000 7f5ddab3a640 1 -- 172.21.15.106:0/3864561027 wait complete. 2023-11-24T08:10:39.383 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:10:39.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.590+0000 7febe44e7640 1 Processor -- start 2023-11-24T08:10:39.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.591+0000 7febe44e7640 1 -- start start 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.592+0000 7febe44e7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7febdc072160 0x7febdc15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.592+0000 7febe44e7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.592+0000 7febe44e7640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febdc15ec90 con 0x7febdc06dc40 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.592+0000 7febe44e7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febdc06e0f0 con 0x7febdc072160 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe44e7640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc0cd530 con 0x7febdc0c49f0 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7febdc072160 0x7febdc15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7febdc072160 0x7febdc15e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35276/0 (socket says 172.21.15.106:35276) 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 -- 172.21.15.106:0/3976202174 learned_addr learned my addr 172.21.15.106:0/3976202174 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2a84640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7febdc06dc40 0x7febdc0cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55252/0 (socket says 172.21.15.106:55252) 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe1a82640 1 --2- 172.21.15.106:0/3976202174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:39.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 -- 172.21.15.106:0/3976202174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 msgr2=0x7febdc06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 --2- 172.21.15.106:0/3976202174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 -- 172.21.15.106:0/3976202174 >> v1:172.21.15.106:6789/0 conn(0x7febdc06dc40 legacy=0x7febdc0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe2283640 1 -- 172.21.15.106:0/3976202174 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febdc0c4ed0 con 0x7febdc072160 2023-11-24T08:10:39.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.593+0000 7febe1281640 1 -- 172.21.15.106:0/3976202174 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4128270817 0 0) 0x7febd4001110 con 0x7febdc06dc40 2023-11-24T08:10:39.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.594+0000 7febe2283640 1 --2- 172.21.15.106:0/3976202174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc15e750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7febcc00d150 tx=0x7febcc02e160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39e4ed7ce4750e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:39.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe1281640 1 -- 172.21.15.106:0/3976202174 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc00b0d0 con 0x7febdc072160 2023-11-24T08:10:39.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe1281640 1 -- 172.21.15.106:0/3976202174 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febcc00b230 con 0x7febdc072160 2023-11-24T08:10:39.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe1281640 1 -- 172.21.15.106:0/3976202174 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febcc009600 con 0x7febdc072160 2023-11-24T08:10:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe44e7640 1 -- 172.21.15.106:0/3976202174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 msgr2=0x7febdc15e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe44e7640 1 --2- 172.21.15.106:0/3976202174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc15e750 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7febcc00d150 tx=0x7febcc02e160 comp rx=0 tx=0).stop 2023-11-24T08:10:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe44e7640 1 -- 172.21.15.106:0/3976202174 shutdown_connections 2023-11-24T08:10:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe44e7640 1 --2- 172.21.15.106:0/3976202174 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe44e7640 1 --2- 172.21.15.106:0/3976202174 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc15e750 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.595+0000 7febe44e7640 1 -- 172.21.15.106:0/3976202174 >> 172.21.15.106:0/3976202174 conn(0x7febdc0b0c50 msgr2=0x7febdc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:39.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.596+0000 7febe44e7640 1 -- 172.21.15.106:0/3976202174 shutdown_connections 2023-11-24T08:10:39.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.596+0000 7febe44e7640 1 -- 172.21.15.106:0/3976202174 wait complete. 2023-11-24T08:10:39.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.597+0000 7febe44e7640 1 Processor -- start 2023-11-24T08:10:39.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.598+0000 7febe44e7640 1 -- start start 2023-11-24T08:10:39.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.598+0000 7febe44e7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febdc06dc40 0x7febdc159080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:39.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe44e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc09af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:39.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe44e7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:39.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe44e7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc1595c0 con 0x7febdc072160 2023-11-24T08:10:39.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe44e7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc09c5e0 con 0x7febdc06dc40 2023-11-24T08:10:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe44e7640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febdc15b220 con 0x7febdc0c49f0 2023-11-24T08:10:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc09af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc09af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35292/0 (socket says 172.21.15.106:35292) 2023-11-24T08:10:39.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 -- 172.21.15.106:0/1568946631 learned_addr learned my addr 172.21.15.106:0/1568946631 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe2a84640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 -- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 msgr2=0x7febdc06cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe2283640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febdc06dc40 0x7febdc159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:39.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 -- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febdc06dc40 msgr2=0x7febdc159080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febdc06dc40 0x7febdc159080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe1a82640 1 -- 172.21.15.106:0/1568946631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febcc00bd40 con 0x7febdc072160 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.599+0000 7febe2a84640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.600+0000 7febe1a82640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc09af10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7febd000d620 tx=0x7febd000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.600+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000b620 con 0x7febdc072160 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.600+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febd000b780 con 0x7febdc072160 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.600+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd000bb30 con 0x7febdc072160 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.600+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc09b9d0 con 0x7febdc072160 2023-11-24T08:10:39.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.600+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc09bf90 con 0x7febdc072160 2023-11-24T08:10:39.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.601+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc04e8b0 con 0x7febdc072160 2023-11-24T08:10:39.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.602+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7febd000bc90 con 0x7febdc072160 2023-11-24T08:10:39.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.603+0000 7febdaffd640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7febc806aed0 0x7febc806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:39.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.603+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5227+0+0 (secure 0 0 0) 0x7febd0089a30 con 0x7febdc072160 2023-11-24T08:10:39.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.604+0000 7febe2283640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7febc806aed0 0x7febc806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:39.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.604+0000 7febe2283640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7febc806aed0 0x7febc806d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7febcc037000 tx=0x7febcc032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:39.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.609+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7febd0054bd0 con 0x7febdc072160 2023-11-24T08:10:39.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:39.799+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7febdc15dab0 con 0x7febdc072160 2023-11-24T08:10:40.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.356+0000 7febdaffd640 1 -- 172.21.15.106:0/1568946631 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v46) v1 ==== 170+0+0 (secure 0 0 0) 0x7febd00581f0 con 0x7febdc072160 2023-11-24T08:10:40.359 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2023-11-24T08:10:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7febc806aed0 msgr2=0x7febc806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:40.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7febc806aed0 0x7febc806d390 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7febcc037000 tx=0x7febcc032000 comp rx=0 tx=0).stop 2023-11-24T08:10:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 msgr2=0x7febdc09af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:40.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc09af10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7febd000d620 tx=0x7febd000dae0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 shutdown_connections 2023-11-24T08:10:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7febc806aed0 0x7febc806d390 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7febdc0c49f0 0x7febdc06cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7febdc072160 0x7febdc09af10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 --2- 172.21.15.106:0/1568946631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febdc06dc40 0x7febdc159080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 >> 172.21.15.106:0/1568946631 conn(0x7febdc0b0c50 msgr2=0x7febdc0ad750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 shutdown_connections 2023-11-24T08:10:40.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.360+0000 7febe44e7640 1 -- 172.21.15.106:0/1568946631 wait complete. 2023-11-24T08:10:40.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:10:40.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.581+0000 7fbbaf59e640 1 Processor -- start 2023-11-24T08:10:40.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.582+0000 7fbbaf59e640 1 -- start start 2023-11-24T08:10:40.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbbaf59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbbb00ca940 0x7fbbb00b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:40.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbbaf59e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00cb960 con 0x7fbbb00c27a0 2023-11-24T08:10:40.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbbaf59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb0161e40 con 0x7fbbb00cb580 2023-11-24T08:10:40.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbbaf59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb0162120 con 0x7fbbb00ca940 2023-11-24T08:10:40.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbbaed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbbb00ca940 0x7fbbb00b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbba7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbbb00c27a0 0x7fbbb00b0e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55268/0 (socket says 172.21.15.106:55268) 2023-11-24T08:10:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbba7fff640 1 -- 172.21.15.106:0/155976656 learned_addr learned my addr 172.21.15.106:0/155976656 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.583+0000 7fbbaed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbbb00ca940 0x7fbbb00b15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35308/0 (socket says 172.21.15.106:35308) 2023-11-24T08:10:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.584+0000 7fbbaed9d640 1 -- 172.21.15.106:0/155976656 >> v1:172.21.15.156:6789/0 conn(0x7fbbb00cb580 legacy=0x7fbbb00c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.584+0000 7fbbaed9d640 1 -- 172.21.15.106:0/155976656 >> v1:172.21.15.106:6789/0 conn(0x7fbbb00c27a0 legacy=0x7fbbb00b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:40.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.584+0000 7fbbaed9d640 1 -- 172.21.15.106:0/155976656 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb0162400 con 0x7fbbb00ca940 2023-11-24T08:10:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.585+0000 7fbbaed9d640 1 --2- 172.21.15.106:0/155976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb00b15a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009d90 tx=0x7fbba802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28b024f7e60c1ef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.585+0000 7fbbadd9b640 1 -- 172.21.15.106:0/155976656 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba802cc60 con 0x7fbbb00ca940 2023-11-24T08:10:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.585+0000 7fbbadd9b640 1 -- 172.21.15.106:0/155976656 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba802cdc0 con 0x7fbbb00ca940 2023-11-24T08:10:40.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.585+0000 7fbbadd9b640 1 -- 172.21.15.106:0/155976656 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba8034660 con 0x7fbbb00ca940 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 -- 172.21.15.106:0/155976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 msgr2=0x7fbbb00b15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/155976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb00b15a0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fbba8009d90 tx=0x7fbba802beb0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 -- 172.21.15.106:0/155976656 shutdown_connections 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/155976656 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb00b15a0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 -- 172.21.15.106:0/155976656 >> 172.21.15.106:0/155976656 conn(0x7fbbb00cbb40 msgr2=0x7fbbb00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 -- 172.21.15.106:0/155976656 shutdown_connections 2023-11-24T08:10:40.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.586+0000 7fbbaf59e640 1 -- 172.21.15.106:0/155976656 wait complete. 2023-11-24T08:10:40.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.588+0000 7fbbaf59e640 1 Processor -- start 2023-11-24T08:10:40.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.588+0000 7fbbaf59e640 1 -- start start 2023-11-24T08:10:40.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbaf59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbb00c27a0 0x7fbbb009a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbaf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb01f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbaf59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb00cb580 0x7fbbb01fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbaf59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00c2ba0 con 0x7fbbb00ca940 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbaf59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00999d0 con 0x7fbbb00cb580 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbaf59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01fb170 con 0x7fbbb00c27a0 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb01f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbba7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb01f6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35320/0 (socket says 172.21.15.106:35320) 2023-11-24T08:10:40.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbba7fff640 1 -- 172.21.15.106:0/3937001857 learned_addr learned my addr 172.21.15.106:0/3937001857 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:40.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.589+0000 7fbbae59c640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbb00c27a0 0x7fbbb009a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:40.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba7fff640 1 -- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbb00c27a0 msgr2=0x7fbbb009a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:40.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba7fff640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbb00c27a0 0x7fbbb009a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba7fff640 1 -- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb00cb580 msgr2=0x7fbbb01fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba7fff640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb00cb580 0x7fbbb01fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba7fff640 1 -- 172.21.15.106:0/3937001857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8009a70 con 0x7fbbb00ca940 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba7fff640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb01f6920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c4a0 tx=0x7fbb9800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98005ce0 con 0x7fbbb00ca940 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98005e40 con 0x7fbbb00ca940 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98007270 con 0x7fbbb00ca940 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb01fb450 con 0x7fbbb00ca940 2023-11-24T08:10:40.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.590+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb0099f50 con 0x7fbbb00ca940 2023-11-24T08:10:40.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.592+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb009a140 con 0x7fbbb00ca940 2023-11-24T08:10:40.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.593+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fbb98007490 con 0x7fbbb00ca940 2023-11-24T08:10:40.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.593+0000 7fbba6ffd640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbb9006aed0 0x7fbb9006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:40.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.594+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fbb98088b30 con 0x7fbbb00ca940 2023-11-24T08:10:40.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.594+0000 7fbbae59c640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbb9006aed0 0x7fbb9006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:40.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.595+0000 7fbbae59c640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbb9006aed0 0x7fbb9006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a750 tx=0x7fbba0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:40.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.599+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbb98053c70 con 0x7fbbb00ca940 2023-11-24T08:10:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:40.828+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fbbb009aa10 con 0x7fbbb00ca940 2023-11-24T08:10:41.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.381+0000 7fbba6ffd640 1 -- 172.21.15.106:0/3937001857 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fbb98017070 con 0x7fbbb00ca940 2023-11-24T08:10:41.383 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 6 and data pool 7 2023-11-24T08:10:41.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbb9006aed0 msgr2=0x7fbb9006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbb9006aed0 0x7fbb9006d390 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbba000a750 tx=0x7fbba0008040 comp rx=0 tx=0).stop 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 msgr2=0x7fbbb01f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb01f6920 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c4a0 tx=0x7fbb9800c960 comp rx=0 tx=0).stop 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 shutdown_connections 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbb00cb580 0x7fbbb01fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbbb00ca940 0x7fbbb01f6920 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbb9006aed0 0x7fbb9006d390 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 --2- 172.21.15.106:0/3937001857 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbbb00c27a0 0x7fbbb009a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.385+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 >> 172.21.15.106:0/3937001857 conn(0x7fbbb00cbb40 msgr2=0x7fbbb0163c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:41.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.386+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 shutdown_connections 2023-11-24T08:10:41.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.386+0000 7fbbaf59e640 1 -- 172.21.15.106:0/3937001857 wait complete. 2023-11-24T08:10:41.423 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:41.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.608+0000 7f1284999640 1 Processor -- start 2023-11-24T08:10:41.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.608+0000 7f1284999640 1 -- start start 2023-11-24T08:10:41.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.609+0000 7f1284999640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 0x7f128015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:41.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.609+0000 7f1284999640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12800b1de0 0x7f12800b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:41.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.609+0000 7f1284999640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128015bf80 con 0x7f1280144b30 2023-11-24T08:10:41.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.609+0000 7f1284999640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12800b27b0 con 0x7f12800b1de0 2023-11-24T08:10:41.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.609+0000 7f1284999640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12800b1560 con 0x7f128015c410 2023-11-24T08:10:41.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.610+0000 7f127effd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 0x7f128015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.610+0000 7f127ffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f12800b1de0 0x7f12800b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.610+0000 7f127effd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 0x7f128015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:59038/0 (socket says 172.21.15.106:59038) 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.610+0000 7f127effd640 1 -- 172.21.15.106:0/1873952478 learned_addr learned my addr 172.21.15.106:0/1873952478 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.610+0000 7f127f7fe640 1 --1- 172.21.15.106:0/1873952478 >> v1:172.21.15.156:6789/0 conn(0x7f1280144b30 0x7f12800b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:50684/0 (socket says 172.21.15.106:50684) 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.611+0000 7f127ffff640 1 -- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 msgr2=0x7f128015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.611+0000 7f127ffff640 1 --2- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 0x7f128015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.611+0000 7f127ffff640 1 -- 172.21.15.106:0/1873952478 >> v1:172.21.15.156:6789/0 conn(0x7f1280144b30 legacy=0x7f12800b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.611+0000 7f127ffff640 1 -- 172.21.15.106:0/1873952478 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12800b1280 con 0x7f12800b1de0 2023-11-24T08:10:41.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.611+0000 7f127effd640 1 --2- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 0x7f128015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:10:41.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.612+0000 7f127ffff640 1 --2- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f12800b2270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f12780097a0 tx=0x7f127802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd9206c505af9396 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:41.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.613+0000 7f127e7fc640 1 -- 172.21.15.106:0/1873952478 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127802d040 con 0x7f12800b1de0 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.613+0000 7f127e7fc640 1 -- 172.21.15.106:0/1873952478 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1278031810 con 0x7f12800b1de0 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.613+0000 7f127e7fc640 1 -- 172.21.15.106:0/1873952478 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278031ba0 con 0x7f12800b1de0 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.613+0000 7f1284999640 1 -- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 msgr2=0x7f12800b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.613+0000 7f1284999640 1 --2- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f12800b2270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f12780097a0 tx=0x7f127802b8b0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.614+0000 7f1284999640 1 -- 172.21.15.106:0/1873952478 shutdown_connections 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.614+0000 7f1284999640 1 --2- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f12800b2270 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.614+0000 7f1284999640 1 --2- 172.21.15.106:0/1873952478 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f128015c410 0x7f128015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.614+0000 7f1284999640 1 -- 172.21.15.106:0/1873952478 >> 172.21.15.106:0/1873952478 conn(0x7f12800cb830 msgr2=0x7f12800cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:41.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.614+0000 7f1284999640 1 -- 172.21.15.106:0/1873952478 shutdown_connections 2023-11-24T08:10:41.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.614+0000 7f1284999640 1 -- 172.21.15.106:0/1873952478 wait complete. 2023-11-24T08:10:41.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.615+0000 7f1284999640 1 Processor -- start 2023-11-24T08:10:41.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.616+0000 7f1284999640 1 -- start start 2023-11-24T08:10:41.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.616+0000 7f1284999640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f128015a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:41.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f1284999640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1280144b30 0x7f12801e4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:41.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f1284999640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 0x7f12800ca900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:41.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f1284999640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12800ca470 con 0x7f128015c410 2023-11-24T08:10:41.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f1284999640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128015a980 con 0x7f12800b1de0 2023-11-24T08:10:41.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f1284999640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12800c9860 con 0x7f1280144b30 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 0x7f12800ca900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 0x7f12800ca900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35336/0 (socket says 172.21.15.106:35336) 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 -- 172.21.15.106:0/2182655476 learned_addr learned my addr 172.21.15.106:0/2182655476 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127f7fe640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f128015a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127effd640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1280144b30 0x7f12801e4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 -- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1280144b30 msgr2=0x7f12801e4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1280144b30 0x7f12801e4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 -- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 msgr2=0x7f128015a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f128015a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127ffff640 1 -- 172.21.15.106:0/2182655476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680094d0 con 0x7f128015c410 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.617+0000 7f127f7fe640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f128015a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.618+0000 7f127ffff640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 0x7f12800ca900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f127802b8f0 tx=0x7f12780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.618+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127803ac60 con 0x7f128015c410 2023-11-24T08:10:41.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.618+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127803ade0 con 0x7f128015c410 2023-11-24T08:10:41.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.618+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1278009400 con 0x7f128015c410 2023-11-24T08:10:41.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.618+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1278044630 con 0x7f128015c410 2023-11-24T08:10:41.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.618+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12800c9e20 con 0x7f128015c410 2023-11-24T08:10:41.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.619+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12800ca620 con 0x7f128015c410 2023-11-24T08:10:41.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.620+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1278044790 con 0x7f128015c410 2023-11-24T08:10:41.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.622+0000 7f1277fff640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f126006aed0 0x7f126006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:41.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.622+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12780aba60 con 0x7f128015c410 2023-11-24T08:10:41.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.622+0000 7f127f7fe640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f126006aed0 0x7f126006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:41.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.623+0000 7f127f7fe640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f126006aed0 0x7f126006d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1270007fd0 tx=0x7f1270009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:41.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.627+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f12800ca620 con 0x7f128015c410 2023-11-24T08:10:41.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.805+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12800b3610 con 0x7f128015c410 2023-11-24T08:10:41.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.806+0000 7f1277fff640 1 -- 172.21.15.106:0/2182655476 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16574 (secure 0 0 0) 0x7f12800b3610 con 0x7f128015c410 2023-11-24T08:10:41.808 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:41.809 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":47,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:10:40.829452+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f126006aed0 msgr2=0x7f126006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f126006aed0 0x7f126006d390 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f1270007fd0 tx=0x7f1270009040 comp rx=0 tx=0).stop 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 msgr2=0x7f12800ca900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 0x7f12800ca900 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f127802b8f0 tx=0x7f12780046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 shutdown_connections 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f128015c410 0x7f12800ca900 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f126006aed0 0x7f126006d390 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1280144b30 0x7f12801e4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 --2- 172.21.15.106:0/2182655476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12800b1de0 0x7f128015a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:41.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 >> 172.21.15.106:0/2182655476 conn(0x7f12800cb830 msgr2=0x7f12800b29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:41.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.809+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 shutdown_connections 2023-11-24T08:10:41.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:41.810+0000 7f1284999640 1 -- 172.21.15.106:0/2182655476 wait complete. 2023-11-24T08:10:41.828 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:42.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.027+0000 7f1c9f761640 1 Processor -- start 2023-11-24T08:10:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.027+0000 7f1c9f761640 1 -- start start 2023-11-24T08:10:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.028+0000 7f1c9f761640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c980b2250 0x7f1c9815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.028+0000 7f1c9f761640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9815ab80 con 0x7f1c980b2630 2023-11-24T08:10:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.028+0000 7f1c9f761640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9815e680 con 0x7f1c980b2aa0 2023-11-24T08:10:42.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.028+0000 7f1c9f761640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9815a990 con 0x7f1c980b2250 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9d4fd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c980b2250 0x7f1c9815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9d4fd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1c980b2250 0x7f1c9815df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35338/0 (socket says 172.21.15.106:35338) 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9d4fd640 1 -- 172.21.15.106:0/3575645862 learned_addr learned my addr 172.21.15.106:0/3575645862 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9ccfc640 1 --1- 172.21.15.106:0/3575645862 >> v1:172.21.15.106:6789/0 conn(0x7f1c980b2630 0x7f1c980ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55282/0 (socket says 172.21.15.106:55282) 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9d4fd640 1 -- 172.21.15.106:0/3575645862 >> v1:172.21.15.156:6789/0 conn(0x7f1c980b2aa0 legacy=0x7f1c9815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9d4fd640 1 -- 172.21.15.106:0/3575645862 >> v1:172.21.15.106:6789/0 conn(0x7f1c980b2630 legacy=0x7f1c980ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c8ffff640 1 -- 172.21.15.106:0/3575645862 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243363184 0 0) 0x7f1c9815ab80 con 0x7f1c980b2630 2023-11-24T08:10:42.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.029+0000 7f1c9d4fd640 1 -- 172.21.15.106:0/3575645862 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9815e960 con 0x7f1c980b2250 2023-11-24T08:10:42.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.030+0000 7f1c9d4fd640 1 --2- 172.21.15.106:0/3575645862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2250 0x7f1c9815df80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1c9815ad70 tx=0x7f1c9402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73d0ead362cd665 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.030+0000 7f1c8ffff640 1 -- 172.21.15.106:0/3575645862 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94030840 con 0x7f1c980b2250 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c8ffff640 1 -- 172.21.15.106:0/3575645862 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c940309a0 con 0x7f1c980b2250 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c8ffff640 1 -- 172.21.15.106:0/3575645862 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94030d30 con 0x7f1c980b2250 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 -- 172.21.15.106:0/3575645862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2250 msgr2=0x7f1c9815df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3575645862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2250 0x7f1c9815df80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f1c9815ad70 tx=0x7f1c9402b980 comp rx=0 tx=0).stop 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 -- 172.21.15.106:0/3575645862 shutdown_connections 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3575645862 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2250 0x7f1c9815df80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 -- 172.21.15.106:0/3575645862 >> 172.21.15.106:0/3575645862 conn(0x7f1c9806a920 msgr2=0x7f1c9806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 -- 172.21.15.106:0/3575645862 shutdown_connections 2023-11-24T08:10:42.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.031+0000 7f1c9f761640 1 -- 172.21.15.106:0/3575645862 wait complete. 2023-11-24T08:10:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.033+0000 7f1c9f761640 1 Processor -- start 2023-11-24T08:10:42.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.033+0000 7f1c9f761640 1 -- start start 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9f761640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c980b2250 0x7f1c98097100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9f761640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c980b2630 0x7f1c980ad790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9f761640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 0x7f1c980aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9f761640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98093480 con 0x7f1c980b2aa0 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9f761640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98093760 con 0x7f1c980b2630 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9f761640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c98093a40 con 0x7f1c980b2250 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9dcfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 0x7f1c980aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9dcfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 0x7f1c980aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35354/0 (socket says 172.21.15.106:35354) 2023-11-24T08:10:42.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.034+0000 7f1c9dcfe640 1 -- 172.21.15.106:0/3717134240 learned_addr learned my addr 172.21.15.106:0/3717134240 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:42.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9dcfe640 1 -- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c980b2250 msgr2=0x7f1c98097100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9d4fd640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c980b2250 0x7f1c98097100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9ccfc640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c980b2630 0x7f1c980ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9dcfe640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c980b2250 0x7f1c98097100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9dcfe640 1 -- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c980b2630 msgr2=0x7f1c980ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9dcfe640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c980b2630 0x7f1c980ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9dcfe640 1 -- 172.21.15.106:0/3717134240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c940094d0 con 0x7f1c980b2aa0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9dcfe640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 0x7f1c980aeb50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000c780 tx=0x7f1c9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c90004b10 con 0x7f1c980b2aa0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c90004c70 con 0x7f1c980b2aa0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000aa90 con 0x7f1c980b2aa0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.035+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c980cd7c0 con 0x7f1c980b2aa0 2023-11-24T08:10:42.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.036+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c980944b0 con 0x7f1c980b2aa0 2023-11-24T08:10:42.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.037+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c98067470 con 0x7f1c980b2aa0 2023-11-24T08:10:42.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.037+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f1c9000abf0 con 0x7f1c980b2aa0 2023-11-24T08:10:42.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.038+0000 7f1c8dffb640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1c7c06aed0 0x7f1c7c06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.038+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c90089690 con 0x7f1c980b2aa0 2023-11-24T08:10:42.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.039+0000 7f1c9d4fd640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1c7c06aed0 0x7f1c7c06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.039+0000 7f1c9d4fd640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1c7c06aed0 0x7f1c7c06d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c94036000 tx=0x7f1c9403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:42.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.043+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1c900547e0 con 0x7f1c980b2aa0 2023-11-24T08:10:42.062 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-24T08:10:42.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.222+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c98167110 con 0x7f1c980b2aa0 2023-11-24T08:10:42.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.224+0000 7f1c8dffb640 1 -- 172.21.15.106:0/3717134240 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+16574 (secure 0 0 0) 0x7f1c90057e00 con 0x7f1c980b2aa0 2023-11-24T08:10:42.226 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:42.227 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":47,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:10:40.829452+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1c7c06aed0 msgr2=0x7f1c7c06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1c7c06aed0 0x7f1c7c06d390 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1c94036000 tx=0x7f1c9403b040 comp rx=0 tx=0).stop 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 msgr2=0x7f1c980aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 0x7f1c980aeb50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f1c9000c780 tx=0x7f1c9000cc40 comp rx=0 tx=0).stop 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 shutdown_connections 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1c980b2aa0 0x7f1c980aeb50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c980b2630 0x7f1c980ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1c7c06aed0 0x7f1c7c06d390 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 --2- 172.21.15.106:0/3717134240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1c980b2250 0x7f1c98097100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 >> 172.21.15.106:0/3717134240 conn(0x7f1c9806a920 msgr2=0x7f1c980ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:42.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.227+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 shutdown_connections 2023-11-24T08:10:42.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.228+0000 7f1c9f761640 1 -- 172.21.15.106:0/3717134240 wait complete. 2023-11-24T08:10:42.247 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:10:42.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.455+0000 7f405d085640 1 Processor -- start 2023-11-24T08:10:42.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.456+0000 7f405d085640 1 -- start start 2023-11-24T08:10:42.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f405d085640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f405815a1c0 0x7f405806d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f405d085640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40580b0b40 0x7f40580b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f405d085640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f405806d430 con 0x7f405815ded0 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f405d085640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f405815ea70 con 0x7f405815a1c0 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f405d085640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f405815ec50 con 0x7f40580b0b40 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f4057fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f405815a1c0 0x7f405806d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f4057fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f405815a1c0 0x7f405806d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35366/0 (socket says 172.21.15.106:35366) 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.457+0000 7f4057fff640 1 -- 172.21.15.106:0/3333756715 learned_addr learned my addr 172.21.15.106:0/3333756715 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:42.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.458+0000 7f4057fff640 1 -- 172.21.15.106:0/3333756715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40580b0b40 msgr2=0x7f40580b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:42.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.458+0000 7f4057fff640 1 --2- 172.21.15.106:0/3333756715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40580b0b40 0x7f40580b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.458+0000 7f4057fff640 1 -- 172.21.15.106:0/3333756715 >> v1:172.21.15.156:6789/0 conn(0x7f405815ded0 legacy=0x7f405815e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:42.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.458+0000 7f4057fff640 1 -- 172.21.15.106:0/3333756715 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40580b14e0 con 0x7f405815a1c0 2023-11-24T08:10:42.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.459+0000 7f4057fff640 1 --2- 172.21.15.106:0/3333756715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f405815a1c0 0x7f405806d8c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4048009870 tx=0x7f404802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95cb44de51618f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:42.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.459+0000 7f40577fe640 1 -- 172.21.15.106:0/3333756715 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048031640 con 0x7f405815a1c0 2023-11-24T08:10:42.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.459+0000 7f40577fe640 1 -- 172.21.15.106:0/3333756715 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40480317a0 con 0x7f405815a1c0 2023-11-24T08:10:42.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.459+0000 7f40577fe640 1 -- 172.21.15.106:0/3333756715 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048031b30 con 0x7f405815a1c0 2023-11-24T08:10:42.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 -- 172.21.15.106:0/3333756715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f405815a1c0 msgr2=0x7f405806d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 --2- 172.21.15.106:0/3333756715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f405815a1c0 0x7f405806d8c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4048009870 tx=0x7f404802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 -- 172.21.15.106:0/3333756715 shutdown_connections 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 --2- 172.21.15.106:0/3333756715 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f40580b0b40 0x7f40580b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 --2- 172.21.15.106:0/3333756715 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f405815a1c0 0x7f405806d8c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 -- 172.21.15.106:0/3333756715 >> 172.21.15.106:0/3333756715 conn(0x7f40580cbb40 msgr2=0x7f40580cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 -- 172.21.15.106:0/3333756715 shutdown_connections 2023-11-24T08:10:42.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.460+0000 7f405d085640 1 -- 172.21.15.106:0/3333756715 wait complete. 2023-11-24T08:10:42.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.462+0000 7f405d085640 1 Processor -- start 2023-11-24T08:10:42.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.462+0000 7f405d085640 1 -- start start 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f405d085640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 0x7f4058159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f405d085640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f405815a1c0 0x7f40581f76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f405d085640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405815ded0 0x7f40580b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f405d085640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40580c9060 con 0x7f40580b0b40 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f405d085640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40580c84c0 con 0x7f405815a1c0 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f405d085640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4058159ad0 con 0x7f405815ded0 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f4057fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 0x7f4058159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f4057fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 0x7f4058159590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35374/0 (socket says 172.21.15.106:35374) 2023-11-24T08:10:42.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.463+0000 7f4057fff640 1 -- 172.21.15.106:0/384305201 learned_addr learned my addr 172.21.15.106:0/384305201 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f405c884640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405815ded0 0x7f40580b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f404ffff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f405815a1c0 0x7f40581f76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f4057fff640 1 -- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405815ded0 msgr2=0x7f40580b3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f4057fff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405815ded0 0x7f40580b3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f4057fff640 1 -- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f405815a1c0 msgr2=0x7f40581f76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f4057fff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f405815a1c0 0x7f40581f76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f4057fff640 1 -- 172.21.15.106:0/384305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480094d0 con 0x7f40580b0b40 2023-11-24T08:10:42.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f405c884640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405815ded0 0x7f40580b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f404ffff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f405815a1c0 0x7f40581f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f4057fff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 0x7f4058159590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c00 tx=0x7f40480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f404803ac60 con 0x7f40580b0b40 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f404803adc0 con 0x7f40580b0b40 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.464+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4048044630 con 0x7f40580b0b40 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.465+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40580c8730 con 0x7f40580b0b40 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.465+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40580c8cf0 con 0x7f40580b0b40 2023-11-24T08:10:42.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.466+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40580cb730 con 0x7f40580b0b40 2023-11-24T08:10:42.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.467+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f404802f050 con 0x7f40580b0b40 2023-11-24T08:10:42.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.467+0000 7f40557fa640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f403806aed0 0x7f403806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:42.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.468+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40480ab830 con 0x7f40580b0b40 2023-11-24T08:10:42.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.468+0000 7f404ffff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f403806aed0 0x7f403806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:42.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.468+0000 7f404ffff640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f403806aed0 0x7f403806d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:42.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.472+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f40480766e0 con 0x7f40580b0b40 2023-11-24T08:10:42.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:42.680+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f405804e840 con 0x7f40580b0b40 2023-11-24T08:10:43.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.425+0000 7f40557fa640 1 -- 172.21.15.106:0/384305201 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4048031e60 con 0x7f40580b0b40 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f403806aed0 msgr2=0x7f403806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f403806aed0 0x7f403806d390 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f4040006fd0 tx=0x7f4040008040 comp rx=0 tx=0).stop 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 msgr2=0x7f4058159590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 0x7f4058159590 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4048000c00 tx=0x7f40480046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 shutdown_connections 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f403806aed0 0x7f403806d390 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f405815ded0 0x7f40580b3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f405815a1c0 0x7f40581f76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 --2- 172.21.15.106:0/384305201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f40580b0b40 0x7f4058159590 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 >> 172.21.15.106:0/384305201 conn(0x7f40580cbb40 msgr2=0x7f4058070f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:43.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 shutdown_connections 2023-11-24T08:10:43.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.429+0000 7f405d085640 1 -- 172.21.15.106:0/384305201 wait complete. 2023-11-24T08:10:43.451 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:10:43.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:10:43.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.659+0000 7f9018ae5640 1 Processor -- start 2023-11-24T08:10:43.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.660+0000 7f9018ae5640 1 -- start start 2023-11-24T08:10:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.661+0000 7f9018ae5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f90140c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.661+0000 7f9018ae5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f901415eea0 con 0x7f90140c57e0 2023-11-24T08:10:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.661+0000 7f9018ae5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90140abf30 con 0x7f901415b440 2023-11-24T08:10:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.661+0000 7f9018ae5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f901415b820 con 0x7f90140cc0f0 2023-11-24T08:10:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f9012d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f90140c57e0 0x7f901406c5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:55288/0 (socket says 172.21.15.106:55288) 2023-11-24T08:10:43.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f9012d9d640 1 -- 172.21.15.106:0/3118247213 learned_addr learned my addr 172.21.15.106:0/3118247213 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f90140c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321966034 0 0) 0x7f901415eea0 con 0x7f90140c57e0 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ffc003520 con 0x7f90140c57e0 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1461192567 0 0) 0x7f8ffc003520 con 0x7f90140c57e0 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 msgr2=0x7f90140c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f901159a640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f90140c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.662+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 >> v1:172.21.15.156:6789/0 conn(0x7f901415b440 legacy=0x7f90140cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f901415f0f0 con 0x7f90140c57e0 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f900c0019f0 con 0x7f90140c57e0 2023-11-24T08:10:43.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 >> v1:172.21.15.106:6789/0 conn(0x7f90140c57e0 legacy=0x7f901406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f8ffc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ffc003f20 0x7f8ffc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc004d00 0x7f8ffc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f901415b820 con 0x7f8ffc003f20 2023-11-24T08:10:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc00bb10 con 0x7f8ffc004d00 2023-11-24T08:10:43.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc00bcd0 con 0x7f90140cc0f0 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ffc003f20 0x7f8ffc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901259c640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f8ffc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9011d9b640 1 -- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 msgr2=0x7f8ffc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f8ffc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9011d9b640 1 -- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc004d00 msgr2=0x7f8ffc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9012d9d640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc004d00 0x7f8ffc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ffc004d00 0x7f8ffc005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f9011d9b640 1 -- 172.21.15.106:0/3118247213 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc003520 con 0x7f8ffc003f20 2023-11-24T08:10:43.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.663+0000 7f901259c640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90140cc0f0 0x7f8ffc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:43.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.664+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ffc003f20 0x7f8ffc0047c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ac0 tx=0x7f8ffc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae32aea24a0d4ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:43.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900802c6b0 con 0x7f8ffc003f20 2023-11-24T08:10:43.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900802c810 con 0x7f8ffc003f20 2023-11-24T08:10:43.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f901159a640 1 -- 172.21.15.106:0/3118247213 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900802cba0 con 0x7f8ffc003f20 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9018ae5640 1 -- 172.21.15.106:0/3118247213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ffc003f20 msgr2=0x7f8ffc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3118247213 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ffc003f20 0x7f8ffc0047c0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9008009ac0 tx=0x7f8ffc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9012d9d640 1 -- 172.21.15.106:0/3118247213 reap_dead start 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9018ae5640 1 -- 172.21.15.106:0/3118247213 shutdown_connections 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9018ae5640 1 -- 172.21.15.106:0/3118247213 >> 172.21.15.106:0/3118247213 conn(0x7f90140b0c50 msgr2=0x7f90140b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9018ae5640 1 -- 172.21.15.106:0/3118247213 shutdown_connections 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.665+0000 7f9018ae5640 1 -- 172.21.15.106:0/3118247213 wait complete. 2023-11-24T08:10:43.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.667+0000 7f9018ae5640 1 Processor -- start 2023-11-24T08:10:43.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.667+0000 7f9018ae5640 1 -- start start 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.668+0000 7f9018ae5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ffc003f20 0x7f90141534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.668+0000 7f9018ae5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 0x7f90140b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.668+0000 7f9018ae5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90141f0e50 0x7f90140ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.668+0000 7f9018ae5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90140c6200 con 0x7f90141f4490 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.668+0000 7f9018ae5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90141f4a50 con 0x7f90141f0e50 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.668+0000 7f9018ae5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90141f4d30 con 0x7f8ffc003f20 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 0x7f90140b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 0x7f90140b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:35398/0 (socket says 172.21.15.106:35398) 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 -- 172.21.15.106:0/3235347722 learned_addr learned my addr 172.21.15.106:0/3235347722 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f901259c640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ffc003f20 0x7f90141534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 -- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ffc003f20 msgr2=0x7f90141534a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ffc003f20 0x7f90141534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9012d9d640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90141f0e50 0x7f90140ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 -- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90141f0e50 msgr2=0x7f90140ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90141f0e50 0x7f90140ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 -- 172.21.15.106:0/3235347722 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90141f5010 con 0x7f90141f4490 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.669+0000 7f9011d9b640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 0x7f90140b3be0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9008037040 tx=0x7f9008004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.670+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008003810 con 0x7f90141f4490 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.670+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008003970 con 0x7f90141f4490 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.670+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008003d00 con 0x7f90141f4490 2023-11-24T08:10:43.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.670+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90140c6200 con 0x7f90141f4490 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.670+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90141539e0 con 0x7f90141f4490 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.671+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90140c6390 con 0x7f90141f4490 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.672+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f9008034440 con 0x7f90141f4490 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.673+0000 7f9002ffd640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8ff406aed0 0x7f8ff406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.673+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f90080ab9b0 con 0x7f90141f4490 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.673+0000 7f901259c640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8ff406aed0 0x7f8ff406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.673+0000 7f901259c640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8ff406aed0 0x7f8ff406d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9004001630 tx=0x7f9004009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:43.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.677+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9008076ab0 con 0x7f90141f4490 2023-11-24T08:10:43.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:43.905+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f901404e8b0 con 0x7f90141f4490 2023-11-24T08:10:44.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.428+0000 7f9002ffd640 1 -- 172.21.15.106:0/3235347722 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9008003e60 con 0x7f90141f4490 2023-11-24T08:10:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.433+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8ff406aed0 msgr2=0x7f8ff406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.433+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8ff406aed0 0x7f8ff406d390 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f9004001630 tx=0x7f9004009040 comp rx=0 tx=0).stop 2023-11-24T08:10:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.433+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 msgr2=0x7f90140b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.433+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 0x7f90140b3be0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9008037040 tx=0x7f9008004730 comp rx=0 tx=0).stop 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 shutdown_connections 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90141f0e50 0x7f90140ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90141f4490 0x7f90140b3be0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8ff406aed0 0x7f8ff406d390 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 --2- 172.21.15.106:0/3235347722 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ffc003f20 0x7f90141534a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 >> 172.21.15.106:0/3235347722 conn(0x7f90140b0c50 msgr2=0x7f90140b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 shutdown_connections 2023-11-24T08:10:44.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.434+0000 7f9018ae5640 1 -- 172.21.15.106:0/3235347722 wait complete. 2023-11-24T08:10:44.454 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:10:44.455 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:44.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.666+0000 7f3bf313c640 1 Processor -- start 2023-11-24T08:10:44.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.666+0000 7f3bf313c640 1 -- start start 2023-11-24T08:10:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf313c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3bec15c7f0 0x7f3bec06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf313c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 0x7f3bec0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf313c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0cece0 con 0x7f3bec0cdfb0 2023-11-24T08:10:44.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf313c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec156010 con 0x7f3bec15c7f0 2023-11-24T08:10:44.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf313c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec06ce30 con 0x7f3bec0cf450 2023-11-24T08:10:44.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf16d9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3bec0cdfb0 0x7f3bec0ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47288/0 (socket says 172.21.15.106:47288) 2023-11-24T08:10:44.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.668+0000 7f3bf16d9640 1 -- 172.21.15.106:0/3522981399 learned_addr learned my addr 172.21.15.106:0/3522981399 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/3522981399 >> v2:172.21.15.156:3300/0 conn(0x7f3bec15c7f0 0x7f3bec06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 108954495 0 0) 0x7f3bec0cece0 con 0x7f3bec0cdfb0 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bdc003520 con 0x7f3bec0cdfb0 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3bebfff640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 0x7f3bec0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2917335099 0 0) 0x7f3bdc003520 con 0x7f3bec0cdfb0 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 msgr2=0x7f3bec0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 0x7f3bec0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 >> v2:172.21.15.156:3300/0 conn(0x7f3bec15c7f0 msgr2=0x7f3bec06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:44.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 --2- 172.21.15.106:0/3522981399 >> v2:172.21.15.156:3300/0 conn(0x7f3bec15c7f0 0x7f3bec06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec06d040 con 0x7f3bec0cdfb0 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f3be40019f0 con 0x7f3bec0cdfb0 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 >> v1:172.21.15.106:6789/0 conn(0x7f3bec0cdfb0 legacy=0x7f3bec0ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 0x7f3bdc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc004d00 0x7f3bdc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec156010 con 0x7f3bdc003970 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec06ce30 con 0x7f3bdc004d00 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.669+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bdc00b980 con 0x7f3bec0cf450 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bebfff640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 0x7f3bdc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bf0ed8640 1 -- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 msgr2=0x7f3bdc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0cf450 0x7f3bdc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bf0ed8640 1 -- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc004d00 msgr2=0x7f3bdc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc004d00 0x7f3bdc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.670+0000 7f3bf0ed8640 1 -- 172.21.15.106:0/3522981399 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc003520 con 0x7f3bdc003970 2023-11-24T08:10:44.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.671+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009af0 tx=0x7f3bdc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49649917b1a752b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.671+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be002c710 con 0x7f3bdc003970 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.671+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be002c870 con 0x7f3bdc003970 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.671+0000 7f3beb7fe640 1 -- 172.21.15.106:0/3522981399 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be002cc00 con 0x7f3bdc003970 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf313c640 1 -- 172.21.15.106:0/3522981399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bdc003970 msgr2=0x7f3bdc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf313c640 1 --2- 172.21.15.106:0/3522981399 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bdc003970 0x7f3bdc003d90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f3be0009af0 tx=0x7f3bdc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf16d9640 1 -- 172.21.15.106:0/3522981399 reap_dead start 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf313c640 1 -- 172.21.15.106:0/3522981399 shutdown_connections 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf313c640 1 -- 172.21.15.106:0/3522981399 >> 172.21.15.106:0/3522981399 conn(0x7f3bec06a920 msgr2=0x7f3bec06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf313c640 1 -- 172.21.15.106:0/3522981399 shutdown_connections 2023-11-24T08:10:44.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.672+0000 7f3bf313c640 1 -- 172.21.15.106:0/3522981399 wait complete. 2023-11-24T08:10:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.674+0000 7f3bf313c640 1 Processor -- start 2023-11-24T08:10:44.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.674+0000 7f3bf313c640 1 -- start start 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf313c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc003970 0x7f3bec0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf313c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0ad640 0x7f3bec151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf313c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 0x7f3bec15a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf313c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec15aed0 con 0x7f3bec150090 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf313c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec150510 con 0x7f3bdc003970 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf313c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec150a10 con 0x7f3bec0ad640 2023-11-24T08:10:44.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf16d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 0x7f3bec15a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf16d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 0x7f3bec15a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42468/0 (socket says 172.21.15.106:42468) 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bf16d9640 1 -- 172.21.15.106:0/2219770821 learned_addr learned my addr 172.21.15.106:0/2219770821 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.675+0000 7f3bebfff640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0ad640 0x7f3bec151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf16d9640 1 -- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0ad640 msgr2=0x7f3bec151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf16d9640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0ad640 0x7f3bec151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc003970 0x7f3bec0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf16d9640 1 -- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc003970 msgr2=0x7f3bec0abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf16d9640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc003970 0x7f3bec0abfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf16d9640 1 -- 172.21.15.106:0/2219770821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0ac5a0 con 0x7f3bec150090 2023-11-24T08:10:44.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf16d9640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 0x7f3bec15a180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0b5270 tx=0x7f3be400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400cec0 con 0x7f3bec150090 2023-11-24T08:10:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be400a610 con 0x7f3bec150090 2023-11-24T08:10:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be400a9a0 con 0x7f3bec150090 2023-11-24T08:10:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0ac880 con 0x7f3bec150090 2023-11-24T08:10:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.676+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0ace40 con 0x7f3bec150090 2023-11-24T08:10:44.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.677+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec15e970 con 0x7f3bec150090 2023-11-24T08:10:44.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.678+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3be400ab00 con 0x7f3bec150090 2023-11-24T08:10:44.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.679+0000 7f3be97fa640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd006af00 0x7f3bd006d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:44.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.679+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3be4088da0 con 0x7f3bec150090 2023-11-24T08:10:44.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.680+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd006af00 0x7f3bd006d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:44.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.680+0000 7f3bf0ed8640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd006af00 0x7f3bd006d3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3be0036000 tx=0x7f3be0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:44.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.684+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3be4057980 con 0x7f3bec150090 2023-11-24T08:10:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.913+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bec067470 con 0x7f3bec150090 2023-11-24T08:10:44.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.914+0000 7f3be97fa640 1 -- 172.21.15.106:0/2219770821 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f3be4057510 con 0x7f3bec150090 2023-11-24T08:10:44.916 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:44.916 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:10:40.829930+0000","modified":"2023-11-24T08:10:43.906826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1634060253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1634060253},{"type":"v1","addr":"172.21.15.106:6837","nonce":1634060253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:10:44.916 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 25 2023-11-24T08:10:44.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.916+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd006af00 msgr2=0x7f3bd006d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.916+0000 7f3bf313c640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd006af00 0x7f3bd006d3c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3be0036000 tx=0x7f3be0030000 comp rx=0 tx=0).stop 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.916+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 msgr2=0x7f3bec15a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.916+0000 7f3bf313c640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 0x7f3bec15a180 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f3bec0b5270 tx=0x7f3be400ccd0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 shutdown_connections 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3bec150090 0x7f3bec15a180 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3bd006af00 0x7f3bd006d3c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3bec0ad640 0x7f3bec151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 --2- 172.21.15.106:0/2219770821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bdc003970 0x7f3bec0abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 >> 172.21.15.106:0/2219770821 conn(0x7f3bec06a920 msgr2=0x7f3bec06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 shutdown_connections 2023-11-24T08:10:44.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:44.917+0000 7f3bf313c640 1 -- 172.21.15.106:0/2219770821 wait complete. 2023-11-24T08:10:44.938 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:10:45.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.141+0000 7f8221f68640 1 Processor -- start 2023-11-24T08:10:45.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.141+0000 7f8221f68640 1 -- start start 2023-11-24T08:10:45.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.142+0000 7f8221f68640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f821c0b2250 0x7f821c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.142+0000 7f8221f68640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f821c0b2aa0 0x7f821c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f8221f68640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c15ab80 con 0x7f821c0b2630 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f8221f68640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c15a990 con 0x7f821c0b2aa0 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f8221f68640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c15e680 con 0x7f821c0b2250 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f821c0b2aa0 0x7f821c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f821c0b2aa0 0x7f821c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42472/0 (socket says 172.21.15.106:42472) 2023-11-24T08:10:45.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 -- 172.21.15.106:0/1756893391 learned_addr learned my addr 172.21.15.106:0/1756893391 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:45.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 -- 172.21.15.106:0/1756893391 >> v2:172.21.15.156:3300/0 conn(0x7f821c0b2250 msgr2=0x7f821c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:45.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 --2- 172.21.15.106:0/1756893391 >> v2:172.21.15.156:3300/0 conn(0x7f821c0b2250 0x7f821c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 -- 172.21.15.106:0/1756893391 >> v1:172.21.15.156:6789/0 conn(0x7f821c0b2630 legacy=0x7f821c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:45.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.143+0000 7f821bfff640 1 -- 172.21.15.106:0/1756893391 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c15e960 con 0x7f821c0b2aa0 2023-11-24T08:10:45.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.144+0000 7f821bfff640 1 --2- 172.21.15.106:0/1756893391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c15b080 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f821c072a70 tx=0x7f821402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d848e1ddd0ace9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:45.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f821affd640 1 -- 172.21.15.106:0/1756893391 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821402ee90 con 0x7f821c0b2aa0 2023-11-24T08:10:45.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f821affd640 1 -- 172.21.15.106:0/1756893391 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8214036440 con 0x7f821c0b2aa0 2023-11-24T08:10:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f821affd640 1 -- 172.21.15.106:0/1756893391 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82140367d0 con 0x7f821c0b2aa0 2023-11-24T08:10:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 -- 172.21.15.106:0/1756893391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 msgr2=0x7f821c15b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 --2- 172.21.15.106:0/1756893391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c15b080 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f821c072a70 tx=0x7f821402e0e0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 -- 172.21.15.106:0/1756893391 shutdown_connections 2023-11-24T08:10:45.149 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 --2- 172.21.15.106:0/1756893391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c15b080 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 --2- 172.21.15.106:0/1756893391 >> v2:172.21.15.156:3300/0 conn(0x7f821c0b2250 0x7f821c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 -- 172.21.15.106:0/1756893391 >> 172.21.15.106:0/1756893391 conn(0x7f821c06a920 msgr2=0x7f821c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 -- 172.21.15.106:0/1756893391 shutdown_connections 2023-11-24T08:10:45.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.145+0000 7f8221f68640 1 -- 172.21.15.106:0/1756893391 wait complete. 2023-11-24T08:10:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.147+0000 7f8221f68640 1 Processor -- start 2023-11-24T08:10:45.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.147+0000 7f8221f68640 1 -- start start 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.148+0000 7f8221f68640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f821c0b2250 0x7f821c1e52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.148+0000 7f8221f68640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f821c0b2630 0x7f821c15cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.148+0000 7f8221f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c1510a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.148+0000 7f8221f68640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c14aff0 con 0x7f821c0b2aa0 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.148+0000 7f8221f68640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c15d4c0 con 0x7f821c0b2630 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.148+0000 7f8221f68640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c1e4230 con 0x7f821c0b2250 2023-11-24T08:10:45.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c1510a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c1510a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42476/0 (socket says 172.21.15.106:42476) 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 -- 172.21.15.106:0/2604524592 learned_addr learned my addr 172.21.15.106:0/2604524592 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 -- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f821c0b2250 msgr2=0x7f821c1e52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821b7fe640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f821c0b2250 0x7f821c1e52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f8212ffd640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f821c0b2630 0x7f821c15cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f821c0b2250 0x7f821c1e52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 -- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f821c0b2630 msgr2=0x7f821c15cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f821c0b2630 0x7f821c15cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 -- 172.21.15.106:0/2604524592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821400b770 con 0x7f821c0b2aa0 2023-11-24T08:10:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821b7fe640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f821c0b2250 0x7f821c1e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f8212ffd640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f821c0b2630 0x7f821c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.149+0000 7f821bfff640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c1510a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f821400b2c0 tx=0x7f8214004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:45.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.150+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214034850 con 0x7f821c0b2aa0 2023-11-24T08:10:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.150+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82140349d0 con 0x7f821c0b2aa0 2023-11-24T08:10:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.150+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8214034d80 con 0x7f821c0b2aa0 2023-11-24T08:10:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.150+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c0cd7c0 con 0x7f821c0b2aa0 2023-11-24T08:10:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.150+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c1e47f0 con 0x7f821c0b2aa0 2023-11-24T08:10:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.151+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c067470 con 0x7f821c0b2aa0 2023-11-24T08:10:45.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.152+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8214041480 con 0x7f821c0b2aa0 2023-11-24T08:10:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.153+0000 7f8218ff9640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f81fc06aed0 0x7f81fc06d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.153+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82140ac740 con 0x7f821c0b2aa0 2023-11-24T08:10:45.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.153+0000 7f821b7fe640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f81fc06aed0 0x7f81fc06d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.154+0000 7f821b7fe640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f81fc06aed0 0x7f81fc06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f820c006fd0 tx=0x7f820c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:45.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.157+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8214077890 con 0x7f821c0b2aa0 2023-11-24T08:10:45.330 INFO:tasks.ceph.mds.a.smithi106.stderr:2023-11-24T08:10:45.328+0000 7f41a233a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:10:45.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.348+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f821c067470 con 0x7f821c0b2aa0 2023-11-24T08:10:45.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.350+0000 7f8218ff9640 1 -- 172.21.15.106:0/2604524592 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f821407aeb0 con 0x7f821c0b2aa0 2023-11-24T08:10:45.353 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:45.353 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":48,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:10:42.399599+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:10:45.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f81fc06aed0 msgr2=0x7f81fc06d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f81fc06aed0 0x7f81fc06d390 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f820c006fd0 tx=0x7f820c008040 comp rx=0 tx=0).stop 2023-11-24T08:10:45.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 msgr2=0x7f821c1510a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c1510a0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f821400b2c0 tx=0x7f8214004730 comp rx=0 tx=0).stop 2023-11-24T08:10:45.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 shutdown_connections 2023-11-24T08:10:45.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f821c0b2aa0 0x7f821c1510a0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f821c0b2630 0x7f821c15cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f81fc06aed0 0x7f81fc06d390 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 --2- 172.21.15.106:0/2604524592 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f821c0b2250 0x7f821c1e52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.353+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 >> 172.21.15.106:0/2604524592 conn(0x7f821c06a920 msgr2=0x7f821c0ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:45.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.354+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 shutdown_connections 2023-11-24T08:10:45.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.354+0000 7f8221f68640 1 -- 172.21.15.106:0/2604524592 wait complete. 2023-11-24T08:10:45.375 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:10:45.375 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.570+0000 7f16377e5640 1 Processor -- start 2023-11-24T08:10:45.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.570+0000 7f16377e5640 1 -- start start 2023-11-24T08:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.571+0000 7f16377e5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f163015c7f0 0x7f163006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f16377e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f16377e5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16300cece0 con 0x7f16300cdfb0 2023-11-24T08:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f16377e5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1630156010 con 0x7f163015c7f0 2023-11-24T08:10:45.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f16377e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163006ce30 con 0x7f16300cf450 2023-11-24T08:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f1634d80640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f1634d80640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:55376/0 (socket says 172.21.15.106:55376) 2023-11-24T08:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.572+0000 7f1634d80640 1 -- 172.21.15.106:0/4177288917 learned_addr learned my addr 172.21.15.106:0/4177288917 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:45.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.573+0000 7f1635581640 1 --2- 172.21.15.106:0/4177288917 >> v2:172.21.15.156:3300/0 conn(0x7f163015c7f0 0x7f163006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.573+0000 7f1634d80640 1 -- 172.21.15.106:0/4177288917 >> v2:172.21.15.156:3300/0 conn(0x7f163015c7f0 msgr2=0x7f163006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:45.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.573+0000 7f1634d80640 1 --2- 172.21.15.106:0/4177288917 >> v2:172.21.15.156:3300/0 conn(0x7f163015c7f0 0x7f163006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.573+0000 7f1634d80640 1 -- 172.21.15.106:0/4177288917 >> v1:172.21.15.156:6789/0 conn(0x7f16300cdfb0 legacy=0x7f16300ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.573+0000 7f1634d80640 1 -- 172.21.15.106:0/4177288917 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163006d040 con 0x7f16300cf450 2023-11-24T08:10:45.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.574+0000 7f1634d80640 1 --2- 172.21.15.106:0/4177288917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300cf8d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1620009870 tx=0x7f162002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d81b94c89526797 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.575+0000 7f1627fff640 1 -- 172.21.15.106:0/4177288917 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620030840 con 0x7f16300cf450 2023-11-24T08:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.575+0000 7f1627fff640 1 -- 172.21.15.106:0/4177288917 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16200309a0 con 0x7f16300cf450 2023-11-24T08:10:45.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.575+0000 7f1627fff640 1 -- 172.21.15.106:0/4177288917 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620030d30 con 0x7f16300cf450 2023-11-24T08:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.575+0000 7f16377e5640 1 -- 172.21.15.106:0/4177288917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 msgr2=0x7f16300cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.575+0000 7f16377e5640 1 --2- 172.21.15.106:0/4177288917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300cf8d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1620009870 tx=0x7f162002b980 comp rx=0 tx=0).stop 2023-11-24T08:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.576+0000 7f16377e5640 1 -- 172.21.15.106:0/4177288917 shutdown_connections 2023-11-24T08:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.576+0000 7f16377e5640 1 --2- 172.21.15.106:0/4177288917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300cf8d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.576+0000 7f16377e5640 1 --2- 172.21.15.106:0/4177288917 >> v2:172.21.15.156:3300/0 conn(0x7f163015c7f0 0x7f163006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.576+0000 7f16377e5640 1 -- 172.21.15.106:0/4177288917 >> 172.21.15.106:0/4177288917 conn(0x7f163006a920 msgr2=0x7f163006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.576+0000 7f16377e5640 1 -- 172.21.15.106:0/4177288917 shutdown_connections 2023-11-24T08:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.576+0000 7f16377e5640 1 -- 172.21.15.106:0/4177288917 wait complete. 2023-11-24T08:10:45.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.577+0000 7f16377e5640 1 Processor -- start 2023-11-24T08:10:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 -- start start 2023-11-24T08:10:45.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 0x7f1630149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f163015c7f0 0x7f16300ac050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b3550 con 0x7f16300cdfb0 2023-11-24T08:10:45.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b37a0 con 0x7f163015c7f0 2023-11-24T08:10:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.578+0000 7f16377e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300b3a80 con 0x7f16300cf450 2023-11-24T08:10:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 0x7f1630149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 0x7f1630149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42484/0 (socket says 172.21.15.106:42484) 2023-11-24T08:10:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 -- 172.21.15.106:0/19553105 learned_addr learned my addr 172.21.15.106:0/19553105 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1634d80640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635d82640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f163015c7f0 0x7f16300ac050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 -- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 msgr2=0x7f16300adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 -- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f163015c7f0 msgr2=0x7f16300ac050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f163015c7f0 0x7f16300ac050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635581640 1 -- 172.21.15.106:0/19553105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16200094d0 con 0x7f16300cdfb0 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1634d80640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.579+0000 7f1635d82640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f163015c7f0 0x7f16300ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.580+0000 7f1635581640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 0x7f1630149820 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f162c00d5f0 tx=0x7f162c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.580+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c0125f0 con 0x7f16300cdfb0 2023-11-24T08:10:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.580+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162c012750 con 0x7f16300cdfb0 2023-11-24T08:10:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.580+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162c012b00 con 0x7f16300cdfb0 2023-11-24T08:10:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.580+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163014dbc0 con 0x7f16300cdfb0 2023-11-24T08:10:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.580+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1630149d60 con 0x7f16300cdfb0 2023-11-24T08:10:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.581+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630067470 con 0x7f16300cdfb0 2023-11-24T08:10:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.582+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f162c012c60 con 0x7f16300cdfb0 2023-11-24T08:10:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.583+0000 7f1625ffb640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161406aed0 0x7f161406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.583+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f162c089a80 con 0x7f16300cdfb0 2023-11-24T08:10:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.583+0000 7f1634d80640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161406aed0 0x7f161406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.584+0000 7f1634d80640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161406aed0 0x7f161406d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1620036000 tx=0x7f162003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.587+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f162c054bd0 con 0x7f16300cdfb0 2023-11-24T08:10:45.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:10:45.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:10:45.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.774+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f163014da40 con 0x7f161406aed0 2023-11-24T08:10:45.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.783+0000 7f1625ffb640 1 -- 172.21.15.106:0/19553105 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275515 (secure 0 0 0) 0x7f1620030b90 con 0x7f161406aed0 2023-11-24T08:10:45.788 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:45.789 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:45.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161406aed0 msgr2=0x7f161406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161406aed0 0x7f161406d390 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1620036000 tx=0x7f162003b040 comp rx=0 tx=0).stop 2023-11-24T08:10:45.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 msgr2=0x7f1630149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:45.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 0x7f1630149820 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f162c00d5f0 tx=0x7f162c00dab0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 shutdown_connections 2023-11-24T08:10:45.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f163015c7f0 0x7f16300ac050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161406aed0 0x7f161406d390 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300cf450 0x7f16300adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 --2- 172.21.15.106:0/19553105 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300cdfb0 0x7f1630149820 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:45.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.787+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 >> 172.21.15.106:0/19553105 conn(0x7f163006a920 msgr2=0x7f16300ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:45.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.788+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 shutdown_connections 2023-11-24T08:10:45.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:45.788+0000 7f16377e5640 1 -- 172.21.15.106:0/19553105 wait complete. 2023-11-24T08:10:45.807 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-11-24T08:10:45.308166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238060,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736660,"statfs":{"total":773094113280,"available":772850339840,"internally_reserved":0,"allocated":5382144,"data_stored":2854272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.955123"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.378983+0000","last_change":"2023-11-24T08:10:40.378983+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.378983+0000","last_undegraded":"2023-11-24T08:10:40.378983+0000","last_fullsized":"2023-11-24T08:10:40.378983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.379469+0000","last_change":"2023-11-24T08:10:40.379469+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.379469+0000","last_undegraded":"2023-11-24T08:10:40.379469+0000","last_fullsized":"2023-11-24T08:10:40.379469+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.381829+0000","last_change":"2023-11-24T08:10:40.381829+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.381829+0000","last_undegraded":"2023-11-24T08:10:40.381829+0000","last_fullsized":"2023-11-24T08:10:40.381829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.380043+0000","last_change":"2023-11-24T08:10:40.380043+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.380043+0000","last_undegraded":"2023-11-24T08:10:40.380043+0000","last_fullsized":"2023-11-24T08:10:40.380043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361522+0000","last_change":"2023-11-24T08:10:39.361522+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361522+0000","last_undegraded":"2023-11-24T08:10:39.361522+0000","last_fullsized":"2023-11-24T08:10:39.361522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.366172+0000","last_change":"2023-11-24T08:10:39.366172+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.366172+0000","last_undegraded":"2023-11-24T08:10:39.366172+0000","last_fullsized":"2023-11-24T08:10:39.366172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.378875+0000","last_change":"2023-11-24T08:10:40.378875+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.378875+0000","last_undegraded":"2023-11-24T08:10:40.378875+0000","last_fullsized":"2023-11-24T08:10:40.378875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372156+0000","last_change":"2023-11-24T08:10:39.372156+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372156+0000","last_undegraded":"2023-11-24T08:10:39.372156+0000","last_fullsized":"2023-11-24T08:10:39.372156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359062+0000","last_change":"2023-11-24T08:10:39.359062+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359062+0000","last_undegraded":"2023-11-24T08:10:39.359062+0000","last_fullsized":"2023-11-24T08:10:39.359062+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.362909+0000","last_change":"2023-11-24T08:10:39.362909+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.362909+0000","last_undegraded":"2023-11-24T08:10:39.362909+0000","last_fullsized":"2023-11-24T08:10:39.362909+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.365806+0000","last_change":"2023-11-24T08:10:39.365806+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.365806+0000","last_undegraded":"2023-11-24T08:10:39.365806+0000","last_fullsized":"2023-11-24T08:10:39.365806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.398927+0000","last_change":"2023-11-24T08:10:40.390774+0000","last_active":"2023-11-24T08:10:40.398927+0000","last_peered":"2023-11-24T08:10:40.398927+0000","last_clean":"2023-11-24T08:10:40.398927+0000","last_became_active":"2023-11-24T08:10:40.390495+0000","last_became_peered":"2023-11-24T08:10:40.390495+0000","last_unstale":"2023-11-24T08:10:40.398927+0000","last_undegraded":"2023-11-24T08:10:40.398927+0000","last_fullsized":"2023-11-24T08:10:40.398927+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:21:13.137284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.370230+0000","last_change":"2023-11-24T08:10:39.370230+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.370230+0000","last_undegraded":"2023-11-24T08:10:39.370230+0000","last_fullsized":"2023-11-24T08:10:39.370230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364602+0000","last_change":"2023-11-24T08:10:39.364602+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364602+0000","last_undegraded":"2023-11-24T08:10:39.364602+0000","last_fullsized":"2023-11-24T08:10:39.364602+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.368230+0000","last_change":"2023-11-24T08:10:39.368230+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.368230+0000","last_undegraded":"2023-11-24T08:10:39.368230+0000","last_fullsized":"2023-11-24T08:10:39.368230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.360521+0000","last_change":"2023-11-24T08:10:39.360521+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.360521+0000","last_undegraded":"2023-11-24T08:10:39.360521+0000","last_fullsized":"2023-11-24T08:10:39.360521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":46,"state":"creating+activating","last_fresh":"2023-11-24T08:10:40.362293+0000","last_change":"2023-11-24T08:10:40.362293+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.362293+0000","last_undegraded":"2023-11-24T08:10:40.362293+0000","last_fullsized":"2023-11-24T08:10:40.362293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:15.573547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.362284+0000","last_change":"2023-11-24T08:10:39.362284+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.362284+0000","last_undegraded":"2023-11-24T08:10:39.362284+0000","last_fullsized":"2023-11-24T08:10:39.362284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359021+0000","last_change":"2023-11-24T08:10:39.359021+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359021+0000","last_undegraded":"2023-11-24T08:10:39.359021+0000","last_fullsized":"2023-11-24T08:10:39.359021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361496+0000","last_change":"2023-11-24T08:10:39.361496+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361496+0000","last_undegraded":"2023-11-24T08:10:39.361496+0000","last_fullsized":"2023-11-24T08:10:39.361496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.369002+0000","last_change":"2023-11-24T08:10:39.369002+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.369002+0000","last_undegraded":"2023-11-24T08:10:39.369002+0000","last_fullsized":"2023-11-24T08:10:39.369002+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":3,"reported_epoch":46,"state":"creating+activating","last_fresh":"2023-11-24T08:10:40.362269+0000","last_change":"2023-11-24T08:10:40.362269+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.362269+0000","last_undegraded":"2023-11-24T08:10:40.362269+0000","last_fullsized":"2023-11-24T08:10:40.362269+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:04.394116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364770+0000","last_change":"2023-11-24T08:10:39.364770+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364770+0000","last_undegraded":"2023-11-24T08:10:39.364770+0000","last_fullsized":"2023-11-24T08:10:39.364770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.398881+0000","last_change":"2023-11-24T08:10:40.390983+0000","last_active":"2023-11-24T08:10:40.398881+0000","last_peered":"2023-11-24T08:10:40.398881+0000","last_clean":"2023-11-24T08:10:40.398881+0000","last_became_active":"2023-11-24T08:10:40.389721+0000","last_became_peered":"2023-11-24T08:10:40.389721+0000","last_unstale":"2023-11-24T08:10:40.398881+0000","last_undegraded":"2023-11-24T08:10:40.398881+0000","last_fullsized":"2023-11-24T08:10:40.398881+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:01:07.182397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361652+0000","last_change":"2023-11-24T08:10:39.361652+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361652+0000","last_undegraded":"2023-11-24T08:10:39.361652+0000","last_fullsized":"2023-11-24T08:10:39.361652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361525+0000","last_change":"2023-11-24T08:10:39.361525+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361525+0000","last_undegraded":"2023-11-24T08:10:39.361525+0000","last_fullsized":"2023-11-24T08:10:39.361525+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.399131+0000","last_change":"2023-11-24T08:10:40.393936+0000","last_active":"2023-11-24T08:10:40.399131+0000","last_peered":"2023-11-24T08:10:40.399131+0000","last_clean":"2023-11-24T08:10:40.399131+0000","last_became_active":"2023-11-24T08:10:40.393617+0000","last_became_peered":"2023-11-24T08:10:40.393617+0000","last_unstale":"2023-11-24T08:10:40.399131+0000","last_undegraded":"2023-11-24T08:10:40.399131+0000","last_fullsized":"2023-11-24T08:10:40.399131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:38:50.199344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.367587+0000","last_change":"2023-11-24T08:10:39.367587+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.367587+0000","last_undegraded":"2023-11-24T08:10:39.367587+0000","last_fullsized":"2023-11-24T08:10:39.367587+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359004+0000","last_change":"2023-11-24T08:10:39.359004+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359004+0000","last_undegraded":"2023-11-24T08:10:39.359004+0000","last_fullsized":"2023-11-24T08:10:39.359004+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361744+0000","last_change":"2023-11-24T08:10:39.361744+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361744+0000","last_undegraded":"2023-11-24T08:10:39.361744+0000","last_fullsized":"2023-11-24T08:10:39.361744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359056+0000","last_change":"2023-11-24T08:10:39.359056+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359056+0000","last_undegraded":"2023-11-24T08:10:39.359056+0000","last_fullsized":"2023-11-24T08:10:39.359056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372125+0000","last_change":"2023-11-24T08:10:39.372125+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372125+0000","last_undegraded":"2023-11-24T08:10:39.372125+0000","last_fullsized":"2023-11-24T08:10:39.372125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.358939+0000","last_change":"2023-11-24T08:10:39.358939+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.358939+0000","last_undegraded":"2023-11-24T08:10:39.358939+0000","last_fullsized":"2023-11-24T08:10:39.358939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364515+0000","last_change":"2023-11-24T08:10:39.364515+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364515+0000","last_undegraded":"2023-11-24T08:10:39.364515+0000","last_fullsized":"2023-11-24T08:10:39.364515+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.379378+0000","last_change":"2023-11-24T08:10:40.379378+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.379378+0000","last_undegraded":"2023-11-24T08:10:40.379378+0000","last_fullsized":"2023-11-24T08:10:40.379378+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372024+0000","last_change":"2023-11-24T08:10:39.372024+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372024+0000","last_undegraded":"2023-11-24T08:10:39.372024+0000","last_fullsized":"2023-11-24T08:10:39.372024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.367903+0000","last_change":"2023-11-24T08:10:39.367903+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.367903+0000","last_undegraded":"2023-11-24T08:10:39.367903+0000","last_fullsized":"2023-11-24T08:10:39.367903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.399653+0000","last_change":"2023-11-24T08:10:40.396648+0000","last_active":"2023-11-24T08:10:40.399653+0000","last_peered":"2023-11-24T08:10:40.399653+0000","last_clean":"2023-11-24T08:10:40.399653+0000","last_became_active":"2023-11-24T08:10:40.395757+0000","last_became_peered":"2023-11-24T08:10:40.395757+0000","last_unstale":"2023-11-24T08:10:40.399653+0000","last_undegraded":"2023-11-24T08:10:40.399653+0000","last_fullsized":"2023-11-24T08:10:40.399653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:14:25.355904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":183,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:10:37.580718+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:10:37.580718+0000","last_peered":"2023-11-24T08:10:37.580718+0000","last_clean":"2023-11-24T08:10:37.580718+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:10:37.580718+0000","last_undegraded":"2023-11-24T08:10:37.580718+0000","last_fullsized":"2023-11-24T08:10:37.580718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":799560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.341},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2230000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.6990000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.276},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.9390000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.3839999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.802}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.091},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2470000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.474}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.371},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.7789999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.631},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.04},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2350000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.353}]},{"osd":2,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.415},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.84499999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.286}]},{"osd":4,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":1.524},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.4890000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.581},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.645}]},{"osd":7,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.621},{"interface":"front","average":{"1min":1.752,"5min":1.752,"15min":1.752},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.9689999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.4590000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.52},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.84399999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.335},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.133}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.571},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.409}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.073}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.6279999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4710000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":1.333}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.198}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.53}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.3},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.157}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.208}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.143},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.111}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.4370000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3979999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.472}]},{"osd":7,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.573}]}]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.038},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.262},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.7709999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":799560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.113}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.151}]},{"osd":4,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.51}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.393},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:45.813 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:46.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.086+0000 7f487f1fb640 1 Processor -- start 2023-11-24T08:10:46.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.086+0000 7f487f1fb640 1 -- start start 2023-11-24T08:10:46.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.087+0000 7f487f1fb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4878159e80 0x7f4878071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487f1fb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f487806d5c0 0x7f4878163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487f1fb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48780ce870 con 0x7f48780cd6a0 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487f1fb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48780cdc60 con 0x7f4878159e80 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487f1fb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48780cdf40 con 0x7f487806d5c0 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4878159e80 0x7f4878071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4878159e80 0x7f4878071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42486/0 (socket says 172.21.15.106:42486) 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 -- 172.21.15.106:0/2766011867 learned_addr learned my addr 172.21.15.106:0/2766011867 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 -- 172.21.15.106:0/2766011867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f487806d5c0 msgr2=0x7f4878163430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 --2- 172.21.15.106:0/2766011867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f487806d5c0 0x7f4878163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 -- 172.21.15.106:0/2766011867 >> v1:172.21.15.156:6789/0 conn(0x7f48780cd6a0 legacy=0x7f487815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:46.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.088+0000 7f487e1f9640 1 -- 172.21.15.106:0/2766011867 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48780ce220 con 0x7f4878159e80 2023-11-24T08:10:46.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.089+0000 7f487e1f9640 1 --2- 172.21.15.106:0/2766011867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878071610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4874009870 tx=0x7f487402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28072ed93f0d4215 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:46.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487d1f7640 1 -- 172.21.15.106:0/2766011867 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48740317c0 con 0x7f4878159e80 2023-11-24T08:10:46.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487d1f7640 1 -- 172.21.15.106:0/2766011867 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4874031920 con 0x7f4878159e80 2023-11-24T08:10:46.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487d1f7640 1 -- 172.21.15.106:0/2766011867 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4874031cb0 con 0x7f4878159e80 2023-11-24T08:10:46.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487f1fb640 1 -- 172.21.15.106:0/2766011867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 msgr2=0x7f4878071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487f1fb640 1 --2- 172.21.15.106:0/2766011867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878071610 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4874009870 tx=0x7f487402b860 comp rx=0 tx=0).stop 2023-11-24T08:10:46.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487f1fb640 1 -- 172.21.15.106:0/2766011867 shutdown_connections 2023-11-24T08:10:46.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487f1fb640 1 --2- 172.21.15.106:0/2766011867 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f487806d5c0 0x7f4878163430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487f1fb640 1 --2- 172.21.15.106:0/2766011867 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878071610 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.090+0000 7f487f1fb640 1 -- 172.21.15.106:0/2766011867 >> 172.21.15.106:0/2766011867 conn(0x7f48780cb830 msgr2=0x7f48780cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:46.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.091+0000 7f487f1fb640 1 -- 172.21.15.106:0/2766011867 shutdown_connections 2023-11-24T08:10:46.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.091+0000 7f487f1fb640 1 -- 172.21.15.106:0/2766011867 wait complete. 2023-11-24T08:10:46.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.092+0000 7f487f1fb640 1 Processor -- start 2023-11-24T08:10:46.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.092+0000 7f487f1fb640 1 -- start start 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.093+0000 7f487f1fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487806d5c0 0x7f4878142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.093+0000 7f487f1fb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48780cd6a0 0x7f487813eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.093+0000 7f487f1fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.093+0000 7f487f1fb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4878140720 con 0x7f4878159e80 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.093+0000 7f487f1fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48781427c0 con 0x7f487806d5c0 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.093+0000 7f487f1fb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487813fb10 con 0x7f48780cd6a0 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878142cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42502/0 (socket says 172.21.15.106:42502) 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 -- 172.21.15.106:0/867083416 learned_addr learned my addr 172.21.15.106:0/867083416 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487d9f8640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48780cd6a0 0x7f487813eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 -- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48780cd6a0 msgr2=0x7f487813eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48780cd6a0 0x7f487813eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e1f9640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487806d5c0 0x7f4878142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 -- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487806d5c0 msgr2=0x7f4878142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487806d5c0 0x7f4878142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 -- 172.21.15.106:0/867083416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740094d0 con 0x7f4878159e80 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487d9f8640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48780cd6a0 0x7f487813eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:46.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.094+0000 7f487e9fa640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878142cd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f487000b970 tx=0x7f487000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.095+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487000cb40 con 0x7f4878159e80 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.095+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f487000ccc0 con 0x7f4878159e80 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.095+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4870014680 con 0x7f4878159e80 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.095+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f487813fdf0 con 0x7f4878159e80 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.095+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48781403b0 con 0x7f4878159e80 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.096+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4878140890 con 0x7f4878159e80 2023-11-24T08:10:46.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.097+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f487001d3b0 con 0x7f4878159e80 2023-11-24T08:10:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.098+0000 7f486effd640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f485c06af00 0x7f485c06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.098+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4870089620 con 0x7f4878159e80 2023-11-24T08:10:46.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.098+0000 7f487e1f9640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f485c06af00 0x7f485c06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.099+0000 7f487e1f9640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f485c06af00 0x7f485c06d3c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4874036000 tx=0x7f487403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:46.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.102+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4870054670 con 0x7f4878159e80 2023-11-24T08:10:46.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.288+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4878070940 con 0x7f485c06af00 2023-11-24T08:10:46.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.299+0000 7f486effd640 1 -- 172.21.15.106:0/867083416 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275515 (secure 0 0 0) 0x7f4874031c10 con 0x7f485c06af00 2023-11-24T08:10:46.302 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:46.306 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:46.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f485c06af00 msgr2=0x7f485c06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f485c06af00 0x7f485c06d3c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4874036000 tx=0x7f487403d040 comp rx=0 tx=0).stop 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 msgr2=0x7f4878142cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878142cd0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f487000b970 tx=0x7f487000be30 comp rx=0 tx=0).stop 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 shutdown_connections 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4878159e80 0x7f4878142cd0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f485c06af00 0x7f485c06d3c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48780cd6a0 0x7f487813eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 --2- 172.21.15.106:0/867083416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f487806d5c0 0x7f4878142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.305+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 >> 172.21.15.106:0/867083416 conn(0x7f48780cb830 msgr2=0x7f48780cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.306+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 shutdown_connections 2023-11-24T08:10:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.306+0000 7f487f1fb640 1 -- 172.21.15.106:0/867083416 wait complete. 2023-11-24T08:10:46.326 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-11-24T08:10:45.308166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238060,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736660,"statfs":{"total":773094113280,"available":772850339840,"internally_reserved":0,"allocated":5382144,"data_stored":2854272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.955123"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.378983+0000","last_change":"2023-11-24T08:10:40.378983+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.378983+0000","last_undegraded":"2023-11-24T08:10:40.378983+0000","last_fullsized":"2023-11-24T08:10:40.378983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.379469+0000","last_change":"2023-11-24T08:10:40.379469+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.379469+0000","last_undegraded":"2023-11-24T08:10:40.379469+0000","last_fullsized":"2023-11-24T08:10:40.379469+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.381829+0000","last_change":"2023-11-24T08:10:40.381829+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.381829+0000","last_undegraded":"2023-11-24T08:10:40.381829+0000","last_fullsized":"2023-11-24T08:10:40.381829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.380043+0000","last_change":"2023-11-24T08:10:40.380043+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.380043+0000","last_undegraded":"2023-11-24T08:10:40.380043+0000","last_fullsized":"2023-11-24T08:10:40.380043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361522+0000","last_change":"2023-11-24T08:10:39.361522+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361522+0000","last_undegraded":"2023-11-24T08:10:39.361522+0000","last_fullsized":"2023-11-24T08:10:39.361522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.366172+0000","last_change":"2023-11-24T08:10:39.366172+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.366172+0000","last_undegraded":"2023-11-24T08:10:39.366172+0000","last_fullsized":"2023-11-24T08:10:39.366172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.378875+0000","last_change":"2023-11-24T08:10:40.378875+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.378875+0000","last_undegraded":"2023-11-24T08:10:40.378875+0000","last_fullsized":"2023-11-24T08:10:40.378875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372156+0000","last_change":"2023-11-24T08:10:39.372156+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372156+0000","last_undegraded":"2023-11-24T08:10:39.372156+0000","last_fullsized":"2023-11-24T08:10:39.372156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359062+0000","last_change":"2023-11-24T08:10:39.359062+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359062+0000","last_undegraded":"2023-11-24T08:10:39.359062+0000","last_fullsized":"2023-11-24T08:10:39.359062+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.362909+0000","last_change":"2023-11-24T08:10:39.362909+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.362909+0000","last_undegraded":"2023-11-24T08:10:39.362909+0000","last_fullsized":"2023-11-24T08:10:39.362909+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.365806+0000","last_change":"2023-11-24T08:10:39.365806+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.365806+0000","last_undegraded":"2023-11-24T08:10:39.365806+0000","last_fullsized":"2023-11-24T08:10:39.365806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.398927+0000","last_change":"2023-11-24T08:10:40.390774+0000","last_active":"2023-11-24T08:10:40.398927+0000","last_peered":"2023-11-24T08:10:40.398927+0000","last_clean":"2023-11-24T08:10:40.398927+0000","last_became_active":"2023-11-24T08:10:40.390495+0000","last_became_peered":"2023-11-24T08:10:40.390495+0000","last_unstale":"2023-11-24T08:10:40.398927+0000","last_undegraded":"2023-11-24T08:10:40.398927+0000","last_fullsized":"2023-11-24T08:10:40.398927+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:21:13.137284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.370230+0000","last_change":"2023-11-24T08:10:39.370230+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.370230+0000","last_undegraded":"2023-11-24T08:10:39.370230+0000","last_fullsized":"2023-11-24T08:10:39.370230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364602+0000","last_change":"2023-11-24T08:10:39.364602+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364602+0000","last_undegraded":"2023-11-24T08:10:39.364602+0000","last_fullsized":"2023-11-24T08:10:39.364602+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.368230+0000","last_change":"2023-11-24T08:10:39.368230+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.368230+0000","last_undegraded":"2023-11-24T08:10:39.368230+0000","last_fullsized":"2023-11-24T08:10:39.368230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.360521+0000","last_change":"2023-11-24T08:10:39.360521+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.360521+0000","last_undegraded":"2023-11-24T08:10:39.360521+0000","last_fullsized":"2023-11-24T08:10:39.360521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":46,"state":"creating+activating","last_fresh":"2023-11-24T08:10:40.362293+0000","last_change":"2023-11-24T08:10:40.362293+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.362293+0000","last_undegraded":"2023-11-24T08:10:40.362293+0000","last_fullsized":"2023-11-24T08:10:40.362293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:15.573547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.362284+0000","last_change":"2023-11-24T08:10:39.362284+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.362284+0000","last_undegraded":"2023-11-24T08:10:39.362284+0000","last_fullsized":"2023-11-24T08:10:39.362284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359021+0000","last_change":"2023-11-24T08:10:39.359021+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359021+0000","last_undegraded":"2023-11-24T08:10:39.359021+0000","last_fullsized":"2023-11-24T08:10:39.359021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361496+0000","last_change":"2023-11-24T08:10:39.361496+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361496+0000","last_undegraded":"2023-11-24T08:10:39.361496+0000","last_fullsized":"2023-11-24T08:10:39.361496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.369002+0000","last_change":"2023-11-24T08:10:39.369002+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.369002+0000","last_undegraded":"2023-11-24T08:10:39.369002+0000","last_fullsized":"2023-11-24T08:10:39.369002+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":3,"reported_epoch":46,"state":"creating+activating","last_fresh":"2023-11-24T08:10:40.362269+0000","last_change":"2023-11-24T08:10:40.362269+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.362269+0000","last_undegraded":"2023-11-24T08:10:40.362269+0000","last_fullsized":"2023-11-24T08:10:40.362269+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:04.394116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364770+0000","last_change":"2023-11-24T08:10:39.364770+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364770+0000","last_undegraded":"2023-11-24T08:10:39.364770+0000","last_fullsized":"2023-11-24T08:10:39.364770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.398881+0000","last_change":"2023-11-24T08:10:40.390983+0000","last_active":"2023-11-24T08:10:40.398881+0000","last_peered":"2023-11-24T08:10:40.398881+0000","last_clean":"2023-11-24T08:10:40.398881+0000","last_became_active":"2023-11-24T08:10:40.389721+0000","last_became_peered":"2023-11-24T08:10:40.389721+0000","last_unstale":"2023-11-24T08:10:40.398881+0000","last_undegraded":"2023-11-24T08:10:40.398881+0000","last_fullsized":"2023-11-24T08:10:40.398881+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:01:07.182397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361652+0000","last_change":"2023-11-24T08:10:39.361652+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361652+0000","last_undegraded":"2023-11-24T08:10:39.361652+0000","last_fullsized":"2023-11-24T08:10:39.361652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361525+0000","last_change":"2023-11-24T08:10:39.361525+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361525+0000","last_undegraded":"2023-11-24T08:10:39.361525+0000","last_fullsized":"2023-11-24T08:10:39.361525+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.399131+0000","last_change":"2023-11-24T08:10:40.393936+0000","last_active":"2023-11-24T08:10:40.399131+0000","last_peered":"2023-11-24T08:10:40.399131+0000","last_clean":"2023-11-24T08:10:40.399131+0000","last_became_active":"2023-11-24T08:10:40.393617+0000","last_became_peered":"2023-11-24T08:10:40.393617+0000","last_unstale":"2023-11-24T08:10:40.399131+0000","last_undegraded":"2023-11-24T08:10:40.399131+0000","last_fullsized":"2023-11-24T08:10:40.399131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:38:50.199344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.367587+0000","last_change":"2023-11-24T08:10:39.367587+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.367587+0000","last_undegraded":"2023-11-24T08:10:39.367587+0000","last_fullsized":"2023-11-24T08:10:39.367587+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359004+0000","last_change":"2023-11-24T08:10:39.359004+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359004+0000","last_undegraded":"2023-11-24T08:10:39.359004+0000","last_fullsized":"2023-11-24T08:10:39.359004+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361744+0000","last_change":"2023-11-24T08:10:39.361744+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361744+0000","last_undegraded":"2023-11-24T08:10:39.361744+0000","last_fullsized":"2023-11-24T08:10:39.361744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359056+0000","last_change":"2023-11-24T08:10:39.359056+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359056+0000","last_undegraded":"2023-11-24T08:10:39.359056+0000","last_fullsized":"2023-11-24T08:10:39.359056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372125+0000","last_change":"2023-11-24T08:10:39.372125+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372125+0000","last_undegraded":"2023-11-24T08:10:39.372125+0000","last_fullsized":"2023-11-24T08:10:39.372125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.358939+0000","last_change":"2023-11-24T08:10:39.358939+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.358939+0000","last_undegraded":"2023-11-24T08:10:39.358939+0000","last_fullsized":"2023-11-24T08:10:39.358939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364515+0000","last_change":"2023-11-24T08:10:39.364515+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364515+0000","last_undegraded":"2023-11-24T08:10:39.364515+0000","last_fullsized":"2023-11-24T08:10:39.364515+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.379378+0000","last_change":"2023-11-24T08:10:40.379378+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.379378+0000","last_undegraded":"2023-11-24T08:10:40.379378+0000","last_fullsized":"2023-11-24T08:10:40.379378+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372024+0000","last_change":"2023-11-24T08:10:39.372024+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372024+0000","last_undegraded":"2023-11-24T08:10:39.372024+0000","last_fullsized":"2023-11-24T08:10:39.372024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.367903+0000","last_change":"2023-11-24T08:10:39.367903+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.367903+0000","last_undegraded":"2023-11-24T08:10:39.367903+0000","last_fullsized":"2023-11-24T08:10:39.367903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.399653+0000","last_change":"2023-11-24T08:10:40.396648+0000","last_active":"2023-11-24T08:10:40.399653+0000","last_peered":"2023-11-24T08:10:40.399653+0000","last_clean":"2023-11-24T08:10:40.399653+0000","last_became_active":"2023-11-24T08:10:40.395757+0000","last_became_peered":"2023-11-24T08:10:40.395757+0000","last_unstale":"2023-11-24T08:10:40.399653+0000","last_undegraded":"2023-11-24T08:10:40.399653+0000","last_fullsized":"2023-11-24T08:10:40.399653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:14:25.355904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":183,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:10:37.580718+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:10:37.580718+0000","last_peered":"2023-11-24T08:10:37.580718+0000","last_clean":"2023-11-24T08:10:37.580718+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:10:37.580718+0000","last_undegraded":"2023-11-24T08:10:37.580718+0000","last_fullsized":"2023-11-24T08:10:37.580718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":799560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.341},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2230000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.6990000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.276},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.9390000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.3839999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.802}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.091},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2470000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.474}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.371},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.7789999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.631},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.04},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2350000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.353}]},{"osd":2,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.415},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.84499999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.286}]},{"osd":4,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":1.524},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.4890000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.581},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.645}]},{"osd":7,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.621},{"interface":"front","average":{"1min":1.752,"5min":1.752,"15min":1.752},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.9689999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.4590000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.52},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.84399999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.335},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.133}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.571},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.409}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.073}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.6279999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4710000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":1.333}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.198}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.53}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.3},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.157}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.208}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.143},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.111}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.4370000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3979999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.472}]},{"osd":7,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.573}]}]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.038},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.262},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.7709999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":799560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.113}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.151}]},{"osd":4,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.51}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.393},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:46.331 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:46.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.587+0000 7f8df7fff640 1 Processor -- start 2023-11-24T08:10:46.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.587+0000 7f8df7fff640 1 -- start start 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8df815aff0 0x7f8df815e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8df80b2cf0 0x7f8df806ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df7fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df806a680 con 0x7f8df8142b70 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df7fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df8142fd0 con 0x7f8df815aff0 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df7fff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df81432a0 con 0x7f8df80b2cf0 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8df815aff0 0x7f8df815e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df67fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8df8142b70 0x7f8df80b25e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47292/0 (socket says 172.21.15.106:47292) 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df6ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8df815aff0 0x7f8df815e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42508/0 (socket says 172.21.15.106:42508) 2023-11-24T08:10:46.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.589+0000 7f8df67fc640 1 -- 172.21.15.106:0/1572953222 learned_addr learned my addr 172.21.15.106:0/1572953222 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:46.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.590+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1572953222 >> v2:172.21.15.156:3300/0 conn(0x7f8df80b2cf0 msgr2=0x7f8df806ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:46.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.590+0000 7f8df6ffd640 1 --2- 172.21.15.106:0/1572953222 >> v2:172.21.15.156:3300/0 conn(0x7f8df80b2cf0 0x7f8df806ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.590+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1572953222 >> v1:172.21.15.106:6789/0 conn(0x7f8df8142b70 legacy=0x7f8df80b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.590+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1572953222 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8143580 con 0x7f8df815aff0 2023-11-24T08:10:46.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.591+0000 7f8df6ffd640 1 --2- 172.21.15.106:0/1572953222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df815aff0 0x7f8df815e3a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8df80d0330 tx=0x7f8de802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c237a1449ba25f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:46.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.591+0000 7f8df5ffb640 1 -- 172.21.15.106:0/1572953222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8031830 con 0x7f8df815aff0 2023-11-24T08:10:46.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.591+0000 7f8df5ffb640 1 -- 172.21.15.106:0/1572953222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de8031990 con 0x7f8df815aff0 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.591+0000 7f8df5ffb640 1 -- 172.21.15.106:0/1572953222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de8031d20 con 0x7f8df815aff0 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 -- 172.21.15.106:0/1572953222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df815aff0 msgr2=0x7f8df815e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1572953222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df815aff0 0x7f8df815e3a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f8df80d0330 tx=0x7f8de802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 -- 172.21.15.106:0/1572953222 shutdown_connections 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1572953222 >> v2:172.21.15.156:3300/0 conn(0x7f8df80b2cf0 0x7f8df806ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1572953222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df815aff0 0x7f8df815e3a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 -- 172.21.15.106:0/1572953222 >> 172.21.15.106:0/1572953222 conn(0x7f8df80cf610 msgr2=0x7f8df80cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 -- 172.21.15.106:0/1572953222 shutdown_connections 2023-11-24T08:10:46.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.592+0000 7f8df7fff640 1 -- 172.21.15.106:0/1572953222 wait complete. 2023-11-24T08:10:46.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.594+0000 7f8df7fff640 1 Processor -- start 2023-11-24T08:10:46.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.594+0000 7f8df7fff640 1 -- start start 2023-11-24T08:10:46.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 0x7f8df81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df8142b70 0x7f8df8143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df7fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df815aff0 0x7f8df8143fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df7fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df815dff0 con 0x7f8df80b2cf0 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df7fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df8141ff0 con 0x7f8df8142b70 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df7fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df81422d0 con 0x7f8df815aff0 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 0x7f8df81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 0x7f8df81597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42522/0 (socket says 172.21.15.106:42522) 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1284143776 learned_addr learned my addr 172.21.15.106:0/1284143776 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df67fc640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df8142b70 0x7f8df8143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df815aff0 msgr2=0x7f8df8143fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df815aff0 0x7f8df8143fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df8142b70 msgr2=0x7f8df8143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df8142b70 0x7f8df8143aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.596+0000 7f8df6ffd640 1 -- 172.21.15.106:0/1284143776 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de8009400 con 0x7f8df80b2cf0 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.597+0000 7f8df6ffd640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 0x7f8df81597b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000c00 tx=0x7f8de80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.597+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de803aa60 con 0x7f8df80b2cf0 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.597+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de803abe0 con 0x7f8df80b2cf0 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.597+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df815e2b0 con 0x7f8df80b2cf0 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.597+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80443f0 con 0x7f8df80b2cf0 2023-11-24T08:10:46.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.597+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df8141300 con 0x7f8df80b2cf0 2023-11-24T08:10:46.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.598+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df804e840 con 0x7f8df80b2cf0 2023-11-24T08:10:46.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.600+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8de804d450 con 0x7f8df80b2cf0 2023-11-24T08:10:46.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.600+0000 7f8ddffff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8de406aed0 0x7f8de406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:46.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.601+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8de80aba10 con 0x7f8df80b2cf0 2023-11-24T08:10:46.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.601+0000 7f8df67fc640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8de406aed0 0x7f8de406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:46.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.601+0000 7f8df67fc640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8de406aed0 0x7f8de406d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8df80b29e0 tx=0x7f8dec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:46.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.606+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8de8076950 con 0x7f8df80b2cf0 2023-11-24T08:10:46.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.792+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8df815de70 con 0x7f8de406aed0 2023-11-24T08:10:46.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.813+0000 7f8ddffff640 1 -- 172.21.15.106:0/1284143776 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275515 (secure 0 0 0) 0x7f8dec0013f0 con 0x7f8de406aed0 2023-11-24T08:10:46.818 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:46.820 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.818+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8de406aed0 msgr2=0x7f8de406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.818+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8de406aed0 0x7f8de406d390 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8df80b29e0 tx=0x7f8dec009040 comp rx=0 tx=0).stop 2023-11-24T08:10:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.818+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 msgr2=0x7f8df81597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:46.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.818+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 0x7f8df81597b0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f8de8000c00 tx=0x7f8de80046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 shutdown_connections 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8de406aed0 0x7f8de406d390 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8df815aff0 0x7f8df8143fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8df8142b70 0x7f8df8143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 --2- 172.21.15.106:0/1284143776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8df80b2cf0 0x7f8df81597b0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 >> 172.21.15.106:0/1284143776 conn(0x7f8df80cf610 msgr2=0x7f8df806d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 shutdown_connections 2023-11-24T08:10:46.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:46.819+0000 7f8df7fff640 1 -- 172.21.15.106:0/1284143776 wait complete. 2023-11-24T08:10:46.838 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2023-11-24T08:10:45.308166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":78,"acting":78,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":238060,"kb_used_data":5256,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736660,"statfs":{"total":773094113280,"available":772850339840,"internally_reserved":0,"allocated":5382144,"data_stored":2854272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.955123"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.378983+0000","last_change":"2023-11-24T08:10:40.378983+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.378983+0000","last_undegraded":"2023-11-24T08:10:40.378983+0000","last_fullsized":"2023-11-24T08:10:40.378983+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.379469+0000","last_change":"2023-11-24T08:10:40.379469+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.379469+0000","last_undegraded":"2023-11-24T08:10:40.379469+0000","last_fullsized":"2023-11-24T08:10:40.379469+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.381829+0000","last_change":"2023-11-24T08:10:40.381829+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.381829+0000","last_undegraded":"2023-11-24T08:10:40.381829+0000","last_fullsized":"2023-11-24T08:10:40.381829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.380043+0000","last_change":"2023-11-24T08:10:40.380043+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.380043+0000","last_undegraded":"2023-11-24T08:10:40.380043+0000","last_fullsized":"2023-11-24T08:10:40.380043+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361522+0000","last_change":"2023-11-24T08:10:39.361522+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361522+0000","last_undegraded":"2023-11-24T08:10:39.361522+0000","last_fullsized":"2023-11-24T08:10:39.361522+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.366172+0000","last_change":"2023-11-24T08:10:39.366172+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.366172+0000","last_undegraded":"2023-11-24T08:10:39.366172+0000","last_fullsized":"2023-11-24T08:10:39.366172+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.378875+0000","last_change":"2023-11-24T08:10:40.378875+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.378875+0000","last_undegraded":"2023-11-24T08:10:40.378875+0000","last_fullsized":"2023-11-24T08:10:40.378875+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372156+0000","last_change":"2023-11-24T08:10:39.372156+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372156+0000","last_undegraded":"2023-11-24T08:10:39.372156+0000","last_fullsized":"2023-11-24T08:10:39.372156+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359062+0000","last_change":"2023-11-24T08:10:39.359062+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359062+0000","last_undegraded":"2023-11-24T08:10:39.359062+0000","last_fullsized":"2023-11-24T08:10:39.359062+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.362909+0000","last_change":"2023-11-24T08:10:39.362909+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.362909+0000","last_undegraded":"2023-11-24T08:10:39.362909+0000","last_fullsized":"2023-11-24T08:10:39.362909+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.365806+0000","last_change":"2023-11-24T08:10:39.365806+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.365806+0000","last_undegraded":"2023-11-24T08:10:39.365806+0000","last_fullsized":"2023-11-24T08:10:39.365806+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.398927+0000","last_change":"2023-11-24T08:10:40.390774+0000","last_active":"2023-11-24T08:10:40.398927+0000","last_peered":"2023-11-24T08:10:40.398927+0000","last_clean":"2023-11-24T08:10:40.398927+0000","last_became_active":"2023-11-24T08:10:40.390495+0000","last_became_peered":"2023-11-24T08:10:40.390495+0000","last_unstale":"2023-11-24T08:10:40.398927+0000","last_undegraded":"2023-11-24T08:10:40.398927+0000","last_fullsized":"2023-11-24T08:10:40.398927+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:21:13.137284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.370230+0000","last_change":"2023-11-24T08:10:39.370230+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.370230+0000","last_undegraded":"2023-11-24T08:10:39.370230+0000","last_fullsized":"2023-11-24T08:10:39.370230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364602+0000","last_change":"2023-11-24T08:10:39.364602+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364602+0000","last_undegraded":"2023-11-24T08:10:39.364602+0000","last_fullsized":"2023-11-24T08:10:39.364602+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.368230+0000","last_change":"2023-11-24T08:10:39.368230+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.368230+0000","last_undegraded":"2023-11-24T08:10:39.368230+0000","last_fullsized":"2023-11-24T08:10:39.368230+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.360521+0000","last_change":"2023-11-24T08:10:39.360521+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.360521+0000","last_undegraded":"2023-11-24T08:10:39.360521+0000","last_fullsized":"2023-11-24T08:10:39.360521+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":3,"reported_epoch":46,"state":"creating+activating","last_fresh":"2023-11-24T08:10:40.362293+0000","last_change":"2023-11-24T08:10:40.362293+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.362293+0000","last_undegraded":"2023-11-24T08:10:40.362293+0000","last_fullsized":"2023-11-24T08:10:40.362293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:42:15.573547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.362284+0000","last_change":"2023-11-24T08:10:39.362284+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.362284+0000","last_undegraded":"2023-11-24T08:10:39.362284+0000","last_fullsized":"2023-11-24T08:10:39.362284+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359021+0000","last_change":"2023-11-24T08:10:39.359021+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359021+0000","last_undegraded":"2023-11-24T08:10:39.359021+0000","last_fullsized":"2023-11-24T08:10:39.359021+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361496+0000","last_change":"2023-11-24T08:10:39.361496+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361496+0000","last_undegraded":"2023-11-24T08:10:39.361496+0000","last_fullsized":"2023-11-24T08:10:39.361496+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.369002+0000","last_change":"2023-11-24T08:10:39.369002+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.369002+0000","last_undegraded":"2023-11-24T08:10:39.369002+0000","last_fullsized":"2023-11-24T08:10:39.369002+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":3,"reported_epoch":46,"state":"creating+activating","last_fresh":"2023-11-24T08:10:40.362269+0000","last_change":"2023-11-24T08:10:40.362269+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.362269+0000","last_undegraded":"2023-11-24T08:10:40.362269+0000","last_fullsized":"2023-11-24T08:10:40.362269+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:04.394116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364770+0000","last_change":"2023-11-24T08:10:39.364770+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364770+0000","last_undegraded":"2023-11-24T08:10:39.364770+0000","last_fullsized":"2023-11-24T08:10:39.364770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.398881+0000","last_change":"2023-11-24T08:10:40.390983+0000","last_active":"2023-11-24T08:10:40.398881+0000","last_peered":"2023-11-24T08:10:40.398881+0000","last_clean":"2023-11-24T08:10:40.398881+0000","last_became_active":"2023-11-24T08:10:40.389721+0000","last_became_peered":"2023-11-24T08:10:40.389721+0000","last_unstale":"2023-11-24T08:10:40.398881+0000","last_undegraded":"2023-11-24T08:10:40.398881+0000","last_fullsized":"2023-11-24T08:10:40.398881+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:01:07.182397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361652+0000","last_change":"2023-11-24T08:10:39.361652+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361652+0000","last_undegraded":"2023-11-24T08:10:39.361652+0000","last_fullsized":"2023-11-24T08:10:39.361652+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361525+0000","last_change":"2023-11-24T08:10:39.361525+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361525+0000","last_undegraded":"2023-11-24T08:10:39.361525+0000","last_fullsized":"2023-11-24T08:10:39.361525+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.399131+0000","last_change":"2023-11-24T08:10:40.393936+0000","last_active":"2023-11-24T08:10:40.399131+0000","last_peered":"2023-11-24T08:10:40.399131+0000","last_clean":"2023-11-24T08:10:40.399131+0000","last_became_active":"2023-11-24T08:10:40.393617+0000","last_became_peered":"2023-11-24T08:10:40.393617+0000","last_unstale":"2023-11-24T08:10:40.399131+0000","last_undegraded":"2023-11-24T08:10:40.399131+0000","last_fullsized":"2023-11-24T08:10:40.399131+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:38:50.199344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.367587+0000","last_change":"2023-11-24T08:10:39.367587+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.367587+0000","last_undegraded":"2023-11-24T08:10:39.367587+0000","last_fullsized":"2023-11-24T08:10:39.367587+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359004+0000","last_change":"2023-11-24T08:10:39.359004+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359004+0000","last_undegraded":"2023-11-24T08:10:39.359004+0000","last_fullsized":"2023-11-24T08:10:39.359004+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.361744+0000","last_change":"2023-11-24T08:10:39.361744+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.361744+0000","last_undegraded":"2023-11-24T08:10:39.361744+0000","last_fullsized":"2023-11-24T08:10:39.361744+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.359056+0000","last_change":"2023-11-24T08:10:39.359056+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.359056+0000","last_undegraded":"2023-11-24T08:10:39.359056+0000","last_fullsized":"2023-11-24T08:10:39.359056+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372125+0000","last_change":"2023-11-24T08:10:39.372125+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372125+0000","last_undegraded":"2023-11-24T08:10:39.372125+0000","last_fullsized":"2023-11-24T08:10:39.372125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.358939+0000","last_change":"2023-11-24T08:10:39.358939+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.358939+0000","last_undegraded":"2023-11-24T08:10:39.358939+0000","last_fullsized":"2023-11-24T08:10:39.358939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.364515+0000","last_change":"2023-11-24T08:10:39.364515+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.364515+0000","last_undegraded":"2023-11-24T08:10:39.364515+0000","last_fullsized":"2023-11-24T08:10:39.364515+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":46,"state":"creating+peering","last_fresh":"2023-11-24T08:10:40.379378+0000","last_change":"2023-11-24T08:10:40.379378+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.379378+0000","last_undegraded":"2023-11-24T08:10:40.379378+0000","last_fullsized":"2023-11-24T08:10:40.379378+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.372024+0000","last_change":"2023-11-24T08:10:39.372024+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.372024+0000","last_undegraded":"2023-11-24T08:10:39.372024+0000","last_fullsized":"2023-11-24T08:10:39.372024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":45,"state":"creating+peering","last_fresh":"2023-11-24T08:10:39.367903+0000","last_change":"2023-11-24T08:10:39.367903+0000","last_active":"2023-11-24T08:10:39.336827+0000","last_peered":"2023-11-24T08:10:39.336827+0000","last_clean":"2023-11-24T08:10:39.336827+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:39.367903+0000","last_undegraded":"2023-11-24T08:10:39.367903+0000","last_fullsized":"2023-11-24T08:10:39.367903+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:10:40.350420+0000","last_change":"2023-11-24T08:10:40.350420+0000","last_active":"2023-11-24T08:10:40.350420+0000","last_peered":"2023-11-24T08:10:40.350420+0000","last_clean":"2023-11-24T08:10:40.350420+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:10:40.350420+0000","last_undegraded":"2023-11-24T08:10:40.350420+0000","last_fullsized":"2023-11-24T08:10:40.350420+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-11-24T08:10:40.399653+0000","last_change":"2023-11-24T08:10:40.396648+0000","last_active":"2023-11-24T08:10:40.399653+0000","last_peered":"2023-11-24T08:10:40.399653+0000","last_clean":"2023-11-24T08:10:40.399653+0000","last_became_active":"2023-11-24T08:10:40.395757+0000","last_became_peered":"2023-11-24T08:10:40.395757+0000","last_unstale":"2023-11-24T08:10:40.399653+0000","last_undegraded":"2023-11-24T08:10:40.399653+0000","last_fullsized":"2023-11-24T08:10:40.399653+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:14:25.355904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":183,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-11-24T08:10:37.580718+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:10:37.580718+0000","last_peered":"2023-11-24T08:10:37.580718+0000","last_clean":"2023-11-24T08:10:37.580718+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:10:37.580718+0000","last_undegraded":"2023-11-24T08:10:37.580718+0000","last_fullsized":"2023-11-24T08:10:37.580718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":1},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542172,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1118208,"data_stored":799560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":1.5329999999999999,"15min":1.5329999999999999},"last":0.96999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.5529999999999999,"5min":1.5529999999999999,"15min":1.5529999999999999},"last":1.341},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.3479999999999999,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.109}]},{"osd":2,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":1.8740000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.755,"5min":0.755,"15min":0.755},"max":{"1min":2.1720000000000002,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2230000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.232,"15min":1.232},"min":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"max":{"1min":1.5640000000000001,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.6579999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.9119999999999999,"5min":1.9119999999999999,"15min":1.9119999999999999},"last":1.6990000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.276},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.738,"5min":2.738,"15min":2.738},"last":1.9390000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.4390000000000001,"15min":1.4390000000000001},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3170000000000002,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.3839999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.472,"5min":1.472,"15min":1.472},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":1.7509999999999999},{"interface":"front","average":{"1min":1.4730000000000001,"5min":1.4730000000000001,"15min":1.4730000000000001},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.802}]}]},{"osd":6,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.091},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2470000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"min":{"1min":0.749,"5min":0.749,"15min":0.749},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":1.474}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.778,"5min":1.778,"15min":1.778},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7709999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.365,"5min":1.365,"15min":1.365},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.371},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":2.4510000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.7789999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":2.343,"5min":2.343,"15min":2.343},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.603,"5min":1.603,"15min":1.603},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.4330000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.75,"15min":1.75},"min":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":3.2829999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.5289999999999999},{"interface":"front","average":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":3.5950000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.5649999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.8080000000000001,"5min":1.8080000000000001,"15min":1.8080000000000001},"min":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"max":{"1min":2.5830000000000002,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.631},{"interface":"front","average":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"min":{"1min":1.246,"5min":1.246,"15min":1.246},"max":{"1min":2.657,"5min":2.657,"15min":2.657},"last":1.6639999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542170,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.6080000000000001,"5min":1.6080000000000001,"15min":1.6080000000000001},"last":1.04},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.71,"5min":1.71,"15min":1.71},"last":1.2350000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.532,"5min":1.532,"15min":1.532},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.353}]},{"osd":2,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":2.085,"5min":2.085,"15min":2.085},"last":1.415},{"interface":"front","average":{"1min":1.377,"5min":1.377,"15min":1.377},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.84499999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":2.0379999999999998,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.542,"5min":1.542,"15min":1.542},"min":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.286}]},{"osd":4,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4650000000000001,"15min":1.4650000000000001},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":1.524},{"interface":"front","average":{"1min":1.732,"5min":1.732,"15min":1.732},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":4.0279999999999996,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.4890000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.621,"15min":1.621},"min":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.581},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5369999999999999,"15min":1.5369999999999999},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.512,"5min":2.512,"15min":2.512},"last":1.645}]},{"osd":7,"last update":"Fri Nov 24 08:09:41 2023","interfaces":[{"interface":"back","average":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7330000000000001},"min":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":2.7400000000000002,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.621},{"interface":"front","average":{"1min":1.752,"5min":1.752,"15min":1.752},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.9689999999999999,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.4590000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31692,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340148,"statfs":{"total":96636764160,"available":96604311552,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6950000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.52},{"interface":"front","average":{"1min":1.091,"5min":1.091,"15min":1.091},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":0.84399999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.155,"15min":1.155},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.859,"5min":1.859,"15min":1.859},"last":1.335},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.962,"5min":1.962,"15min":1.962},"last":1.133}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.571},{"interface":"front","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.995,"5min":0.995,"15min":0.995},"max":{"1min":2.746,"5min":2.746,"15min":2.746},"last":1.409}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.329,"15min":1.329},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.2809999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.4209999999999998,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.073}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":2.5030000000000001,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.5680000000000001,"5min":1.5680000000000001,"15min":1.5680000000000001},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.641,"5min":2.641,"15min":2.641},"last":1.6279999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.512,"5min":1.512,"15min":1.512},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.6599999999999999},{"interface":"front","average":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.6869999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.4710000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":1.333}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.96799999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.198}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.4690000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.3620000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.53}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.3},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.157}]}]},{"osd":2,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.369,"5min":1.369,"15min":1.369},"last":1.208}]},{"osd":1,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.6479999999999999,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.143},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.58899999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.111}]},{"osd":4,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.222,"15min":1.222},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.581,"5min":1.581,"15min":1.581},"last":1.4370000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.466,"5min":1.466,"15min":1.466},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":4.0129999999999999,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3979999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":2.7509999999999999,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.343,"15min":1.343},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"max":{"1min":1.7370000000000001,"5min":1.7370000000000001,"15min":1.7370000000000001},"last":1.472}]},{"osd":7,"last update":"Fri Nov 24 08:09:39 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":5.1379999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.452,"5min":1.452,"15min":1.452},"min":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"max":{"1min":2.4009999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.573}]}]},{"osd":1,"up_from":14,"seq":60129542171,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27532,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344308,"statfs":{"total":96636764160,"available":96608571392,"internally_reserved":0,"allocated":524288,"data_stored":209192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.038},{"interface":"front","average":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0659999999999998,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.621,"5min":1.621,"15min":1.621},"last":0.97499999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":2.1240000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.3380000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.165,"15min":1.165},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.9199999999999999,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.262},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0920000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.222}]},{"osd":5,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.321,"15min":1.321},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.5269999999999999},{"interface":"front","average":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.9710000000000001,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.165}]},{"osd":6,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.363,"15min":1.363},"min":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"min":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":2.105,"5min":2.105,"15min":2.105},"last":1.3859999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:09:42 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.1400000000000001,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.369,"15min":1.369},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":1.7709999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542171,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28112,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343728,"statfs":{"total":96636764160,"available":96607977472,"internally_reserved":0,"allocated":1118208,"data_stored":799560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.403,"5min":1.403,"15min":1.403},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.7070000000000001,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.82999999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.986,"5min":1.986,"15min":1.986},"last":1.113}]},{"osd":3,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":2.1389999999999998,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.151}]},{"osd":4,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.292,"5min":1.292,"15min":1.292},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.51}]},{"osd":5,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.23,"15min":1.23},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":2.0390000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.1459999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":0.97399999999999998}]},{"osd":6,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.2749999999999999,"15min":1.2749999999999999},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.0190000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.288,"5min":1.288,"15min":1.288},"min":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":0.79900000000000004}]},{"osd":7,"last update":"Fri Nov 24 08:09:40 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":2.4260000000000002,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.393},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2789999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3f is not active+clean 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3e is not active+clean 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:40.378983+0000', 'last_change': '2023-11-24T08:10:40.378983+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.378983+0000', 'last_undegraded': '2023-11-24T08:10:40.378983+0000', 'last_fullsized': '2023-11-24T08:10:40.378983+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3d is not active+clean 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3c is not active+clean 2023-11-24T08:10:46.843 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3b is not active+clean 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:40.379469+0000', 'last_change': '2023-11-24T08:10:40.379469+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.379469+0000', 'last_undegraded': '2023-11-24T08:10:40.379469+0000', 'last_fullsized': '2023-11-24T08:10:40.379469+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3a is not active+clean 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.39 is not active+clean 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.38 is not active+clean 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.37 is not active+clean 2023-11-24T08:10:46.844 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.36 is not active+clean 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.35 is not active+clean 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.34 is not active+clean 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:40.381829+0000', 'last_change': '2023-11-24T08:10:40.381829+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.381829+0000', 'last_undegraded': '2023-11-24T08:10:40.381829+0000', 'last_fullsized': '2023-11-24T08:10:40.381829+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.33 is not active+clean 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.32 is not active+clean 2023-11-24T08:10:46.845 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.31 is not active+clean 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.30 is not active+clean 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2f is not active+clean 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2e is not active+clean 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2d is not active+clean 2023-11-24T08:10:46.846 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2c is not active+clean 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2b is not active+clean 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2a is not active+clean 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.29 is not active+clean 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.28 is not active+clean 2023-11-24T08:10:46.847 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.27 is not active+clean 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.26 is not active+clean 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.25 is not active+clean 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.24 is not active+clean 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:40.380043+0000', 'last_change': '2023-11-24T08:10:40.380043+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.380043+0000', 'last_undegraded': '2023-11-24T08:10:40.380043+0000', 'last_fullsized': '2023-11-24T08:10:40.380043+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.23 is not active+clean 2023-11-24T08:10:46.848 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.22 is not active+clean 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.21 is not active+clean 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.20 is not active+clean 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1f is not active+clean 2023-11-24T08:10:46.849 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.361522+0000', 'last_change': '2023-11-24T08:10:39.361522+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.361522+0000', 'last_undegraded': '2023-11-24T08:10:39.361522+0000', 'last_fullsized': '2023-11-24T08:10:39.361522+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1e is not active+clean 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.366172+0000', 'last_change': '2023-11-24T08:10:39.366172+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.366172+0000', 'last_undegraded': '2023-11-24T08:10:39.366172+0000', 'last_fullsized': '2023-11-24T08:10:39.366172+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:40.378875+0000', 'last_change': '2023-11-24T08:10:40.378875+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.378875+0000', 'last_undegraded': '2023-11-24T08:10:40.378875+0000', 'last_fullsized': '2023-11-24T08:10:40.378875+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1d is not active+clean 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.372156+0000', 'last_change': '2023-11-24T08:10:39.372156+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.372156+0000', 'last_undegraded': '2023-11-24T08:10:39.372156+0000', 'last_fullsized': '2023-11-24T08:10:39.372156+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-11-24T08:10:46.850 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1c is not active+clean 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.359062+0000', 'last_change': '2023-11-24T08:10:39.359062+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.359062+0000', 'last_undegraded': '2023-11-24T08:10:39.359062+0000', 'last_fullsized': '2023-11-24T08:10:39.359062+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1b is not active+clean 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.362909+0000', 'last_change': '2023-11-24T08:10:39.362909+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.362909+0000', 'last_undegraded': '2023-11-24T08:10:39.362909+0000', 'last_fullsized': '2023-11-24T08:10:39.362909+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1a is not active+clean 2023-11-24T08:10:46.851 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.365806+0000', 'last_change': '2023-11-24T08:10:39.365806+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.365806+0000', 'last_undegraded': '2023-11-24T08:10:39.365806+0000', 'last_fullsized': '2023-11-24T08:10:39.365806+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.18 is not active+clean 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.370230+0000', 'last_change': '2023-11-24T08:10:39.370230+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.370230+0000', 'last_undegraded': '2023-11-24T08:10:39.370230+0000', 'last_fullsized': '2023-11-24T08:10:39.370230+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.17 is not active+clean 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.364602+0000', 'last_change': '2023-11-24T08:10:39.364602+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.364602+0000', 'last_undegraded': '2023-11-24T08:10:39.364602+0000', 'last_fullsized': '2023-11-24T08:10:39.364602+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:10:46.852 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.16 is not active+clean 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.368230+0000', 'last_change': '2023-11-24T08:10:39.368230+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.368230+0000', 'last_undegraded': '2023-11-24T08:10:39.368230+0000', 'last_fullsized': '2023-11-24T08:10:39.368230+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.15 is not active+clean 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.360521+0000', 'last_change': '2023-11-24T08:10:39.360521+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.360521+0000', 'last_undegraded': '2023-11-24T08:10:39.360521+0000', 'last_fullsized': '2023-11-24T08:10:39.360521+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.14 is not active+clean 2023-11-24T08:10:46.853 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.14', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 46, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:10:40.362293+0000', 'last_change': '2023-11-24T08:10:40.362293+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.362293+0000', 'last_undegraded': '2023-11-24T08:10:40.362293+0000', 'last_fullsized': '2023-11-24T08:10:40.362293+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T19:42:15.573547+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.13 is not active+clean 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.362284+0000', 'last_change': '2023-11-24T08:10:39.362284+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.362284+0000', 'last_undegraded': '2023-11-24T08:10:39.362284+0000', 'last_fullsized': '2023-11-24T08:10:39.362284+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.12 is not active+clean 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.359021+0000', 'last_change': '2023-11-24T08:10:39.359021+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.359021+0000', 'last_undegraded': '2023-11-24T08:10:39.359021+0000', 'last_fullsized': '2023-11-24T08:10:39.359021+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-11-24T08:10:46.854 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.11 is not active+clean 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.361496+0000', 'last_change': '2023-11-24T08:10:39.361496+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.361496+0000', 'last_undegraded': '2023-11-24T08:10:39.361496+0000', 'last_fullsized': '2023-11-24T08:10:39.361496+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.10 is not active+clean 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.369002+0000', 'last_change': '2023-11-24T08:10:39.369002+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.369002+0000', 'last_undegraded': '2023-11-24T08:10:39.369002+0000', 'last_fullsized': '2023-11-24T08:10:39.369002+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.f is not active+clean 2023-11-24T08:10:46.855 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 46, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:10:40.362269+0000', 'last_change': '2023-11-24T08:10:40.362269+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.362269+0000', 'last_undegraded': '2023-11-24T08:10:40.362269+0000', 'last_fullsized': '2023-11-24T08:10:40.362269+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T18:03:04.394116+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.e is not active+clean 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.364770+0000', 'last_change': '2023-11-24T08:10:39.364770+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.364770+0000', 'last_undegraded': '2023-11-24T08:10:39.364770+0000', 'last_fullsized': '2023-11-24T08:10:39.364770+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-11-24T08:10:46.856 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.c is not active+clean 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.361652+0000', 'last_change': '2023-11-24T08:10:39.361652+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.361652+0000', 'last_undegraded': '2023-11-24T08:10:39.361652+0000', 'last_fullsized': '2023-11-24T08:10:39.361652+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.b is not active+clean 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.361525+0000', 'last_change': '2023-11-24T08:10:39.361525+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.361525+0000', 'last_undegraded': '2023-11-24T08:10:39.361525+0000', 'last_fullsized': '2023-11-24T08:10:39.361525+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-11-24T08:10:46.857 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.9 is not active+clean 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.367587+0000', 'last_change': '2023-11-24T08:10:39.367587+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.367587+0000', 'last_undegraded': '2023-11-24T08:10:39.367587+0000', 'last_fullsized': '2023-11-24T08:10:39.367587+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.8 is not active+clean 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.359004+0000', 'last_change': '2023-11-24T08:10:39.359004+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.359004+0000', 'last_undegraded': '2023-11-24T08:10:39.359004+0000', 'last_fullsized': '2023-11-24T08:10:39.359004+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.6 is not active+clean 2023-11-24T08:10:46.858 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.361744+0000', 'last_change': '2023-11-24T08:10:39.361744+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.361744+0000', 'last_undegraded': '2023-11-24T08:10:39.361744+0000', 'last_fullsized': '2023-11-24T08:10:39.361744+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.5 is not active+clean 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.359056+0000', 'last_change': '2023-11-24T08:10:39.359056+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.359056+0000', 'last_undegraded': '2023-11-24T08:10:39.359056+0000', 'last_fullsized': '2023-11-24T08:10:39.359056+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.4 is not active+clean 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.372125+0000', 'last_change': '2023-11-24T08:10:39.372125+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.372125+0000', 'last_undegraded': '2023-11-24T08:10:39.372125+0000', 'last_fullsized': '2023-11-24T08:10:39.372125+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-11-24T08:10:46.859 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.3 is not active+clean 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.358939+0000', 'last_change': '2023-11-24T08:10:39.358939+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.358939+0000', 'last_undegraded': '2023-11-24T08:10:39.358939+0000', 'last_fullsized': '2023-11-24T08:10:39.358939+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.2 is not active+clean 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.364515+0000', 'last_change': '2023-11-24T08:10:39.364515+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.364515+0000', 'last_undegraded': '2023-11-24T08:10:39.364515+0000', 'last_fullsized': '2023-11-24T08:10:39.364515+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 46, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:40.379378+0000', 'last_change': '2023-11-24T08:10:40.379378+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.379378+0000', 'last_undegraded': '2023-11-24T08:10:40.379378+0000', 'last_fullsized': '2023-11-24T08:10:40.379378+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 46, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:10:46.860 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.1 is not active+clean 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.372024+0000', 'last_change': '2023-11-24T08:10:39.372024+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.372024+0000', 'last_undegraded': '2023-11-24T08:10:39.372024+0000', 'last_fullsized': '2023-11-24T08:10:39.372024+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 6.0 is not active+clean 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '6.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 45, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:10:39.367903+0000', 'last_change': '2023-11-24T08:10:39.367903+0000', 'last_active': '2023-11-24T08:10:39.336827+0000', 'last_peered': '2023-11-24T08:10:39.336827+0000', 'last_clean': '2023-11-24T08:10:39.336827+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:39.367903+0000', 'last_undegraded': '2023-11-24T08:10:39.367903+0000', 'last_fullsized': '2023-11-24T08:10:39.367903+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 45, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:39.336827+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-11-24T08:10:46.861 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-24T08:10:40.350420+0000', 'last_change': '2023-11-24T08:10:40.350420+0000', 'last_active': '2023-11-24T08:10:40.350420+0000', 'last_peered': '2023-11-24T08:10:40.350420+0000', 'last_clean': '2023-11-24T08:10:40.350420+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:10:40.350420+0000', 'last_undegraded': '2023-11-24T08:10:40.350420+0000', 'last_fullsized': '2023-11-24T08:10:40.350420+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'last_clean_scrub_stamp': '2023-11-24T08:10:40.350420+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-24T08:10:46.862 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T08:10:47.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.095+0000 7f8ebc8c9640 1 Processor -- start 2023-11-24T08:10:47.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.096+0000 7f8ebc8c9640 1 -- start start 2023-11-24T08:10:47.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8ebc8c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8eb80af870 0x7f8eb80cd2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8ebc8c9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80cc210 con 0x7f8eb80cbc70 2023-11-24T08:10:47.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8ebc8c9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb80cc4f0 con 0x7f8eb80af490 2023-11-24T08:10:47.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8ebc8c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80cce90 con 0x7f8eb80af870 2023-11-24T08:10:47.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8eb6d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8eb80cbc70 0x7f8eb815f030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47298/0 (socket says 172.21.15.106:47298) 2023-11-24T08:10:47.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8eb6d9d640 1 -- 172.21.15.106:0/3096403482 learned_addr learned my addr 172.21.15.106:0/3096403482 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:47.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.097+0000 7f8eb5d9b640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8eb80af870 0x7f8eb80cd2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366732078 0 0) 0x7f8eb80cc210 con 0x7f8eb80cbc70 2023-11-24T08:10:47.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ea4003520 con 0x7f8eb80cbc70 2023-11-24T08:10:47.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2261053832 0 0) 0x7f8ea4003520 con 0x7f8eb80cbc70 2023-11-24T08:10:47.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8eb80af870 msgr2=0x7f8eb80cd2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8eb80af870 0x7f8eb80cd2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 >> v1:172.21.15.156:6789/0 conn(0x7f8eb80af490 legacy=0x7f8eb806ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80cc7d0 con 0x7f8eb80cbc70 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f8eb0001670 con 0x7f8eb80cbc70 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 >> v1:172.21.15.106:6789/0 conn(0x7f8eb80cbc70 legacy=0x7f8eb815f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40047c0 0x7f8ea4004c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea4005140 0x7f8ea400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80cce90 con 0x7f8ea40039a0 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea400bf70 con 0x7f8ea4005140 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea400c100 con 0x7f8ea40047c0 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.098+0000 7f8eb659c640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb5d9b640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40047c0 0x7f8ea4004c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb6d9d640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea4005140 0x7f8ea400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb659c640 1 -- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40047c0 msgr2=0x7f8ea4004c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb659c640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40047c0 0x7f8ea4004c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb659c640 1 -- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea4005140 msgr2=0x7f8ea400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb659c640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea4005140 0x7f8ea400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb659c640 1 -- 172.21.15.106:0/3096403482 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ea4003520 con 0x7f8ea40039a0 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.099+0000 7f8eb5d9b640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40047c0 0x7f8ea4004c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:47.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.100+0000 7f8eb659c640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001e80 tx=0x7f8ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7938d1471dc051a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:47.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.100+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8003b70 con 0x7f8ea40039a0 2023-11-24T08:10:47.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.100+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea8003cd0 con 0x7f8ea40039a0 2023-11-24T08:10:47.104 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.100+0000 7f8eb559a640 1 -- 172.21.15.106:0/3096403482 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8005310 con 0x7f8ea40039a0 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/3096403482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ea40039a0 msgr2=0x7f8ea4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/3096403482 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ea40039a0 0x7f8ea4003dc0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001e80 tx=0x7f8ea40057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8eb6d9d640 1 -- 172.21.15.106:0/3096403482 reap_dead start 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/3096403482 shutdown_connections 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/3096403482 >> 172.21.15.106:0/3096403482 conn(0x7f8eb80999d0 msgr2=0x7f8eb80b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/3096403482 shutdown_connections 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.101+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/3096403482 wait complete. 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.103+0000 7f8ebc8c9640 1 Processor -- start 2023-11-24T08:10:47.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.103+0000 7f8ebc8c9640 1 -- start start 2023-11-24T08:10:47.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8ebc8c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40039a0 0x7f8eb80c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8ebc8c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 0x7f8eb8158730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8ebc8c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb8158c70 0x7f8eb81f6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8ebc8c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb8159100 con 0x7f8eb8153680 2023-11-24T08:10:47.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8ebc8c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb81f73c0 con 0x7f8eb8158c70 2023-11-24T08:10:47.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8ebc8c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb80c81f0 con 0x7f8ea40039a0 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8eb5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 0x7f8eb8158730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8eb5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 0x7f8eb8158730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42542/0 (socket says 172.21.15.106:42542) 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.104+0000 7f8eb5d9b640 1 -- 172.21.15.106:0/324747051 learned_addr learned my addr 172.21.15.106:0/324747051 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb659c640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40039a0 0x7f8eb80c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb5d9b640 1 -- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40039a0 msgr2=0x7f8eb80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb5d9b640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40039a0 0x7f8eb80c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb6d9d640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb8158c70 0x7f8eb81f6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb5d9b640 1 -- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb8158c70 msgr2=0x7f8eb81f6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb5d9b640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb8158c70 0x7f8eb81f6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb5d9b640 1 -- 172.21.15.106:0/324747051 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb80c84d0 con 0x7f8eb8153680 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb659c640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40039a0 0x7f8eb80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb6d9d640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb8158c70 0x7f8eb81f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:47.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eb5d9b640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 0x7f8eb8158730 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e7e0 tx=0x7f8ea000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.105+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000c6d0 con 0x7f8eb8153680 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.106+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea000c830 con 0x7f8eb8153680 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.106+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea000cba0 con 0x7f8eb8153680 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.106+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb80c87b0 con 0x7f8eb8153680 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.106+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb8153b00 con 0x7f8eb8153680 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.107+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb8067470 con 0x7f8eb8153680 2023-11-24T08:10:47.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.108+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f8ea000cd00 con 0x7f8eb8153680 2023-11-24T08:10:47.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.109+0000 7f8eaeffd640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e9806aed0 0x7f8e9806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.109+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ea0089420 con 0x7f8eb8153680 2023-11-24T08:10:47.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.109+0000 7f8eb659c640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e9806aed0 0x7f8e9806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.110+0000 7f8eb659c640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e9806aed0 0x7f8e9806d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034000 tx=0x7f8ea8039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:47.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.113+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ea0054460 con 0x7f8eb8153680 2023-11-24T08:10:47.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.377+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8eb8071940 con 0x7f8eb8153680 2023-11-24T08:10:47.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.378+0000 7f8eaeffd640 1 -- 172.21.15.106:0/324747051 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f8ea0057a80 con 0x7f8eb8153680 2023-11-24T08:10:47.380 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:47.380 INFO:teuthology.orchestra.run.smithi106.stdout:{"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":139,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1700813319,"num_in_osds":8,"osd_in_since":1700813311,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":58},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":5},{"state_name":"creating+activating","count":2}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":243773440,"bytes_avail":772850339840,"bytes_total":773094113280,"unknown_pgs_ratio":0.59793812036514282,"inactive_pgs_ratio":0.3505154550075531},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5562}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-24T08:10:21.301176+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T08:10:47.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.381+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e9806aed0 msgr2=0x7f8e9806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.381+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e9806aed0 0x7f8e9806d390 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8034000 tx=0x7f8ea8039040 comp rx=0 tx=0).stop 2023-11-24T08:10:47.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.381+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 msgr2=0x7f8eb8158730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.381+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 0x7f8eb8158730 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000e7e0 tx=0x7f8ea000eca0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 shutdown_connections 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8eb8158c70 0x7f8eb81f6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8eb8153680 0x7f8eb8158730 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e9806aed0 0x7f8e9806d390 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 --2- 172.21.15.106:0/324747051 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ea40039a0 0x7f8eb80c8fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 >> 172.21.15.106:0/324747051 conn(0x7f8eb80999d0 msgr2=0x7f8eb80cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 shutdown_connections 2023-11-24T08:10:47.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.382+0000 7f8ebc8c9640 1 -- 172.21.15.106:0/324747051 wait complete. 2023-11-24T08:10:47.400 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T08:10:47.400 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:47.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.609+0000 7fe0cffff640 1 Processor -- start 2023-11-24T08:10:47.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.609+0000 7fe0cffff640 1 -- start start 2023-11-24T08:10:47.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe0d015d4e0 0x7fe0d00d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 0x7fe0d00cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cffff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0d00ce990 con 0x7fe0d00d0070 2023-11-24T08:10:47.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cffff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0d00d4110 con 0x7fe0d015d4e0 2023-11-24T08:10:47.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cffff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00d04e0 con 0x7fe0d00d38e0 2023-11-24T08:10:47.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cf7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe0d00d0070 0x7fe0d00d0da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47310/0 (socket says 172.21.15.106:47310) 2023-11-24T08:10:47.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0cf7fe640 1 -- 172.21.15.106:0/3942767407 learned_addr learned my addr 172.21.15.106:0/3942767407 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:47.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0ceffd640 1 --2- 172.21.15.106:0/3942767407 >> v2:172.21.15.156:3300/0 conn(0x7fe0d015d4e0 0x7fe0d00d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.611+0000 7fe0ce7fc640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 0x7fe0d00cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4274553372 0 0) 0x7fe0d00ce990 con 0x7fe0d00d0070 2023-11-24T08:10:47.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b8003520 con 0x7fe0d00d0070 2023-11-24T08:10:47.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4231799829 0 0) 0x7fe0b8003520 con 0x7fe0d00d0070 2023-11-24T08:10:47.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 msgr2=0x7fe0d00cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 0x7fe0d00cfb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 >> v2:172.21.15.156:3300/0 conn(0x7fe0d015d4e0 msgr2=0x7fe0d00d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 --2- 172.21.15.106:0/3942767407 >> v2:172.21.15.156:3300/0 conn(0x7fe0d015d4e0 0x7fe0d00d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00cec70 con 0x7fe0d00d0070 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fe0c8001a10 con 0x7fe0d00d0070 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 >> v1:172.21.15.106:6789/0 conn(0x7fe0d00d0070 legacy=0x7fe0d00d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 0x7fe0b80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0b8003f70 0x7fe0b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00d4110 con 0x7fe0b8004d00 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00d04e0 con 0x7fe0b8003f70 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b800b9b0 con 0x7fe0d00d38e0 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.612+0000 7fe0cf7fe640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0ceffd640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 0x7fe0b80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0ce7fc640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0b8003f70 0x7fe0b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0cf7fe640 1 -- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 msgr2=0x7fe0b80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0cf7fe640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00d38e0 0x7fe0b80039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0cf7fe640 1 -- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0b8003f70 msgr2=0x7fe0b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:47.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0cf7fe640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0b8003f70 0x7fe0b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0cf7fe640 1 -- 172.21.15.106:0/3942767407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b8003520 con 0x7fe0b8004d00 2023-11-24T08:10:47.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.613+0000 7fe0ce7fc640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0b8003f70 0x7fe0b80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:47.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.614+0000 7fe0cf7fe640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0b8005180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800a740 tx=0x7fe0b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd18725fcb9b50f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:47.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.614+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8034630 con 0x7fe0b8004d00 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.614+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c802f070 con 0x7fe0b8004d00 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.614+0000 7fe0cdffb640 1 -- 172.21.15.106:0/3942767407 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c80349d0 con 0x7fe0b8004d00 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cffff640 1 -- 172.21.15.106:0/3942767407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 msgr2=0x7fe0b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cffff640 1 --2- 172.21.15.106:0/3942767407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0b8005180 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800a740 tx=0x7fe0b80057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cf7fe640 1 -- 172.21.15.106:0/3942767407 reap_dead start 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cffff640 1 -- 172.21.15.106:0/3942767407 shutdown_connections 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cffff640 1 -- 172.21.15.106:0/3942767407 >> 172.21.15.106:0/3942767407 conn(0x7fe0d009c510 msgr2=0x7fe0d00d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cffff640 1 -- 172.21.15.106:0/3942767407 shutdown_connections 2023-11-24T08:10:47.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.615+0000 7fe0cffff640 1 -- 172.21.15.106:0/3942767407 wait complete. 2023-11-24T08:10:47.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.617+0000 7fe0cffff640 1 Processor -- start 2023-11-24T08:10:47.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.617+0000 7fe0cffff640 1 -- start start 2023-11-24T08:10:47.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0d0151fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0d015fec0 0x7fe0d0160320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0cffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00c6ec0 0x7fe0d00c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0cffff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d015dec0 con 0x7fe0b8004d00 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0cffff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d0160860 con 0x7fe0d015fec0 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0cffff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0d00c81d0 con 0x7fe0d00c6ec0 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0d0151fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0d0151fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42550/0 (socket says 172.21.15.106:42550) 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 -- 172.21.15.106:0/4045621723 learned_addr learned my addr 172.21.15.106:0/4045621723 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 -- 172.21.15.106:0/4045621723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00c6ec0 msgr2=0x7fe0d00c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00c6ec0 0x7fe0d00c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 -- 172.21.15.106:0/4045621723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0d015fec0 msgr2=0x7fe0d0160320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0d015fec0 0x7fe0d0160320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.618+0000 7fe0ceffd640 1 -- 172.21.15.106:0/4045621723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0d00c7c10 con 0x7fe0b8004d00 2023-11-24T08:10:47.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.619+0000 7fe0ceffd640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0d0151fc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0010910 tx=0x7fe0c0010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:47.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.619+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0009690 con 0x7fe0b8004d00 2023-11-24T08:10:47.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.619+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0c000eba0 con 0x7fe0b8004d00 2023-11-24T08:10:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.619+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0c001c400 con 0x7fe0b8004d00 2023-11-24T08:10:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.619+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0d00c7ef0 con 0x7fe0b8004d00 2023-11-24T08:10:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.619+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0d0152500 con 0x7fe0b8004d00 2023-11-24T08:10:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.620+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0d004e840 con 0x7fe0b8004d00 2023-11-24T08:10:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.621+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fe0c001c560 con 0x7fe0b8004d00 2023-11-24T08:10:47.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.623+0000 7fe0b77fe640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe0bc06af00 0x7fe0bc06d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:47.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.623+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe0c0089bd0 con 0x7fe0b8004d00 2023-11-24T08:10:47.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.623+0000 7fe0ce7fc640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe0bc06af00 0x7fe0bc06d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:47.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.623+0000 7fe0ce7fc640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe0bc06af00 0x7fe0bc06d3c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0d015cad0 tx=0x7fe0c4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:47.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.627+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe0c0054d10 con 0x7fe0b8004d00 2023-11-24T08:10:47.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.813+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe0d00cd4e0 con 0x7fe0bc06af00 2023-11-24T08:10:47.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.835+0000 7fe0b77fe640 1 -- 172.21.15.106:0/4045621723 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287264 (secure 0 0 0) 0x7fe0d00cd4e0 con 0x7fe0bc06af00 2023-11-24T08:10:47.841 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:47.842 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe0bc06af00 msgr2=0x7fe0bc06d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe0bc06af00 0x7fe0bc06d3c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fe0d015cad0 tx=0x7fe0c4008040 comp rx=0 tx=0).stop 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 msgr2=0x7fe0d0151fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0d0151fc0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0010910 tx=0x7fe0c0010dd0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 shutdown_connections 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe0bc06af00 0x7fe0bc06d3c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe0d00c6ec0 0x7fe0d00c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0d015fec0 0x7fe0d0160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 --2- 172.21.15.106:0/4045621723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe0b8004d00 0x7fe0d0151fc0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 >> 172.21.15.106:0/4045621723 conn(0x7fe0d009c510 msgr2=0x7fe0d00b5540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:47.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 shutdown_connections 2023-11-24T08:10:47.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:47.842+0000 7fe0cffff640 1 -- 172.21.15.106:0/4045621723 wait complete. 2023-11-24T08:10:47.864 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2023-11-24T08:10:47.308620+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238412,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736308,"statfs":{"total":773094113280,"available":772849979392,"internally_reserved":0,"allocated":5742592,"data_stored":3013828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.949356"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.413242+0000","last_change":"2023-11-24T08:10:41.423631+0000","last_active":"2023-11-24T08:10:42.413242+0000","last_peered":"2023-11-24T08:10:42.413242+0000","last_clean":"2023-11-24T08:10:42.413242+0000","last_became_active":"2023-11-24T08:10:41.423175+0000","last_became_peered":"2023-11-24T08:10:41.423175+0000","last_unstale":"2023-11-24T08:10:42.413242+0000","last_undegraded":"2023-11-24T08:10:42.413242+0000","last_fullsized":"2023-11-24T08:10:42.413242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:55.816463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579726+0000","last_change":"2023-11-24T08:10:41.422931+0000","last_active":"2023-11-24T08:10:43.579726+0000","last_peered":"2023-11-24T08:10:43.579726+0000","last_clean":"2023-11-24T08:10:43.579726+0000","last_became_active":"2023-11-24T08:10:41.421975+0000","last_became_peered":"2023-11-24T08:10:41.421975+0000","last_unstale":"2023-11-24T08:10:43.579726+0000","last_undegraded":"2023-11-24T08:10:43.579726+0000","last_fullsized":"2023-11-24T08:10:43.579726+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:40.576676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641332+0000","last_change":"2023-11-24T08:10:41.428763+0000","last_active":"2023-11-24T08:10:42.641332+0000","last_peered":"2023-11-24T08:10:42.641332+0000","last_clean":"2023-11-24T08:10:42.641332+0000","last_became_active":"2023-11-24T08:10:41.428385+0000","last_became_peered":"2023-11-24T08:10:41.428385+0000","last_unstale":"2023-11-24T08:10:42.641332+0000","last_undegraded":"2023-11-24T08:10:42.641332+0000","last_fullsized":"2023-11-24T08:10:42.641332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:36:31.790671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460447+0000","last_change":"2023-11-24T08:10:41.423533+0000","last_active":"2023-11-24T08:10:43.460447+0000","last_peered":"2023-11-24T08:10:43.460447+0000","last_clean":"2023-11-24T08:10:43.460447+0000","last_became_active":"2023-11-24T08:10:41.421574+0000","last_became_peered":"2023-11-24T08:10:41.421574+0000","last_unstale":"2023-11-24T08:10:43.460447+0000","last_undegraded":"2023-11-24T08:10:43.460447+0000","last_fullsized":"2023-11-24T08:10:43.460447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:34.014751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579011+0000","last_change":"2023-11-24T08:10:41.413638+0000","last_active":"2023-11-24T08:10:43.579011+0000","last_peered":"2023-11-24T08:10:43.579011+0000","last_clean":"2023-11-24T08:10:43.579011+0000","last_became_active":"2023-11-24T08:10:41.412839+0000","last_became_peered":"2023-11-24T08:10:41.412839+0000","last_unstale":"2023-11-24T08:10:43.579011+0000","last_undegraded":"2023-11-24T08:10:43.579011+0000","last_fullsized":"2023-11-24T08:10:43.579011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:41.556777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408829+0000","last_change":"2023-11-24T08:10:41.420154+0000","last_active":"2023-11-24T08:10:42.408829+0000","last_peered":"2023-11-24T08:10:42.408829+0000","last_clean":"2023-11-24T08:10:42.408829+0000","last_became_active":"2023-11-24T08:10:41.419937+0000","last_became_peered":"2023-11-24T08:10:41.419937+0000","last_unstale":"2023-11-24T08:10:42.408829+0000","last_undegraded":"2023-11-24T08:10:42.408829+0000","last_fullsized":"2023-11-24T08:10:42.408829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:27.391759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412418+0000","last_change":"2023-11-24T08:10:41.426274+0000","last_active":"2023-11-24T08:10:42.412418+0000","last_peered":"2023-11-24T08:10:42.412418+0000","last_clean":"2023-11-24T08:10:42.412418+0000","last_became_active":"2023-11-24T08:10:41.426014+0000","last_became_peered":"2023-11-24T08:10:41.426014+0000","last_unstale":"2023-11-24T08:10:42.412418+0000","last_undegraded":"2023-11-24T08:10:42.412418+0000","last_fullsized":"2023-11-24T08:10:42.412418+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:34.014751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.459931+0000","last_change":"2023-11-24T08:10:41.426539+0000","last_active":"2023-11-24T08:10:43.459931+0000","last_peered":"2023-11-24T08:10:43.459931+0000","last_clean":"2023-11-24T08:10:43.459931+0000","last_became_active":"2023-11-24T08:10:41.425609+0000","last_became_peered":"2023-11-24T08:10:41.425609+0000","last_unstale":"2023-11-24T08:10:43.459931+0000","last_undegraded":"2023-11-24T08:10:43.459931+0000","last_fullsized":"2023-11-24T08:10:43.459931+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:43.691758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460780+0000","last_change":"2023-11-24T08:10:41.412301+0000","last_active":"2023-11-24T08:10:43.460780+0000","last_peered":"2023-11-24T08:10:43.460780+0000","last_clean":"2023-11-24T08:10:43.460780+0000","last_became_active":"2023-11-24T08:10:41.411937+0000","last_became_peered":"2023-11-24T08:10:41.411937+0000","last_unstale":"2023-11-24T08:10:43.460780+0000","last_undegraded":"2023-11-24T08:10:43.460780+0000","last_fullsized":"2023-11-24T08:10:43.460780+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:55.816463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460823+0000","last_change":"2023-11-24T08:10:41.421738+0000","last_active":"2023-11-24T08:10:43.460823+0000","last_peered":"2023-11-24T08:10:43.460823+0000","last_clean":"2023-11-24T08:10:43.460823+0000","last_became_active":"2023-11-24T08:10:41.421430+0000","last_became_peered":"2023-11-24T08:10:41.421430+0000","last_unstale":"2023-11-24T08:10:43.460823+0000","last_undegraded":"2023-11-24T08:10:43.460823+0000","last_fullsized":"2023-11-24T08:10:43.460823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:11:18.836423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460593+0000","last_change":"2023-11-24T08:10:41.422139+0000","last_active":"2023-11-24T08:10:43.460593+0000","last_peered":"2023-11-24T08:10:43.460593+0000","last_clean":"2023-11-24T08:10:43.460593+0000","last_became_active":"2023-11-24T08:10:41.421863+0000","last_became_peered":"2023-11-24T08:10:41.421863+0000","last_unstale":"2023-11-24T08:10:43.460593+0000","last_undegraded":"2023-11-24T08:10:43.460593+0000","last_fullsized":"2023-11-24T08:10:43.460593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:50.101523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579261+0000","last_change":"2023-11-24T08:10:41.432577+0000","last_active":"2023-11-24T08:10:43.579261+0000","last_peered":"2023-11-24T08:10:43.579261+0000","last_clean":"2023-11-24T08:10:43.579261+0000","last_became_active":"2023-11-24T08:10:41.432317+0000","last_became_peered":"2023-11-24T08:10:41.432317+0000","last_unstale":"2023-11-24T08:10:43.579261+0000","last_undegraded":"2023-11-24T08:10:43.579261+0000","last_fullsized":"2023-11-24T08:10:43.579261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:17.035500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578429+0000","last_change":"2023-11-24T08:10:41.416555+0000","last_active":"2023-11-24T08:10:43.578429+0000","last_peered":"2023-11-24T08:10:43.578429+0000","last_clean":"2023-11-24T08:10:43.578429+0000","last_became_active":"2023-11-24T08:10:41.416334+0000","last_became_peered":"2023-11-24T08:10:41.416334+0000","last_unstale":"2023-11-24T08:10:43.578429+0000","last_undegraded":"2023-11-24T08:10:43.578429+0000","last_fullsized":"2023-11-24T08:10:43.578429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:46.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460269+0000","last_change":"2023-11-24T08:10:41.423044+0000","last_active":"2023-11-24T08:10:43.460269+0000","last_peered":"2023-11-24T08:10:43.460269+0000","last_clean":"2023-11-24T08:10:43.460269+0000","last_became_active":"2023-11-24T08:10:41.422488+0000","last_became_peered":"2023-11-24T08:10:41.422488+0000","last_unstale":"2023-11-24T08:10:43.460269+0000","last_undegraded":"2023-11-24T08:10:43.460269+0000","last_fullsized":"2023-11-24T08:10:43.460269+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:16.545838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408585+0000","last_change":"2023-11-24T08:10:41.410547+0000","last_active":"2023-11-24T08:10:42.408585+0000","last_peered":"2023-11-24T08:10:42.408585+0000","last_clean":"2023-11-24T08:10:42.408585+0000","last_became_active":"2023-11-24T08:10:41.410256+0000","last_became_peered":"2023-11-24T08:10:41.410256+0000","last_unstale":"2023-11-24T08:10:42.408585+0000","last_undegraded":"2023-11-24T08:10:42.408585+0000","last_fullsized":"2023-11-24T08:10:42.408585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:37.188403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409950+0000","last_change":"2023-11-24T08:10:41.433098+0000","last_active":"2023-11-24T08:10:42.409950+0000","last_peered":"2023-11-24T08:10:42.409950+0000","last_clean":"2023-11-24T08:10:42.409950+0000","last_became_active":"2023-11-24T08:10:41.432814+0000","last_became_peered":"2023-11-24T08:10:41.432814+0000","last_unstale":"2023-11-24T08:10:42.409950+0000","last_undegraded":"2023-11-24T08:10:42.409950+0000","last_fullsized":"2023-11-24T08:10:42.409950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:30.553777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641911+0000","last_change":"2023-11-24T08:10:41.418337+0000","last_active":"2023-11-24T08:10:42.641911+0000","last_peered":"2023-11-24T08:10:42.641911+0000","last_clean":"2023-11-24T08:10:42.641911+0000","last_became_active":"2023-11-24T08:10:41.416490+0000","last_became_peered":"2023-11-24T08:10:41.416490+0000","last_unstale":"2023-11-24T08:10:42.641911+0000","last_undegraded":"2023-11-24T08:10:42.641911+0000","last_fullsized":"2023-11-24T08:10:42.641911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:16.545838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641675+0000","last_change":"2023-11-24T08:10:41.427781+0000","last_active":"2023-11-24T08:10:42.641675+0000","last_peered":"2023-11-24T08:10:42.641675+0000","last_clean":"2023-11-24T08:10:42.641675+0000","last_became_active":"2023-11-24T08:10:41.427474+0000","last_became_peered":"2023-11-24T08:10:41.427474+0000","last_unstale":"2023-11-24T08:10:42.641675+0000","last_undegraded":"2023-11-24T08:10:42.641675+0000","last_fullsized":"2023-11-24T08:10:42.641675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:28:57.920963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412142+0000","last_change":"2023-11-24T08:10:41.429736+0000","last_active":"2023-11-24T08:10:42.412142+0000","last_peered":"2023-11-24T08:10:42.412142+0000","last_clean":"2023-11-24T08:10:42.412142+0000","last_became_active":"2023-11-24T08:10:41.428905+0000","last_became_peered":"2023-11-24T08:10:41.428905+0000","last_unstale":"2023-11-24T08:10:42.412142+0000","last_undegraded":"2023-11-24T08:10:42.412142+0000","last_fullsized":"2023-11-24T08:10:42.412142+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:35.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578477+0000","last_change":"2023-11-24T08:10:41.428988+0000","last_active":"2023-11-24T08:10:43.578477+0000","last_peered":"2023-11-24T08:10:43.578477+0000","last_clean":"2023-11-24T08:10:43.578477+0000","last_became_active":"2023-11-24T08:10:41.428498+0000","last_became_peered":"2023-11-24T08:10:41.428498+0000","last_unstale":"2023-11-24T08:10:43.578477+0000","last_undegraded":"2023-11-24T08:10:43.578477+0000","last_fullsized":"2023-11-24T08:10:43.578477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:46.805190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409473+0000","last_change":"2023-11-24T08:10:41.411951+0000","last_active":"2023-11-24T08:10:42.409473+0000","last_peered":"2023-11-24T08:10:42.409473+0000","last_clean":"2023-11-24T08:10:42.409473+0000","last_became_active":"2023-11-24T08:10:41.411683+0000","last_became_peered":"2023-11-24T08:10:41.411683+0000","last_unstale":"2023-11-24T08:10:42.409473+0000","last_undegraded":"2023-11-24T08:10:42.409473+0000","last_fullsized":"2023-11-24T08:10:42.409473+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:16.278290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460241+0000","last_change":"2023-11-24T08:10:41.421911+0000","last_active":"2023-11-24T08:10:43.460241+0000","last_peered":"2023-11-24T08:10:43.460241+0000","last_clean":"2023-11-24T08:10:43.460241+0000","last_became_active":"2023-11-24T08:10:41.421454+0000","last_became_peered":"2023-11-24T08:10:41.421454+0000","last_unstale":"2023-11-24T08:10:43.460241+0000","last_undegraded":"2023-11-24T08:10:43.460241+0000","last_fullsized":"2023-11-24T08:10:43.460241+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:28:57.920963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410258+0000","last_change":"2023-11-24T08:10:41.417943+0000","last_active":"2023-11-24T08:10:42.410258+0000","last_peered":"2023-11-24T08:10:42.410258+0000","last_clean":"2023-11-24T08:10:42.410258+0000","last_became_active":"2023-11-24T08:10:41.416617+0000","last_became_peered":"2023-11-24T08:10:41.416617+0000","last_unstale":"2023-11-24T08:10:42.410258+0000","last_undegraded":"2023-11-24T08:10:42.410258+0000","last_fullsized":"2023-11-24T08:10:42.410258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:08.645484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410783+0000","last_change":"2023-11-24T08:10:41.421027+0000","last_active":"2023-11-24T08:10:42.410783+0000","last_peered":"2023-11-24T08:10:42.410783+0000","last_clean":"2023-11-24T08:10:42.410783+0000","last_became_active":"2023-11-24T08:10:41.420532+0000","last_became_peered":"2023-11-24T08:10:41.420532+0000","last_unstale":"2023-11-24T08:10:42.410783+0000","last_undegraded":"2023-11-24T08:10:42.410783+0000","last_fullsized":"2023-11-24T08:10:42.410783+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:49.565301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.411729+0000","last_change":"2023-11-24T08:10:41.410624+0000","last_active":"2023-11-24T08:10:42.411729+0000","last_peered":"2023-11-24T08:10:42.411729+0000","last_clean":"2023-11-24T08:10:42.411729+0000","last_became_active":"2023-11-24T08:10:41.409598+0000","last_became_peered":"2023-11-24T08:10:41.409598+0000","last_unstale":"2023-11-24T08:10:42.411729+0000","last_undegraded":"2023-11-24T08:10:42.411729+0000","last_fullsized":"2023-11-24T08:10:42.411729+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:54.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578541+0000","last_change":"2023-11-24T08:10:41.417578+0000","last_active":"2023-11-24T08:10:43.578541+0000","last_peered":"2023-11-24T08:10:43.578541+0000","last_clean":"2023-11-24T08:10:43.578541+0000","last_became_active":"2023-11-24T08:10:41.416835+0000","last_became_peered":"2023-11-24T08:10:41.416835+0000","last_unstale":"2023-11-24T08:10:43.578541+0000","last_undegraded":"2023-11-24T08:10:43.578541+0000","last_fullsized":"2023-11-24T08:10:43.578541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:50.461147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641650+0000","last_change":"2023-11-24T08:10:41.414079+0000","last_active":"2023-11-24T08:10:42.641650+0000","last_peered":"2023-11-24T08:10:42.641650+0000","last_clean":"2023-11-24T08:10:42.641650+0000","last_became_active":"2023-11-24T08:10:41.413805+0000","last_became_peered":"2023-11-24T08:10:41.413805+0000","last_unstale":"2023-11-24T08:10:42.641650+0000","last_undegraded":"2023-11-24T08:10:42.641650+0000","last_fullsized":"2023-11-24T08:10:42.641650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:01.261020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579474+0000","last_change":"2023-11-24T08:10:41.432464+0000","last_active":"2023-11-24T08:10:43.579474+0000","last_peered":"2023-11-24T08:10:43.579474+0000","last_clean":"2023-11-24T08:10:43.579474+0000","last_became_active":"2023-11-24T08:10:41.432206+0000","last_became_peered":"2023-11-24T08:10:41.432206+0000","last_unstale":"2023-11-24T08:10:43.579474+0000","last_undegraded":"2023-11-24T08:10:43.579474+0000","last_fullsized":"2023-11-24T08:10:43.579474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:11.818427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410442+0000","last_change":"2023-11-24T08:10:41.411330+0000","last_active":"2023-11-24T08:10:42.410442+0000","last_peered":"2023-11-24T08:10:42.410442+0000","last_clean":"2023-11-24T08:10:42.410442+0000","last_became_active":"2023-11-24T08:10:41.406404+0000","last_became_peered":"2023-11-24T08:10:41.406404+0000","last_unstale":"2023-11-24T08:10:42.410442+0000","last_undegraded":"2023-11-24T08:10:42.410442+0000","last_fullsized":"2023-11-24T08:10:42.410442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:32.711956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641144+0000","last_change":"2023-11-24T08:10:41.415700+0000","last_active":"2023-11-24T08:10:42.641144+0000","last_peered":"2023-11-24T08:10:42.641144+0000","last_clean":"2023-11-24T08:10:42.641144+0000","last_became_active":"2023-11-24T08:10:41.415422+0000","last_became_peered":"2023-11-24T08:10:41.415422+0000","last_unstale":"2023-11-24T08:10:42.641144+0000","last_undegraded":"2023-11-24T08:10:42.641144+0000","last_fullsized":"2023-11-24T08:10:42.641144+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:55.962668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408844+0000","last_change":"2023-11-24T08:10:41.413932+0000","last_active":"2023-11-24T08:10:42.408844+0000","last_peered":"2023-11-24T08:10:42.408844+0000","last_clean":"2023-11-24T08:10:42.408844+0000","last_became_active":"2023-11-24T08:10:41.413612+0000","last_became_peered":"2023-11-24T08:10:41.413612+0000","last_unstale":"2023-11-24T08:10:42.408844+0000","last_undegraded":"2023-11-24T08:10:42.408844+0000","last_fullsized":"2023-11-24T08:10:42.408844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:09:35.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578110+0000","last_change":"2023-11-24T08:10:41.416554+0000","last_active":"2023-11-24T08:10:43.578110+0000","last_peered":"2023-11-24T08:10:43.578110+0000","last_clean":"2023-11-24T08:10:43.578110+0000","last_became_active":"2023-11-24T08:10:41.416251+0000","last_became_peered":"2023-11-24T08:10:41.416251+0000","last_unstale":"2023-11-24T08:10:43.578110+0000","last_undegraded":"2023-11-24T08:10:43.578110+0000","last_fullsized":"2023-11-24T08:10:43.578110+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:23.791573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.551681+0000","last_change":"2023-11-24T08:10:41.412005+0000","last_active":"2023-11-24T08:10:43.551681+0000","last_peered":"2023-11-24T08:10:43.551681+0000","last_clean":"2023-11-24T08:10:43.551681+0000","last_became_active":"2023-11-24T08:10:41.411675+0000","last_became_peered":"2023-11-24T08:10:41.411675+0000","last_unstale":"2023-11-24T08:10:43.551681+0000","last_undegraded":"2023-11-24T08:10:43.551681+0000","last_fullsized":"2023-11-24T08:10:43.551681+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:42.301586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.425101+0000","last_change":"2023-11-24T08:10:40.393255+0000","last_active":"2023-11-24T08:10:41.425101+0000","last_peered":"2023-11-24T08:10:41.425101+0000","last_clean":"2023-11-24T08:10:41.425101+0000","last_became_active":"2023-11-24T08:10:40.392572+0000","last_became_peered":"2023-11-24T08:10:40.392572+0000","last_unstale":"2023-11-24T08:10:41.425101+0000","last_undegraded":"2023-11-24T08:10:41.425101+0000","last_fullsized":"2023-11-24T08:10:41.425101+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:49.087931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412088+0000","last_change":"2023-11-24T08:10:41.409867+0000","last_active":"2023-11-24T08:10:42.412088+0000","last_peered":"2023-11-24T08:10:42.412088+0000","last_clean":"2023-11-24T08:10:42.412088+0000","last_became_active":"2023-11-24T08:10:41.409513+0000","last_became_peered":"2023-11-24T08:10:41.409513+0000","last_unstale":"2023-11-24T08:10:42.412088+0000","last_undegraded":"2023-11-24T08:10:42.412088+0000","last_fullsized":"2023-11-24T08:10:42.412088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:39.663253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.420757+0000","last_change":"2023-11-24T08:10:40.392959+0000","last_active":"2023-11-24T08:10:41.420757+0000","last_peered":"2023-11-24T08:10:41.420757+0000","last_clean":"2023-11-24T08:10:41.420757+0000","last_became_active":"2023-11-24T08:10:40.392620+0000","last_became_peered":"2023-11-24T08:10:40.392620+0000","last_unstale":"2023-11-24T08:10:41.420757+0000","last_undegraded":"2023-11-24T08:10:41.420757+0000","last_fullsized":"2023-11-24T08:10:41.420757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:41.287994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579554+0000","last_change":"2023-11-24T08:10:41.413632+0000","last_active":"2023-11-24T08:10:43.579554+0000","last_peered":"2023-11-24T08:10:43.579554+0000","last_clean":"2023-11-24T08:10:43.579554+0000","last_became_active":"2023-11-24T08:10:41.413348+0000","last_became_peered":"2023-11-24T08:10:41.413348+0000","last_unstale":"2023-11-24T08:10:43.579554+0000","last_undegraded":"2023-11-24T08:10:43.579554+0000","last_fullsized":"2023-11-24T08:10:43.579554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:25.940595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.413125+0000","last_change":"2023-11-24T08:10:40.391717+0000","last_active":"2023-11-24T08:10:41.413125+0000","last_peered":"2023-11-24T08:10:41.413125+0000","last_clean":"2023-11-24T08:10:41.413125+0000","last_became_active":"2023-11-24T08:10:40.391477+0000","last_became_peered":"2023-11-24T08:10:40.391477+0000","last_unstale":"2023-11-24T08:10:41.413125+0000","last_undegraded":"2023-11-24T08:10:41.413125+0000","last_fullsized":"2023-11-24T08:10:41.413125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:06.464620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.552452+0000","last_change":"2023-11-24T08:10:41.410142+0000","last_active":"2023-11-24T08:10:43.552452+0000","last_peered":"2023-11-24T08:10:43.552452+0000","last_clean":"2023-11-24T08:10:43.552452+0000","last_became_active":"2023-11-24T08:10:41.409736+0000","last_became_peered":"2023-11-24T08:10:41.409736+0000","last_unstale":"2023-11-24T08:10:43.552452+0000","last_undegraded":"2023-11-24T08:10:43.552452+0000","last_fullsized":"2023-11-24T08:10:43.552452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:39.663253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.423598+0000","last_change":"2023-11-24T08:10:40.400930+0000","last_active":"2023-11-24T08:10:41.423598+0000","last_peered":"2023-11-24T08:10:41.423598+0000","last_clean":"2023-11-24T08:10:41.423598+0000","last_became_active":"2023-11-24T08:10:40.400220+0000","last_became_peered":"2023-11-24T08:10:40.400220+0000","last_unstale":"2023-11-24T08:10:41.423598+0000","last_undegraded":"2023-11-24T08:10:41.423598+0000","last_fullsized":"2023-11-24T08:10:41.423598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:50:19.777593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641163+0000","last_change":"2023-11-24T08:10:41.416316+0000","last_active":"2023-11-24T08:10:42.641163+0000","last_peered":"2023-11-24T08:10:42.641163+0000","last_clean":"2023-11-24T08:10:42.641163+0000","last_became_active":"2023-11-24T08:10:41.416038+0000","last_became_peered":"2023-11-24T08:10:41.416038+0000","last_unstale":"2023-11-24T08:10:42.641163+0000","last_undegraded":"2023-11-24T08:10:42.641163+0000","last_fullsized":"2023-11-24T08:10:42.641163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:43.691758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417158+0000","last_change":"2023-11-24T08:10:40.394178+0000","last_active":"2023-11-24T08:10:41.417158+0000","last_peered":"2023-11-24T08:10:41.417158+0000","last_clean":"2023-11-24T08:10:41.417158+0000","last_became_active":"2023-11-24T08:10:40.393917+0000","last_became_peered":"2023-11-24T08:10:40.393917+0000","last_unstale":"2023-11-24T08:10:41.417158+0000","last_undegraded":"2023-11-24T08:10:41.417158+0000","last_fullsized":"2023-11-24T08:10:41.417158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:54.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460437+0000","last_change":"2023-11-24T08:10:41.429543+0000","last_active":"2023-11-24T08:10:43.460437+0000","last_peered":"2023-11-24T08:10:43.460437+0000","last_clean":"2023-11-24T08:10:43.460437+0000","last_became_active":"2023-11-24T08:10:41.429272+0000","last_became_peered":"2023-11-24T08:10:41.429272+0000","last_unstale":"2023-11-24T08:10:43.460437+0000","last_undegraded":"2023-11-24T08:10:43.460437+0000","last_fullsized":"2023-11-24T08:10:43.460437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:28.940501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.414400+0000","last_change":"2023-11-24T08:10:40.390840+0000","last_active":"2023-11-24T08:10:41.414400+0000","last_peered":"2023-11-24T08:10:41.414400+0000","last_clean":"2023-11-24T08:10:41.414400+0000","last_became_active":"2023-11-24T08:10:40.390315+0000","last_became_peered":"2023-11-24T08:10:40.390315+0000","last_unstale":"2023-11-24T08:10:41.414400+0000","last_undegraded":"2023-11-24T08:10:41.414400+0000","last_fullsized":"2023-11-24T08:10:41.414400+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:06.464620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460175+0000","last_change":"2023-11-24T08:10:41.434664+0000","last_active":"2023-11-24T08:10:43.460175+0000","last_peered":"2023-11-24T08:10:43.460175+0000","last_clean":"2023-11-24T08:10:43.460175+0000","last_became_active":"2023-11-24T08:10:41.426919+0000","last_became_peered":"2023-11-24T08:10:41.426919+0000","last_unstale":"2023-11-24T08:10:43.460175+0000","last_undegraded":"2023-11-24T08:10:43.460175+0000","last_fullsized":"2023-11-24T08:10:43.460175+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:01.261020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.396243+0000","last_change":"2023-11-24T08:10:40.390774+0000","last_active":"2023-11-24T08:10:41.396243+0000","last_peered":"2023-11-24T08:10:41.396243+0000","last_clean":"2023-11-24T08:10:41.396243+0000","last_became_active":"2023-11-24T08:10:40.390495+0000","last_became_peered":"2023-11-24T08:10:40.390495+0000","last_unstale":"2023-11-24T08:10:41.396243+0000","last_undegraded":"2023-11-24T08:10:41.396243+0000","last_fullsized":"2023-11-24T08:10:41.396243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:14.655179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409909+0000","last_change":"2023-11-24T08:10:41.418194+0000","last_active":"2023-11-24T08:10:42.409909+0000","last_peered":"2023-11-24T08:10:42.409909+0000","last_clean":"2023-11-24T08:10:42.409909+0000","last_became_active":"2023-11-24T08:10:41.416986+0000","last_became_peered":"2023-11-24T08:10:41.416986+0000","last_unstale":"2023-11-24T08:10:42.409909+0000","last_undegraded":"2023-11-24T08:10:42.409909+0000","last_fullsized":"2023-11-24T08:10:42.409909+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:23.175828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.411720+0000","last_change":"2023-11-24T08:10:40.393357+0000","last_active":"2023-11-24T08:10:41.411720+0000","last_peered":"2023-11-24T08:10:41.411720+0000","last_clean":"2023-11-24T08:10:41.411720+0000","last_became_active":"2023-11-24T08:10:40.391265+0000","last_became_peered":"2023-11-24T08:10:40.391265+0000","last_unstale":"2023-11-24T08:10:41.411720+0000","last_undegraded":"2023-11-24T08:10:41.411720+0000","last_fullsized":"2023-11-24T08:10:41.411720+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:18.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641924+0000","last_change":"2023-11-24T08:10:41.427725+0000","last_active":"2023-11-24T08:10:42.641924+0000","last_peered":"2023-11-24T08:10:42.641924+0000","last_clean":"2023-11-24T08:10:42.641924+0000","last_became_active":"2023-11-24T08:10:41.427451+0000","last_became_peered":"2023-11-24T08:10:41.427451+0000","last_unstale":"2023-11-24T08:10:42.641924+0000","last_undegraded":"2023-11-24T08:10:42.641924+0000","last_fullsized":"2023-11-24T08:10:42.641924+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:28.940501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.403287+0000","last_change":"2023-11-24T08:10:40.382626+0000","last_active":"2023-11-24T08:10:41.403287+0000","last_peered":"2023-11-24T08:10:41.403287+0000","last_clean":"2023-11-24T08:10:41.403287+0000","last_became_active":"2023-11-24T08:10:40.381229+0000","last_became_peered":"2023-11-24T08:10:40.381229+0000","last_unstale":"2023-11-24T08:10:41.403287+0000","last_undegraded":"2023-11-24T08:10:41.403287+0000","last_fullsized":"2023-11-24T08:10:41.403287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:34.574303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.413359+0000","last_change":"2023-11-24T08:10:41.424059+0000","last_active":"2023-11-24T08:10:42.413359+0000","last_peered":"2023-11-24T08:10:42.413359+0000","last_clean":"2023-11-24T08:10:42.413359+0000","last_became_active":"2023-11-24T08:10:41.423565+0000","last_became_peered":"2023-11-24T08:10:41.423565+0000","last_unstale":"2023-11-24T08:10:42.413359+0000","last_undegraded":"2023-11-24T08:10:42.413359+0000","last_fullsized":"2023-11-24T08:10:42.413359+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:11:18.836423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.398130+0000","last_change":"2023-11-24T08:10:40.396039+0000","last_active":"2023-11-24T08:10:41.398130+0000","last_peered":"2023-11-24T08:10:41.398130+0000","last_clean":"2023-11-24T08:10:41.398130+0000","last_became_active":"2023-11-24T08:10:40.395680+0000","last_became_peered":"2023-11-24T08:10:40.395680+0000","last_unstale":"2023-11-24T08:10:41.398130+0000","last_undegraded":"2023-11-24T08:10:41.398130+0000","last_fullsized":"2023-11-24T08:10:41.398130+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:31.698363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409720+0000","last_change":"2023-11-24T08:10:41.432591+0000","last_active":"2023-11-24T08:10:42.409720+0000","last_peered":"2023-11-24T08:10:42.409720+0000","last_clean":"2023-11-24T08:10:42.409720+0000","last_became_active":"2023-11-24T08:10:41.432305+0000","last_became_peered":"2023-11-24T08:10:41.432305+0000","last_unstale":"2023-11-24T08:10:42.409720+0000","last_undegraded":"2023-11-24T08:10:42.409720+0000","last_fullsized":"2023-11-24T08:10:42.409720+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:16:28.486289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.430812+0000","last_change":"2023-11-24T08:10:40.400513+0000","last_active":"2023-11-24T08:10:41.430812+0000","last_peered":"2023-11-24T08:10:41.430812+0000","last_clean":"2023-11-24T08:10:41.430812+0000","last_became_active":"2023-11-24T08:10:40.399876+0000","last_became_peered":"2023-11-24T08:10:40.399876+0000","last_unstale":"2023-11-24T08:10:41.430812+0000","last_undegraded":"2023-11-24T08:10:41.430812+0000","last_fullsized":"2023-11-24T08:10:41.430812+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:28:56.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412803+0000","last_change":"2023-11-24T08:10:41.427142+0000","last_active":"2023-11-24T08:10:42.412803+0000","last_peered":"2023-11-24T08:10:42.412803+0000","last_clean":"2023-11-24T08:10:42.412803+0000","last_became_active":"2023-11-24T08:10:41.426847+0000","last_became_peered":"2023-11-24T08:10:41.426847+0000","last_unstale":"2023-11-24T08:10:42.412803+0000","last_undegraded":"2023-11-24T08:10:42.412803+0000","last_fullsized":"2023-11-24T08:10:42.412803+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:12.381653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.436940+0000","last_change":"2023-11-24T08:10:40.378220+0000","last_active":"2023-11-24T08:10:41.436940+0000","last_peered":"2023-11-24T08:10:41.436940+0000","last_clean":"2023-11-24T08:10:41.436940+0000","last_became_active":"2023-11-24T08:10:40.377915+0000","last_became_peered":"2023-11-24T08:10:40.377915+0000","last_unstale":"2023-11-24T08:10:41.436940+0000","last_undegraded":"2023-11-24T08:10:41.436940+0000","last_fullsized":"2023-11-24T08:10:41.436940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:12.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410905+0000","last_change":"2023-11-24T08:10:41.414931+0000","last_active":"2023-11-24T08:10:42.410905+0000","last_peered":"2023-11-24T08:10:42.410905+0000","last_clean":"2023-11-24T08:10:42.410905+0000","last_became_active":"2023-11-24T08:10:41.414621+0000","last_became_peered":"2023-11-24T08:10:41.414621+0000","last_unstale":"2023-11-24T08:10:42.410905+0000","last_undegraded":"2023-11-24T08:10:42.410905+0000","last_fullsized":"2023-11-24T08:10:42.410905+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:07.478213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.425599+0000","last_change":"2023-11-24T08:10:40.395756+0000","last_active":"2023-11-24T08:10:41.425599+0000","last_peered":"2023-11-24T08:10:41.425599+0000","last_clean":"2023-11-24T08:10:41.425599+0000","last_became_active":"2023-11-24T08:10:40.395493+0000","last_became_peered":"2023-11-24T08:10:40.395493+0000","last_unstale":"2023-11-24T08:10:41.425599+0000","last_undegraded":"2023-11-24T08:10:41.425599+0000","last_fullsized":"2023-11-24T08:10:41.425599+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:11.368061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460619+0000","last_change":"2023-11-24T08:10:41.422495+0000","last_active":"2023-11-24T08:10:43.460619+0000","last_peered":"2023-11-24T08:10:43.460619+0000","last_clean":"2023-11-24T08:10:43.460619+0000","last_became_active":"2023-11-24T08:10:41.422232+0000","last_became_peered":"2023-11-24T08:10:41.422232+0000","last_unstale":"2023-11-24T08:10:43.460619+0000","last_undegraded":"2023-11-24T08:10:43.460619+0000","last_fullsized":"2023-11-24T08:10:43.460619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:42.167949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.416197+0000","last_change":"2023-11-24T08:10:40.397297+0000","last_active":"2023-11-24T08:10:41.416197+0000","last_peered":"2023-11-24T08:10:41.416197+0000","last_clean":"2023-11-24T08:10:41.416197+0000","last_became_active":"2023-11-24T08:10:40.397019+0000","last_became_peered":"2023-11-24T08:10:40.397019+0000","last_unstale":"2023-11-24T08:10:41.416197+0000","last_undegraded":"2023-11-24T08:10:41.416197+0000","last_fullsized":"2023-11-24T08:10:41.416197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:18.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408655+0000","last_change":"2023-11-24T08:10:41.429888+0000","last_active":"2023-11-24T08:10:42.408655+0000","last_peered":"2023-11-24T08:10:42.408655+0000","last_clean":"2023-11-24T08:10:42.408655+0000","last_became_active":"2023-11-24T08:10:41.429609+0000","last_became_peered":"2023-11-24T08:10:41.429609+0000","last_unstale":"2023-11-24T08:10:42.408655+0000","last_undegraded":"2023-11-24T08:10:42.408655+0000","last_fullsized":"2023-11-24T08:10:42.408655+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:51:20.855433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.403023+0000","last_change":"2023-11-24T08:10:40.396293+0000","last_active":"2023-11-24T08:10:41.403023+0000","last_peered":"2023-11-24T08:10:41.403023+0000","last_clean":"2023-11-24T08:10:41.403023+0000","last_became_active":"2023-11-24T08:10:40.393238+0000","last_became_peered":"2023-11-24T08:10:40.393238+0000","last_unstale":"2023-11-24T08:10:41.403023+0000","last_undegraded":"2023-11-24T08:10:41.403023+0000","last_fullsized":"2023-11-24T08:10:41.403023+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:51.265083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412759+0000","last_change":"2023-11-24T08:10:41.422966+0000","last_active":"2023-11-24T08:10:42.412759+0000","last_peered":"2023-11-24T08:10:42.412759+0000","last_clean":"2023-11-24T08:10:42.412759+0000","last_became_active":"2023-11-24T08:10:41.422642+0000","last_became_peered":"2023-11-24T08:10:41.422642+0000","last_unstale":"2023-11-24T08:10:42.412759+0000","last_undegraded":"2023-11-24T08:10:42.412759+0000","last_fullsized":"2023-11-24T08:10:42.412759+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:50.101523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.437314+0000","last_change":"2023-11-24T08:10:40.391642+0000","last_active":"2023-11-24T08:10:41.437314+0000","last_peered":"2023-11-24T08:10:41.437314+0000","last_clean":"2023-11-24T08:10:41.437314+0000","last_became_active":"2023-11-24T08:10:40.389315+0000","last_became_peered":"2023-11-24T08:10:40.389315+0000","last_unstale":"2023-11-24T08:10:41.437314+0000","last_undegraded":"2023-11-24T08:10:41.437314+0000","last_fullsized":"2023-11-24T08:10:41.437314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:50:19.777593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578735+0000","last_change":"2023-11-24T08:10:41.411050+0000","last_active":"2023-11-24T08:10:43.578735+0000","last_peered":"2023-11-24T08:10:43.578735+0000","last_clean":"2023-11-24T08:10:43.578735+0000","last_became_active":"2023-11-24T08:10:41.410799+0000","last_became_peered":"2023-11-24T08:10:41.410799+0000","last_unstale":"2023-11-24T08:10:43.578735+0000","last_undegraded":"2023-11-24T08:10:43.578735+0000","last_fullsized":"2023-11-24T08:10:43.578735+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:58:21.722380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417836+0000","last_change":"2023-11-24T08:10:40.387475+0000","last_active":"2023-11-24T08:10:41.417836+0000","last_peered":"2023-11-24T08:10:41.417836+0000","last_clean":"2023-11-24T08:10:41.417836+0000","last_became_active":"2023-11-24T08:10:40.386342+0000","last_became_peered":"2023-11-24T08:10:40.386342+0000","last_unstale":"2023-11-24T08:10:41.417836+0000","last_undegraded":"2023-11-24T08:10:41.417836+0000","last_fullsized":"2023-11-24T08:10:41.417836+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:15.430616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460933+0000","last_change":"2023-11-24T08:10:41.423141+0000","last_active":"2023-11-24T08:10:43.460933+0000","last_peered":"2023-11-24T08:10:43.460933+0000","last_clean":"2023-11-24T08:10:43.460933+0000","last_became_active":"2023-11-24T08:10:41.422861+0000","last_became_peered":"2023-11-24T08:10:41.422861+0000","last_unstale":"2023-11-24T08:10:43.460933+0000","last_undegraded":"2023-11-24T08:10:43.460933+0000","last_fullsized":"2023-11-24T08:10:43.460933+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:37.188403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.418770+0000","last_change":"2023-11-24T08:10:40.382415+0000","last_active":"2023-11-24T08:10:41.418770+0000","last_peered":"2023-11-24T08:10:41.418770+0000","last_clean":"2023-11-24T08:10:41.418770+0000","last_became_active":"2023-11-24T08:10:40.382090+0000","last_became_peered":"2023-11-24T08:10:40.382090+0000","last_unstale":"2023-11-24T08:10:41.418770+0000","last_undegraded":"2023-11-24T08:10:41.418770+0000","last_fullsized":"2023-11-24T08:10:41.418770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:18.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.411535+0000","last_change":"2023-11-24T08:10:41.411739+0000","last_active":"2023-11-24T08:10:42.411535+0000","last_peered":"2023-11-24T08:10:42.411535+0000","last_clean":"2023-11-24T08:10:42.411535+0000","last_became_active":"2023-11-24T08:10:41.411478+0000","last_became_peered":"2023-11-24T08:10:41.411478+0000","last_unstale":"2023-11-24T08:10:42.411535+0000","last_undegraded":"2023-11-24T08:10:42.411535+0000","last_fullsized":"2023-11-24T08:10:42.411535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:41.371090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.416456+0000","last_change":"2023-11-24T08:10:40.390983+0000","last_active":"2023-11-24T08:10:41.416456+0000","last_peered":"2023-11-24T08:10:41.416456+0000","last_clean":"2023-11-24T08:10:41.416456+0000","last_became_active":"2023-11-24T08:10:40.389721+0000","last_became_peered":"2023-11-24T08:10:40.389721+0000","last_unstale":"2023-11-24T08:10:41.416456+0000","last_undegraded":"2023-11-24T08:10:41.416456+0000","last_fullsized":"2023-11-24T08:10:41.416456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:04.284926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.411889+0000","last_change":"2023-11-24T08:10:41.407461+0000","last_active":"2023-11-24T08:10:42.411889+0000","last_peered":"2023-11-24T08:10:42.411889+0000","last_clean":"2023-11-24T08:10:42.411889+0000","last_became_active":"2023-11-24T08:10:41.407137+0000","last_became_peered":"2023-11-24T08:10:41.407137+0000","last_unstale":"2023-11-24T08:10:42.411889+0000","last_undegraded":"2023-11-24T08:10:42.411889+0000","last_fullsized":"2023-11-24T08:10:42.411889+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:42.301586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.402508+0000","last_change":"2023-11-24T08:10:40.396743+0000","last_active":"2023-11-24T08:10:41.402508+0000","last_peered":"2023-11-24T08:10:41.402508+0000","last_clean":"2023-11-24T08:10:41.402508+0000","last_became_active":"2023-11-24T08:10:40.396292+0000","last_became_peered":"2023-11-24T08:10:40.396292+0000","last_unstale":"2023-11-24T08:10:41.402508+0000","last_undegraded":"2023-11-24T08:10:41.402508+0000","last_fullsized":"2023-11-24T08:10:41.402508+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:11:17.822830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408950+0000","last_change":"2023-11-24T08:10:41.433540+0000","last_active":"2023-11-24T08:10:42.408950+0000","last_peered":"2023-11-24T08:10:42.408950+0000","last_clean":"2023-11-24T08:10:42.408950+0000","last_became_active":"2023-11-24T08:10:41.433249+0000","last_became_peered":"2023-11-24T08:10:41.433249+0000","last_unstale":"2023-11-24T08:10:42.408950+0000","last_undegraded":"2023-11-24T08:10:42.408950+0000","last_fullsized":"2023-11-24T08:10:42.408950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:49.150477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417254+0000","last_change":"2023-11-24T08:10:40.394138+0000","last_active":"2023-11-24T08:10:41.417254+0000","last_peered":"2023-11-24T08:10:41.417254+0000","last_clean":"2023-11-24T08:10:41.417254+0000","last_became_active":"2023-11-24T08:10:40.393872+0000","last_became_peered":"2023-11-24T08:10:40.393872+0000","last_unstale":"2023-11-24T08:10:41.417254+0000","last_undegraded":"2023-11-24T08:10:41.417254+0000","last_fullsized":"2023-11-24T08:10:41.417254+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:41.154356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.551388+0000","last_change":"2023-11-24T08:10:41.409585+0000","last_active":"2023-11-24T08:10:43.551388+0000","last_peered":"2023-11-24T08:10:43.551388+0000","last_clean":"2023-11-24T08:10:43.551388+0000","last_became_active":"2023-11-24T08:10:41.407565+0000","last_became_peered":"2023-11-24T08:10:41.407565+0000","last_unstale":"2023-11-24T08:10:43.551388+0000","last_undegraded":"2023-11-24T08:10:43.551388+0000","last_fullsized":"2023-11-24T08:10:43.551388+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:54.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.392098+0000","last_change":"2023-11-24T08:10:40.393936+0000","last_active":"2023-11-24T08:10:41.392098+0000","last_peered":"2023-11-24T08:10:41.392098+0000","last_clean":"2023-11-24T08:10:41.392098+0000","last_became_active":"2023-11-24T08:10:40.393617+0000","last_became_peered":"2023-11-24T08:10:40.393617+0000","last_unstale":"2023-11-24T08:10:41.392098+0000","last_undegraded":"2023-11-24T08:10:41.392098+0000","last_fullsized":"2023-11-24T08:10:41.392098+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:06:17.056254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578132+0000","last_change":"2023-11-24T08:10:41.423108+0000","last_active":"2023-11-24T08:10:43.578132+0000","last_peered":"2023-11-24T08:10:43.578132+0000","last_clean":"2023-11-24T08:10:43.578132+0000","last_became_active":"2023-11-24T08:10:41.422850+0000","last_became_peered":"2023-11-24T08:10:41.422850+0000","last_unstale":"2023-11-24T08:10:43.578132+0000","last_undegraded":"2023-11-24T08:10:43.578132+0000","last_fullsized":"2023-11-24T08:10:43.578132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:27:48.503618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.399236+0000","last_change":"2023-11-24T08:10:40.392572+0000","last_active":"2023-11-24T08:10:41.399236+0000","last_peered":"2023-11-24T08:10:41.399236+0000","last_clean":"2023-11-24T08:10:41.399236+0000","last_became_active":"2023-11-24T08:10:40.392267+0000","last_became_peered":"2023-11-24T08:10:40.392267+0000","last_unstale":"2023-11-24T08:10:41.399236+0000","last_undegraded":"2023-11-24T08:10:41.399236+0000","last_fullsized":"2023-11-24T08:10:41.399236+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:53.195860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.459940+0000","last_change":"2023-11-24T08:10:41.413307+0000","last_active":"2023-11-24T08:10:43.459940+0000","last_peered":"2023-11-24T08:10:43.459940+0000","last_clean":"2023-11-24T08:10:43.459940+0000","last_became_active":"2023-11-24T08:10:41.413026+0000","last_became_peered":"2023-11-24T08:10:41.413026+0000","last_unstale":"2023-11-24T08:10:43.459940+0000","last_undegraded":"2023-11-24T08:10:43.459940+0000","last_fullsized":"2023-11-24T08:10:43.459940+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:36:31.790671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.415970+0000","last_change":"2023-11-24T08:10:40.397606+0000","last_active":"2023-11-24T08:10:41.415970+0000","last_peered":"2023-11-24T08:10:41.415970+0000","last_clean":"2023-11-24T08:10:41.415970+0000","last_became_active":"2023-11-24T08:10:40.397331+0000","last_became_peered":"2023-11-24T08:10:40.397331+0000","last_unstale":"2023-11-24T08:10:41.415970+0000","last_undegraded":"2023-11-24T08:10:41.415970+0000","last_fullsized":"2023-11-24T08:10:41.415970+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:51.581068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412422+0000","last_change":"2023-11-24T08:10:41.413102+0000","last_active":"2023-11-24T08:10:42.412422+0000","last_peered":"2023-11-24T08:10:42.412422+0000","last_clean":"2023-11-24T08:10:42.412422+0000","last_became_active":"2023-11-24T08:10:41.411422+0000","last_became_peered":"2023-11-24T08:10:41.411422+0000","last_unstale":"2023-11-24T08:10:42.412422+0000","last_undegraded":"2023-11-24T08:10:42.412422+0000","last_fullsized":"2023-11-24T08:10:42.412422+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:52.594661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409471+0000","last_change":"2023-11-24T08:10:40.390684+0000","last_active":"2023-11-24T08:10:42.409471+0000","last_peered":"2023-11-24T08:10:42.409471+0000","last_clean":"2023-11-24T08:10:42.409471+0000","last_became_active":"2023-11-24T08:10:40.389362+0000","last_became_peered":"2023-11-24T08:10:40.389362+0000","last_unstale":"2023-11-24T08:10:42.409471+0000","last_undegraded":"2023-11-24T08:10:42.409471+0000","last_fullsized":"2023-11-24T08:10:42.409471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:15.532245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460622+0000","last_change":"2023-11-24T08:10:41.426539+0000","last_active":"2023-11-24T08:10:43.460622+0000","last_peered":"2023-11-24T08:10:43.460622+0000","last_clean":"2023-11-24T08:10:43.460622+0000","last_became_active":"2023-11-24T08:10:41.426124+0000","last_became_peered":"2023-11-24T08:10:41.426124+0000","last_unstale":"2023-11-24T08:10:43.460622+0000","last_undegraded":"2023-11-24T08:10:43.460622+0000","last_fullsized":"2023-11-24T08:10:43.460622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:12.381653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.422174+0000","last_change":"2023-11-24T08:10:40.400533+0000","last_active":"2023-11-24T08:10:41.422174+0000","last_peered":"2023-11-24T08:10:41.422174+0000","last_clean":"2023-11-24T08:10:41.422174+0000","last_became_active":"2023-11-24T08:10:40.400241+0000","last_became_peered":"2023-11-24T08:10:40.400241+0000","last_unstale":"2023-11-24T08:10:41.422174+0000","last_undegraded":"2023-11-24T08:10:41.422174+0000","last_fullsized":"2023-11-24T08:10:41.422174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:15.532245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412881+0000","last_change":"2023-11-24T08:10:41.430166+0000","last_active":"2023-11-24T08:10:42.412881+0000","last_peered":"2023-11-24T08:10:42.412881+0000","last_clean":"2023-11-24T08:10:42.412881+0000","last_became_active":"2023-11-24T08:10:41.429887+0000","last_became_peered":"2023-11-24T08:10:41.429887+0000","last_unstale":"2023-11-24T08:10:42.412881+0000","last_undegraded":"2023-11-24T08:10:42.412881+0000","last_fullsized":"2023-11-24T08:10:42.412881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:42.167949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.388387+0000","last_change":"2023-11-24T08:10:40.391406+0000","last_active":"2023-11-24T08:10:41.388387+0000","last_peered":"2023-11-24T08:10:41.388387+0000","last_clean":"2023-11-24T08:10:41.388387+0000","last_became_active":"2023-11-24T08:10:40.391073+0000","last_became_peered":"2023-11-24T08:10:40.391073+0000","last_unstale":"2023-11-24T08:10:41.388387+0000","last_undegraded":"2023-11-24T08:10:41.388387+0000","last_fullsized":"2023-11-24T08:10:41.388387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:22.162235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409066+0000","last_change":"2023-11-24T08:10:41.416681+0000","last_active":"2023-11-24T08:10:42.409066+0000","last_peered":"2023-11-24T08:10:42.409066+0000","last_clean":"2023-11-24T08:10:42.409066+0000","last_became_active":"2023-11-24T08:10:41.416399+0000","last_became_peered":"2023-11-24T08:10:41.416399+0000","last_unstale":"2023-11-24T08:10:42.409066+0000","last_undegraded":"2023-11-24T08:10:42.409066+0000","last_fullsized":"2023-11-24T08:10:42.409066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:27.465599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417937+0000","last_change":"2023-11-24T08:10:40.400318+0000","last_active":"2023-11-24T08:10:41.417937+0000","last_peered":"2023-11-24T08:10:41.417937+0000","last_clean":"2023-11-24T08:10:41.417937+0000","last_became_active":"2023-11-24T08:10:40.400006+0000","last_became_peered":"2023-11-24T08:10:40.400006+0000","last_unstale":"2023-11-24T08:10:41.417937+0000","last_undegraded":"2023-11-24T08:10:41.417937+0000","last_fullsized":"2023-11-24T08:10:41.417937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:16.638357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409292+0000","last_change":"2023-11-24T08:10:41.432402+0000","last_active":"2023-11-24T08:10:42.409292+0000","last_peered":"2023-11-24T08:10:42.409292+0000","last_clean":"2023-11-24T08:10:42.409292+0000","last_became_active":"2023-11-24T08:10:41.432035+0000","last_became_peered":"2023-11-24T08:10:41.432035+0000","last_unstale":"2023-11-24T08:10:42.409292+0000","last_undegraded":"2023-11-24T08:10:42.409292+0000","last_fullsized":"2023-11-24T08:10:42.409292+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:06:36.606640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.401563+0000","last_change":"2023-11-24T08:10:40.387069+0000","last_active":"2023-11-24T08:10:41.401563+0000","last_peered":"2023-11-24T08:10:41.401563+0000","last_clean":"2023-11-24T08:10:41.401563+0000","last_became_active":"2023-11-24T08:10:40.386462+0000","last_became_peered":"2023-11-24T08:10:40.386462+0000","last_unstale":"2023-11-24T08:10:41.401563+0000","last_undegraded":"2023-11-24T08:10:41.401563+0000","last_fullsized":"2023-11-24T08:10:41.401563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:53.195860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578829+0000","last_change":"2023-11-24T08:10:41.410791+0000","last_active":"2023-11-24T08:10:43.578829+0000","last_peered":"2023-11-24T08:10:43.578829+0000","last_clean":"2023-11-24T08:10:43.578829+0000","last_became_active":"2023-11-24T08:10:41.410506+0000","last_became_peered":"2023-11-24T08:10:41.410506+0000","last_unstale":"2023-11-24T08:10:43.578829+0000","last_undegraded":"2023-11-24T08:10:43.578829+0000","last_fullsized":"2023-11-24T08:10:43.578829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:25.388885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.388896+0000","last_change":"2023-11-24T08:10:40.394410+0000","last_active":"2023-11-24T08:10:41.388896+0000","last_peered":"2023-11-24T08:10:41.388896+0000","last_clean":"2023-11-24T08:10:41.388896+0000","last_became_active":"2023-11-24T08:10:40.394009+0000","last_became_peered":"2023-11-24T08:10:40.394009+0000","last_unstale":"2023-11-24T08:10:41.388896+0000","last_undegraded":"2023-11-24T08:10:41.388896+0000","last_fullsized":"2023-11-24T08:10:41.388896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:07.631892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641132+0000","last_change":"2023-11-24T08:10:41.411977+0000","last_active":"2023-11-24T08:10:42.641132+0000","last_peered":"2023-11-24T08:10:42.641132+0000","last_clean":"2023-11-24T08:10:42.641132+0000","last_became_active":"2023-11-24T08:10:41.411633+0000","last_became_peered":"2023-11-24T08:10:41.411633+0000","last_unstale":"2023-11-24T08:10:42.641132+0000","last_undegraded":"2023-11-24T08:10:42.641132+0000","last_fullsized":"2023-11-24T08:10:42.641132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:17.651949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.412009+0000","last_change":"2023-11-24T08:10:40.385164+0000","last_active":"2023-11-24T08:10:41.412009+0000","last_peered":"2023-11-24T08:10:41.412009+0000","last_clean":"2023-11-24T08:10:41.412009+0000","last_became_active":"2023-11-24T08:10:40.384523+0000","last_became_peered":"2023-11-24T08:10:40.384523+0000","last_unstale":"2023-11-24T08:10:41.412009+0000","last_undegraded":"2023-11-24T08:10:41.412009+0000","last_fullsized":"2023-11-24T08:10:41.412009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:16.638357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460887+0000","last_change":"2023-11-24T08:10:41.421154+0000","last_active":"2023-11-24T08:10:43.460887+0000","last_peered":"2023-11-24T08:10:43.460887+0000","last_clean":"2023-11-24T08:10:43.460887+0000","last_became_active":"2023-11-24T08:10:41.420904+0000","last_became_peered":"2023-11-24T08:10:41.420904+0000","last_unstale":"2023-11-24T08:10:43.460887+0000","last_undegraded":"2023-11-24T08:10:43.460887+0000","last_fullsized":"2023-11-24T08:10:43.460887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:52.278676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417967+0000","last_change":"2023-11-24T08:10:40.396648+0000","last_active":"2023-11-24T08:10:41.417967+0000","last_peered":"2023-11-24T08:10:41.417967+0000","last_clean":"2023-11-24T08:10:41.417967+0000","last_became_active":"2023-11-24T08:10:40.395757+0000","last_became_peered":"2023-11-24T08:10:40.395757+0000","last_unstale":"2023-11-24T08:10:41.417967+0000","last_undegraded":"2023-11-24T08:10:41.417967+0000","last_fullsized":"2023-11-24T08:10:41.417967+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:25.960984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412225+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:10:42.412225+0000","last_peered":"2023-11-24T08:10:42.412225+0000","last_clean":"2023-11-24T08:10:42.412225+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:10:42.412225+0000","last_undegraded":"2023-11-24T08:10:42.412225+0000","last_fullsized":"2023-11-24T08:10:42.412225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":818992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.018,"15min":1.0129999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.014,"15min":0.99299999999999999},"min":{"1min":0.65300000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.141,"15min":1.129},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.323},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.179,"15min":1.167},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2250000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.093,"15min":1.0589999999999999},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.2150000000000001,"15min":1.206},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0699999999999998,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.96599999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.25,"15min":1.222},"min":{"1min":0.97999999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.151}]},{"osd":4,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.2330000000000001,"15min":1.2110000000000001},"min":{"1min":0.95599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.113},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3959999999999999,"15min":1.3779999999999999},"min":{"1min":1.0920000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.2909999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.4399999999999999,"15min":1.4390000000000001},"min":{"1min":1.0449999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.931,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.395},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.371,"15min":1.355},"min":{"1min":1.0149999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.494,"15min":1.4790000000000001},"min":{"1min":1.1890000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.482},"min":{"1min":1.016,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.8450000000000002,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.3620000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":229206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.091,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6220000000000001,"5min":2.085,"15min":2.085},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2390000000000001,"15min":1.2450000000000001},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.766,"5min":1.794,"15min":1.794},"last":1.282}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.121,"15min":1.127},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.837,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.087},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.77400000000000002,"5min":0.749,"15min":0.749},"max":{"1min":1.615,"5min":1.98,"15min":1.98},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7729999999999999,"5min":1.778,"15min":1.778},"last":1.21},{"interface":"front","average":{"1min":1.105,"5min":1.1859999999999999,"15min":1.2},"min":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7649999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.0409999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3340000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7010000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4399999999999999,"15min":1.448},"min":{"1min":0.83299999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.948,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3240000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.359,"15min":1.3740000000000001},"min":{"1min":0.92200000000000004,"5min":0.622,"15min":0.622},"max":{"1min":1.9750000000000001,"5min":2.343,"15min":2.343},"last":1.159},{"interface":"front","average":{"1min":1.444,"5min":1.571,"15min":1.5920000000000001},"min":{"1min":0.79800000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9099999999999999,"5min":2.661,"15min":2.661},"last":1.5129999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.6859999999999999,"15min":1.728},"min":{"1min":0.88400000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.012,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.635,"15min":1.6659999999999999},"min":{"1min":0.70699999999999996,"5min":0.505,"15min":0.505},"max":{"1min":2.2610000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.375}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.7709999999999999,"15min":1.7949999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.3140000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.573,"5min":1.7669999999999999,"15min":1.7989999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.109,"5min":2.657,"15min":2.657},"last":1.464}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":565248,"data_stored":228604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.042,"15min":1.022},"min":{"1min":0.63600000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1639999999999999,"15min":1.1679999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.878},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1850000000000001,"15min":1.1970000000000001},"min":{"1min":0.70199999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5469999999999999,"5min":1.722,"15min":1.722},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.76500000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.9179999999999999,"5min":2.085,"15min":2.085},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.3620000000000001,"15min":1.3720000000000001},"min":{"1min":0.748,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.75,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.748}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4330000000000001,"15min":1.4450000000000001},"min":{"1min":0.85599999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7130000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.512,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9510000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.214}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":0.91800000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7849999999999999,"5min":3.577,"15min":3.577},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.694,"15min":1.7190000000000001},"min":{"1min":1.133,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.9750000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.28}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6100000000000001,"15min":1.617},"min":{"1min":0.98299999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.121,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.546,"15min":1.54},"min":{"1min":1.1279999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.996,"5min":2.512,"15min":2.512},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.7130000000000001,"15min":1.726},"min":{"1min":1.2070000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.897,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.413},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.74,"15min":1.748},"min":{"1min":1.3220000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1989999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.337}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":573440,"data_stored":230146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.747,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.524,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.88},{"interface":"front","average":{"1min":1.077,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.69599999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.167}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.135,"15min":1.1479999999999999},"min":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4790000000000001,"5min":1.859,"15min":1.859},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.171,"15min":1.157},"min":{"1min":0.83399999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.4370000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.871,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8759999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.004,"5min":2.25,"15min":2.25},"last":1.5},{"interface":"front","average":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4199999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0899999999999999,"5min":2.746,"15min":2.746},"last":1.276}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.319,"15min":1.325},"min":{"1min":0.82399999999999995,"5min":0.627,"15min":0.627},"max":{"1min":1.7350000000000001,"5min":1.923,"15min":1.923},"last":1.119},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4530000000000001,"15min":1.458},"min":{"1min":0.93100000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.145,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.661}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.444,"15min":1.452},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.946,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.554},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5609999999999999,"15min":1.5649999999999999},"min":{"1min":0.84799999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9239999999999999,"5min":2.641,"15min":2.641},"last":1.369}]},{"osd":7,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5169999999999999,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.3769999999999998,"5min":2.548,"15min":2.548},"last":1.611},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.77100000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1989999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.8520000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":229172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.268}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.246},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.6180000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.4570000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.7370000000000001},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.8420000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":230168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.78900000000000003,"15min":0.76700000000000002},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85999999999999999,"15min":0.874},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.4339999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.97199999999999998,"15min":0.96999999999999997},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.466,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.157},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.3560000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.752,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5960000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.095},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2150000000000001,"15min":1.2190000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2729999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.4179999999999999,"15min":1.45},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.895,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.208}]},{"osd":6,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4039999999999999,"15min":1.423},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8440000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.401},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":1.482}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4650000000000001,"15min":1.4870000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.9219999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4350000000000001,"15min":1.446},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9910000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.4670000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":565248,"data_stored":228604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.175,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.125,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85899999999999999,"15min":0.85499999999999998},"min":{"1min":0.46700000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.52,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0209999999999999,"15min":1.01},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4219999999999999,"5min":1.621,"15min":1.621},"last":0.85499999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.157,"15min":1.163},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5509999999999999,"5min":1.907,"15min":1.907},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.999,"5min":1.048,"15min":1.056},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.069,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0700000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.91700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.7270000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3380000000000001,"15min":1.335},"min":{"1min":0.86299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8280000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2809999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.625,"5min":2.661,"15min":2.661},"last":1.022},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.3300000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6279999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.1000000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3580000000000001,"15min":1.361},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.337,"15min":1.333},"min":{"1min":0.98299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.704,"5min":2.105,"15min":2.105},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4139999999999999,"15min":1.4059999999999999},"min":{"1min":1.0569999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.0339999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3580000000000001,"15min":1.365},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7709999999999999,"5min":2.008,"15min":2.008},"last":1.3740000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1159168,"data_stored":818936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2909999999999999,"5min":1.403,"15min":1.403},"last":1.105},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.494,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2150000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.44,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92600000000000005,"15min":0.93400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.335,"5min":1.986,"15min":1.986},"last":1.0649999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0649999999999999,"15min":1.077},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.704,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.323},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95499999999999996,"15min":0.97399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.585,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.163,"15min":1.1699999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7849999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4159999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.228},"min":{"1min":0.747,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7270000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5860000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.288,"15min":1.2789999999999999},"min":{"1min":0.755,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.9139999999999999,"5min":2.149,"15min":2.149},"last":1.371}]},{"osd":7,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.85399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7969999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.423,"15min":1.425},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.0600000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2829999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:50.869 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:10:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.089+0000 7f0f42363640 1 Processor -- start 2023-11-24T08:10:51.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.089+0000 7f0f42363640 1 -- start start 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.090+0000 7f0f42363640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f3c144b30 0x7f0f3c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.090+0000 7f0f42363640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f3c15c410 0x7f0f3c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.090+0000 7f0f42363640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.090+0000 7f0f42363640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3c0b27b0 con 0x7f0f3c15c410 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.090+0000 7f0f42363640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f3c15bf80 con 0x7f0f3c144b30 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.090+0000 7f0f42363640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c0b1560 con 0x7f0f3c0b1de0 2023-11-24T08:10:51.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f3c15c410 0x7f0f3c0b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f3c15c410 0x7f0f3c0b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42566/0 (socket says 172.21.15.106:42566) 2023-11-24T08:10:51.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 -- 172.21.15.106:0/2736772298 learned_addr learned my addr 172.21.15.106:0/2736772298 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 -- 172.21.15.106:0/2736772298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 msgr2=0x7f0f3c0b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 --2- 172.21.15.106:0/2736772298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 -- 172.21.15.106:0/2736772298 >> v2:172.21.15.156:3300/0 conn(0x7f0f3c144b30 msgr2=0x7f0f3c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f41361640 1 --2- 172.21.15.106:0/2736772298 >> v2:172.21.15.156:3300/0 conn(0x7f0f3c144b30 0x7f0f3c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 --2- 172.21.15.106:0/2736772298 >> v2:172.21.15.156:3300/0 conn(0x7f0f3c144b30 0x7f0f3c15f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.091+0000 7f0f40b60640 1 -- 172.21.15.106:0/2736772298 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c0b0fb0 con 0x7f0f3c15c410 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.092+0000 7f0f40b60640 1 --2- 172.21.15.106:0/2736772298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c15c410 0x7f0f3c0b0a70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c06f690 tx=0x7f0f2c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f8814a2202508e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:51.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f33fff640 1 -- 172.21.15.106:0/2736772298 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c031870 con 0x7f0f3c15c410 2023-11-24T08:10:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f33fff640 1 -- 172.21.15.106:0/2736772298 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c0319d0 con 0x7f0f3c15c410 2023-11-24T08:10:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f33fff640 1 -- 172.21.15.106:0/2736772298 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c031d60 con 0x7f0f3c15c410 2023-11-24T08:10:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 -- 172.21.15.106:0/2736772298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c15c410 msgr2=0x7f0f3c0b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 --2- 172.21.15.106:0/2736772298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c15c410 0x7f0f3c0b0a70 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c06f690 tx=0x7f0f2c02b910 comp rx=0 tx=0).stop 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 -- 172.21.15.106:0/2736772298 shutdown_connections 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 --2- 172.21.15.106:0/2736772298 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c0b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 --2- 172.21.15.106:0/2736772298 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c15c410 0x7f0f3c0b0a70 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 --2- 172.21.15.106:0/2736772298 >> v2:172.21.15.156:3300/0 conn(0x7f0f3c144b30 0x7f0f3c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 -- 172.21.15.106:0/2736772298 >> 172.21.15.106:0/2736772298 conn(0x7f0f3c0cb830 msgr2=0x7f0f3c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 -- 172.21.15.106:0/2736772298 shutdown_connections 2023-11-24T08:10:51.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.093+0000 7f0f42363640 1 -- 172.21.15.106:0/2736772298 wait complete. 2023-11-24T08:10:51.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.095+0000 7f0f42363640 1 Processor -- start 2023-11-24T08:10:51.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.095+0000 7f0f42363640 1 -- start start 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f42363640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c139b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f42363640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 0x7f0f3c142cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f42363640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c15c410 0x7f0f3c15a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f42363640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c13b700 con 0x7f0f3c144b30 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f42363640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c13b9e0 con 0x7f0f3c15c410 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f42363640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f3c13bcc0 con 0x7f0f3c0b1de0 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.096+0000 7f0f40b60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 0x7f0f3c142cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 0x7f0f3c142cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42580/0 (socket says 172.21.15.106:42580) 2023-11-24T08:10:51.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 -- 172.21.15.106:0/3062863548 learned_addr learned my addr 172.21.15.106:0/3062863548 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f41361640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c139b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 -- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 msgr2=0x7f0f3c139b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f41b62640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c15c410 0x7f0f3c15a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c139b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 -- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c15c410 msgr2=0x7f0f3c15a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c15c410 0x7f0f3c15a1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 -- 172.21.15.106:0/3062863548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f2c0094d0 con 0x7f0f3c144b30 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f41361640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.097+0000 7f0f40b60640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 0x7f0f3c142cf0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0090b0 tx=0x7f0f2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:51.100 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.098+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c03add0 con 0x7f0f3c144b30 2023-11-24T08:10:51.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.098+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2c0443f0 con 0x7f0f3c144b30 2023-11-24T08:10:51.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.098+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2c0447a0 con 0x7f0f3c144b30 2023-11-24T08:10:51.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.098+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f3c13bfa0 con 0x7f0f3c144b30 2023-11-24T08:10:51.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.098+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f3c13a050 con 0x7f0f3c144b30 2023-11-24T08:10:51.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.099+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f3c13a240 con 0x7f0f3c144b30 2023-11-24T08:10:51.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.100+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f0f2c04d450 con 0x7f0f3c144b30 2023-11-24T08:10:51.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.101+0000 7f0f31ffb640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f2006aed0 0x7f0f2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.101+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f2c02d040 con 0x7f0f3c144b30 2023-11-24T08:10:51.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.101+0000 7f0f41361640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f2006aed0 0x7f0f2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.102+0000 7f0f41361640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f2006aed0 0x7f0f2006d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0f38008890 tx=0x7f0f38008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:51.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.105+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f2c076d80 con 0x7f0f3c144b30 2023-11-24T08:10:51.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.294+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f3c15bc10 con 0x7f0f2006aed0 2023-11-24T08:10:51.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.305+0000 7f0f31ffb640 1 -- 172.21.15.106:0/3062863548 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287264 (secure 0 0 0) 0x7f0f3800c040 con 0x7f0f2006aed0 2023-11-24T08:10:51.308 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:51.312 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:10:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f2006aed0 msgr2=0x7f0f2006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f2006aed0 0x7f0f2006d390 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f0f38008890 tx=0x7f0f38008040 comp rx=0 tx=0).stop 2023-11-24T08:10:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 msgr2=0x7f0f3c142cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 0x7f0f3c142cf0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0090b0 tx=0x7f0f2c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 shutdown_connections 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f3c15c410 0x7f0f3c15a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f3c144b30 0x7f0f3c142cf0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f2006aed0 0x7f0f2006d390 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 --2- 172.21.15.106:0/3062863548 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f3c0b1de0 0x7f0f3c139b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 >> 172.21.15.106:0/3062863548 conn(0x7f0f3c0cb830 msgr2=0x7f0f3c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 shutdown_connections 2023-11-24T08:10:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.312+0000 7f0f42363640 1 -- 172.21.15.106:0/3062863548 wait complete. 2023-11-24T08:10:51.333 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2023-11-24T08:10:49.309572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":238412,"kb_used_data":5608,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754736308,"statfs":{"total":773094113280,"available":772849979392,"internally_reserved":0,"allocated":5742592,"data_stored":3013828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003067"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.413242+0000","last_change":"2023-11-24T08:10:41.423631+0000","last_active":"2023-11-24T08:10:42.413242+0000","last_peered":"2023-11-24T08:10:42.413242+0000","last_clean":"2023-11-24T08:10:42.413242+0000","last_became_active":"2023-11-24T08:10:41.423175+0000","last_became_peered":"2023-11-24T08:10:41.423175+0000","last_unstale":"2023-11-24T08:10:42.413242+0000","last_undegraded":"2023-11-24T08:10:42.413242+0000","last_fullsized":"2023-11-24T08:10:42.413242+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:55.816463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579726+0000","last_change":"2023-11-24T08:10:41.422931+0000","last_active":"2023-11-24T08:10:43.579726+0000","last_peered":"2023-11-24T08:10:43.579726+0000","last_clean":"2023-11-24T08:10:43.579726+0000","last_became_active":"2023-11-24T08:10:41.421975+0000","last_became_peered":"2023-11-24T08:10:41.421975+0000","last_unstale":"2023-11-24T08:10:43.579726+0000","last_undegraded":"2023-11-24T08:10:43.579726+0000","last_fullsized":"2023-11-24T08:10:43.579726+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:54:40.576676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641332+0000","last_change":"2023-11-24T08:10:41.428763+0000","last_active":"2023-11-24T08:10:42.641332+0000","last_peered":"2023-11-24T08:10:42.641332+0000","last_clean":"2023-11-24T08:10:42.641332+0000","last_became_active":"2023-11-24T08:10:41.428385+0000","last_became_peered":"2023-11-24T08:10:41.428385+0000","last_unstale":"2023-11-24T08:10:42.641332+0000","last_undegraded":"2023-11-24T08:10:42.641332+0000","last_fullsized":"2023-11-24T08:10:42.641332+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:36:31.790671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460447+0000","last_change":"2023-11-24T08:10:41.423533+0000","last_active":"2023-11-24T08:10:43.460447+0000","last_peered":"2023-11-24T08:10:43.460447+0000","last_clean":"2023-11-24T08:10:43.460447+0000","last_became_active":"2023-11-24T08:10:41.421574+0000","last_became_peered":"2023-11-24T08:10:41.421574+0000","last_unstale":"2023-11-24T08:10:43.460447+0000","last_undegraded":"2023-11-24T08:10:43.460447+0000","last_fullsized":"2023-11-24T08:10:43.460447+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:34.014751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579011+0000","last_change":"2023-11-24T08:10:41.413638+0000","last_active":"2023-11-24T08:10:43.579011+0000","last_peered":"2023-11-24T08:10:43.579011+0000","last_clean":"2023-11-24T08:10:43.579011+0000","last_became_active":"2023-11-24T08:10:41.412839+0000","last_became_peered":"2023-11-24T08:10:41.412839+0000","last_unstale":"2023-11-24T08:10:43.579011+0000","last_undegraded":"2023-11-24T08:10:43.579011+0000","last_fullsized":"2023-11-24T08:10:43.579011+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:28:41.556777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408829+0000","last_change":"2023-11-24T08:10:41.420154+0000","last_active":"2023-11-24T08:10:42.408829+0000","last_peered":"2023-11-24T08:10:42.408829+0000","last_clean":"2023-11-24T08:10:42.408829+0000","last_became_active":"2023-11-24T08:10:41.419937+0000","last_became_peered":"2023-11-24T08:10:41.419937+0000","last_unstale":"2023-11-24T08:10:42.408829+0000","last_undegraded":"2023-11-24T08:10:42.408829+0000","last_fullsized":"2023-11-24T08:10:42.408829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:27.391759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412418+0000","last_change":"2023-11-24T08:10:41.426274+0000","last_active":"2023-11-24T08:10:42.412418+0000","last_peered":"2023-11-24T08:10:42.412418+0000","last_clean":"2023-11-24T08:10:42.412418+0000","last_became_active":"2023-11-24T08:10:41.426014+0000","last_became_peered":"2023-11-24T08:10:41.426014+0000","last_unstale":"2023-11-24T08:10:42.412418+0000","last_undegraded":"2023-11-24T08:10:42.412418+0000","last_fullsized":"2023-11-24T08:10:42.412418+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:08:34.014751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.459931+0000","last_change":"2023-11-24T08:10:41.426539+0000","last_active":"2023-11-24T08:10:43.459931+0000","last_peered":"2023-11-24T08:10:43.459931+0000","last_clean":"2023-11-24T08:10:43.459931+0000","last_became_active":"2023-11-24T08:10:41.425609+0000","last_became_peered":"2023-11-24T08:10:41.425609+0000","last_unstale":"2023-11-24T08:10:43.459931+0000","last_undegraded":"2023-11-24T08:10:43.459931+0000","last_fullsized":"2023-11-24T08:10:43.459931+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:43.691758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460780+0000","last_change":"2023-11-24T08:10:41.412301+0000","last_active":"2023-11-24T08:10:43.460780+0000","last_peered":"2023-11-24T08:10:43.460780+0000","last_clean":"2023-11-24T08:10:43.460780+0000","last_became_active":"2023-11-24T08:10:41.411937+0000","last_became_peered":"2023-11-24T08:10:41.411937+0000","last_unstale":"2023-11-24T08:10:43.460780+0000","last_undegraded":"2023-11-24T08:10:43.460780+0000","last_fullsized":"2023-11-24T08:10:43.460780+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:55.816463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460823+0000","last_change":"2023-11-24T08:10:41.421738+0000","last_active":"2023-11-24T08:10:43.460823+0000","last_peered":"2023-11-24T08:10:43.460823+0000","last_clean":"2023-11-24T08:10:43.460823+0000","last_became_active":"2023-11-24T08:10:41.421430+0000","last_became_peered":"2023-11-24T08:10:41.421430+0000","last_unstale":"2023-11-24T08:10:43.460823+0000","last_undegraded":"2023-11-24T08:10:43.460823+0000","last_fullsized":"2023-11-24T08:10:43.460823+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:11:18.836423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460593+0000","last_change":"2023-11-24T08:10:41.422139+0000","last_active":"2023-11-24T08:10:43.460593+0000","last_peered":"2023-11-24T08:10:43.460593+0000","last_clean":"2023-11-24T08:10:43.460593+0000","last_became_active":"2023-11-24T08:10:41.421863+0000","last_became_peered":"2023-11-24T08:10:41.421863+0000","last_unstale":"2023-11-24T08:10:43.460593+0000","last_undegraded":"2023-11-24T08:10:43.460593+0000","last_fullsized":"2023-11-24T08:10:43.460593+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:50.101523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579261+0000","last_change":"2023-11-24T08:10:41.432577+0000","last_active":"2023-11-24T08:10:43.579261+0000","last_peered":"2023-11-24T08:10:43.579261+0000","last_clean":"2023-11-24T08:10:43.579261+0000","last_became_active":"2023-11-24T08:10:41.432317+0000","last_became_peered":"2023-11-24T08:10:41.432317+0000","last_unstale":"2023-11-24T08:10:43.579261+0000","last_undegraded":"2023-11-24T08:10:43.579261+0000","last_fullsized":"2023-11-24T08:10:43.579261+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:17.035500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578429+0000","last_change":"2023-11-24T08:10:41.416555+0000","last_active":"2023-11-24T08:10:43.578429+0000","last_peered":"2023-11-24T08:10:43.578429+0000","last_clean":"2023-11-24T08:10:43.578429+0000","last_became_active":"2023-11-24T08:10:41.416334+0000","last_became_peered":"2023-11-24T08:10:41.416334+0000","last_unstale":"2023-11-24T08:10:43.578429+0000","last_undegraded":"2023-11-24T08:10:43.578429+0000","last_fullsized":"2023-11-24T08:10:43.578429+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:35:46.065379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460269+0000","last_change":"2023-11-24T08:10:41.423044+0000","last_active":"2023-11-24T08:10:43.460269+0000","last_peered":"2023-11-24T08:10:43.460269+0000","last_clean":"2023-11-24T08:10:43.460269+0000","last_became_active":"2023-11-24T08:10:41.422488+0000","last_became_peered":"2023-11-24T08:10:41.422488+0000","last_unstale":"2023-11-24T08:10:43.460269+0000","last_undegraded":"2023-11-24T08:10:43.460269+0000","last_fullsized":"2023-11-24T08:10:43.460269+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:16.545838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408585+0000","last_change":"2023-11-24T08:10:41.410547+0000","last_active":"2023-11-24T08:10:42.408585+0000","last_peered":"2023-11-24T08:10:42.408585+0000","last_clean":"2023-11-24T08:10:42.408585+0000","last_became_active":"2023-11-24T08:10:41.410256+0000","last_became_peered":"2023-11-24T08:10:41.410256+0000","last_unstale":"2023-11-24T08:10:42.408585+0000","last_undegraded":"2023-11-24T08:10:42.408585+0000","last_fullsized":"2023-11-24T08:10:42.408585+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:37.188403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409950+0000","last_change":"2023-11-24T08:10:41.433098+0000","last_active":"2023-11-24T08:10:42.409950+0000","last_peered":"2023-11-24T08:10:42.409950+0000","last_clean":"2023-11-24T08:10:42.409950+0000","last_became_active":"2023-11-24T08:10:41.432814+0000","last_became_peered":"2023-11-24T08:10:41.432814+0000","last_unstale":"2023-11-24T08:10:42.409950+0000","last_undegraded":"2023-11-24T08:10:42.409950+0000","last_fullsized":"2023-11-24T08:10:42.409950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:40:30.553777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641911+0000","last_change":"2023-11-24T08:10:41.418337+0000","last_active":"2023-11-24T08:10:42.641911+0000","last_peered":"2023-11-24T08:10:42.641911+0000","last_clean":"2023-11-24T08:10:42.641911+0000","last_became_active":"2023-11-24T08:10:41.416490+0000","last_became_peered":"2023-11-24T08:10:41.416490+0000","last_unstale":"2023-11-24T08:10:42.641911+0000","last_undegraded":"2023-11-24T08:10:42.641911+0000","last_fullsized":"2023-11-24T08:10:42.641911+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:16.545838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641675+0000","last_change":"2023-11-24T08:10:41.427781+0000","last_active":"2023-11-24T08:10:42.641675+0000","last_peered":"2023-11-24T08:10:42.641675+0000","last_clean":"2023-11-24T08:10:42.641675+0000","last_became_active":"2023-11-24T08:10:41.427474+0000","last_became_peered":"2023-11-24T08:10:41.427474+0000","last_unstale":"2023-11-24T08:10:42.641675+0000","last_undegraded":"2023-11-24T08:10:42.641675+0000","last_fullsized":"2023-11-24T08:10:42.641675+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:28:57.920963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412142+0000","last_change":"2023-11-24T08:10:41.429736+0000","last_active":"2023-11-24T08:10:42.412142+0000","last_peered":"2023-11-24T08:10:42.412142+0000","last_clean":"2023-11-24T08:10:42.412142+0000","last_became_active":"2023-11-24T08:10:41.428905+0000","last_became_peered":"2023-11-24T08:10:41.428905+0000","last_unstale":"2023-11-24T08:10:42.412142+0000","last_undegraded":"2023-11-24T08:10:42.412142+0000","last_fullsized":"2023-11-24T08:10:42.412142+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:35.587896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578477+0000","last_change":"2023-11-24T08:10:41.428988+0000","last_active":"2023-11-24T08:10:43.578477+0000","last_peered":"2023-11-24T08:10:43.578477+0000","last_clean":"2023-11-24T08:10:43.578477+0000","last_became_active":"2023-11-24T08:10:41.428498+0000","last_became_peered":"2023-11-24T08:10:41.428498+0000","last_unstale":"2023-11-24T08:10:43.578477+0000","last_undegraded":"2023-11-24T08:10:43.578477+0000","last_fullsized":"2023-11-24T08:10:43.578477+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:27:46.805190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409473+0000","last_change":"2023-11-24T08:10:41.411951+0000","last_active":"2023-11-24T08:10:42.409473+0000","last_peered":"2023-11-24T08:10:42.409473+0000","last_clean":"2023-11-24T08:10:42.409473+0000","last_became_active":"2023-11-24T08:10:41.411683+0000","last_became_peered":"2023-11-24T08:10:41.411683+0000","last_unstale":"2023-11-24T08:10:42.409473+0000","last_undegraded":"2023-11-24T08:10:42.409473+0000","last_fullsized":"2023-11-24T08:10:42.409473+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:16.278290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460241+0000","last_change":"2023-11-24T08:10:41.421911+0000","last_active":"2023-11-24T08:10:43.460241+0000","last_peered":"2023-11-24T08:10:43.460241+0000","last_clean":"2023-11-24T08:10:43.460241+0000","last_became_active":"2023-11-24T08:10:41.421454+0000","last_became_peered":"2023-11-24T08:10:41.421454+0000","last_unstale":"2023-11-24T08:10:43.460241+0000","last_undegraded":"2023-11-24T08:10:43.460241+0000","last_fullsized":"2023-11-24T08:10:43.460241+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:28:57.920963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410258+0000","last_change":"2023-11-24T08:10:41.417943+0000","last_active":"2023-11-24T08:10:42.410258+0000","last_peered":"2023-11-24T08:10:42.410258+0000","last_clean":"2023-11-24T08:10:42.410258+0000","last_became_active":"2023-11-24T08:10:41.416617+0000","last_became_peered":"2023-11-24T08:10:41.416617+0000","last_unstale":"2023-11-24T08:10:42.410258+0000","last_undegraded":"2023-11-24T08:10:42.410258+0000","last_fullsized":"2023-11-24T08:10:42.410258+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:08.645484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410783+0000","last_change":"2023-11-24T08:10:41.421027+0000","last_active":"2023-11-24T08:10:42.410783+0000","last_peered":"2023-11-24T08:10:42.410783+0000","last_clean":"2023-11-24T08:10:42.410783+0000","last_became_active":"2023-11-24T08:10:41.420532+0000","last_became_peered":"2023-11-24T08:10:41.420532+0000","last_unstale":"2023-11-24T08:10:42.410783+0000","last_undegraded":"2023-11-24T08:10:42.410783+0000","last_fullsized":"2023-11-24T08:10:42.410783+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:29:49.565301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.411729+0000","last_change":"2023-11-24T08:10:41.410624+0000","last_active":"2023-11-24T08:10:42.411729+0000","last_peered":"2023-11-24T08:10:42.411729+0000","last_clean":"2023-11-24T08:10:42.411729+0000","last_became_active":"2023-11-24T08:10:41.409598+0000","last_became_peered":"2023-11-24T08:10:41.409598+0000","last_unstale":"2023-11-24T08:10:42.411729+0000","last_undegraded":"2023-11-24T08:10:42.411729+0000","last_fullsized":"2023-11-24T08:10:42.411729+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:54.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578541+0000","last_change":"2023-11-24T08:10:41.417578+0000","last_active":"2023-11-24T08:10:43.578541+0000","last_peered":"2023-11-24T08:10:43.578541+0000","last_clean":"2023-11-24T08:10:43.578541+0000","last_became_active":"2023-11-24T08:10:41.416835+0000","last_became_peered":"2023-11-24T08:10:41.416835+0000","last_unstale":"2023-11-24T08:10:43.578541+0000","last_undegraded":"2023-11-24T08:10:43.578541+0000","last_fullsized":"2023-11-24T08:10:43.578541+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:50.461147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641650+0000","last_change":"2023-11-24T08:10:41.414079+0000","last_active":"2023-11-24T08:10:42.641650+0000","last_peered":"2023-11-24T08:10:42.641650+0000","last_clean":"2023-11-24T08:10:42.641650+0000","last_became_active":"2023-11-24T08:10:41.413805+0000","last_became_peered":"2023-11-24T08:10:41.413805+0000","last_unstale":"2023-11-24T08:10:42.641650+0000","last_undegraded":"2023-11-24T08:10:42.641650+0000","last_fullsized":"2023-11-24T08:10:42.641650+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:01.261020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579474+0000","last_change":"2023-11-24T08:10:41.432464+0000","last_active":"2023-11-24T08:10:43.579474+0000","last_peered":"2023-11-24T08:10:43.579474+0000","last_clean":"2023-11-24T08:10:43.579474+0000","last_became_active":"2023-11-24T08:10:41.432206+0000","last_became_peered":"2023-11-24T08:10:41.432206+0000","last_unstale":"2023-11-24T08:10:43.579474+0000","last_undegraded":"2023-11-24T08:10:43.579474+0000","last_fullsized":"2023-11-24T08:10:43.579474+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:50:11.818427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410442+0000","last_change":"2023-11-24T08:10:41.411330+0000","last_active":"2023-11-24T08:10:42.410442+0000","last_peered":"2023-11-24T08:10:42.410442+0000","last_clean":"2023-11-24T08:10:42.410442+0000","last_became_active":"2023-11-24T08:10:41.406404+0000","last_became_peered":"2023-11-24T08:10:41.406404+0000","last_unstale":"2023-11-24T08:10:42.410442+0000","last_undegraded":"2023-11-24T08:10:42.410442+0000","last_fullsized":"2023-11-24T08:10:42.410442+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:32.711956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641144+0000","last_change":"2023-11-24T08:10:41.415700+0000","last_active":"2023-11-24T08:10:42.641144+0000","last_peered":"2023-11-24T08:10:42.641144+0000","last_clean":"2023-11-24T08:10:42.641144+0000","last_became_active":"2023-11-24T08:10:41.415422+0000","last_became_peered":"2023-11-24T08:10:41.415422+0000","last_unstale":"2023-11-24T08:10:42.641144+0000","last_undegraded":"2023-11-24T08:10:42.641144+0000","last_fullsized":"2023-11-24T08:10:42.641144+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:02:55.962668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408844+0000","last_change":"2023-11-24T08:10:41.413932+0000","last_active":"2023-11-24T08:10:42.408844+0000","last_peered":"2023-11-24T08:10:42.408844+0000","last_clean":"2023-11-24T08:10:42.408844+0000","last_became_active":"2023-11-24T08:10:41.413612+0000","last_became_peered":"2023-11-24T08:10:41.413612+0000","last_unstale":"2023-11-24T08:10:42.408844+0000","last_undegraded":"2023-11-24T08:10:42.408844+0000","last_fullsized":"2023-11-24T08:10:42.408844+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:09:35.104083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578110+0000","last_change":"2023-11-24T08:10:41.416554+0000","last_active":"2023-11-24T08:10:43.578110+0000","last_peered":"2023-11-24T08:10:43.578110+0000","last_clean":"2023-11-24T08:10:43.578110+0000","last_became_active":"2023-11-24T08:10:41.416251+0000","last_became_peered":"2023-11-24T08:10:41.416251+0000","last_unstale":"2023-11-24T08:10:43.578110+0000","last_undegraded":"2023-11-24T08:10:43.578110+0000","last_fullsized":"2023-11-24T08:10:43.578110+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:47:23.791573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.551681+0000","last_change":"2023-11-24T08:10:41.412005+0000","last_active":"2023-11-24T08:10:43.551681+0000","last_peered":"2023-11-24T08:10:43.551681+0000","last_clean":"2023-11-24T08:10:43.551681+0000","last_became_active":"2023-11-24T08:10:41.411675+0000","last_became_peered":"2023-11-24T08:10:41.411675+0000","last_unstale":"2023-11-24T08:10:43.551681+0000","last_undegraded":"2023-11-24T08:10:43.551681+0000","last_fullsized":"2023-11-24T08:10:43.551681+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:42.301586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"47'4","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.425101+0000","last_change":"2023-11-24T08:10:40.393255+0000","last_active":"2023-11-24T08:10:41.425101+0000","last_peered":"2023-11-24T08:10:41.425101+0000","last_clean":"2023-11-24T08:10:41.425101+0000","last_became_active":"2023-11-24T08:10:40.392572+0000","last_became_peered":"2023-11-24T08:10:40.392572+0000","last_unstale":"2023-11-24T08:10:41.425101+0000","last_undegraded":"2023-11-24T08:10:41.425101+0000","last_fullsized":"2023-11-24T08:10:41.425101+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:49.087931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412088+0000","last_change":"2023-11-24T08:10:41.409867+0000","last_active":"2023-11-24T08:10:42.412088+0000","last_peered":"2023-11-24T08:10:42.412088+0000","last_clean":"2023-11-24T08:10:42.412088+0000","last_became_active":"2023-11-24T08:10:41.409513+0000","last_became_peered":"2023-11-24T08:10:41.409513+0000","last_unstale":"2023-11-24T08:10:42.412088+0000","last_undegraded":"2023-11-24T08:10:42.412088+0000","last_fullsized":"2023-11-24T08:10:42.412088+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:39.663253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.420757+0000","last_change":"2023-11-24T08:10:40.392959+0000","last_active":"2023-11-24T08:10:41.420757+0000","last_peered":"2023-11-24T08:10:41.420757+0000","last_clean":"2023-11-24T08:10:41.420757+0000","last_became_active":"2023-11-24T08:10:40.392620+0000","last_became_peered":"2023-11-24T08:10:40.392620+0000","last_unstale":"2023-11-24T08:10:41.420757+0000","last_undegraded":"2023-11-24T08:10:41.420757+0000","last_fullsized":"2023-11-24T08:10:41.420757+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:41.287994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.579554+0000","last_change":"2023-11-24T08:10:41.413632+0000","last_active":"2023-11-24T08:10:43.579554+0000","last_peered":"2023-11-24T08:10:43.579554+0000","last_clean":"2023-11-24T08:10:43.579554+0000","last_became_active":"2023-11-24T08:10:41.413348+0000","last_became_peered":"2023-11-24T08:10:41.413348+0000","last_unstale":"2023-11-24T08:10:43.579554+0000","last_undegraded":"2023-11-24T08:10:43.579554+0000","last_fullsized":"2023-11-24T08:10:43.579554+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:25:25.940595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.413125+0000","last_change":"2023-11-24T08:10:40.391717+0000","last_active":"2023-11-24T08:10:41.413125+0000","last_peered":"2023-11-24T08:10:41.413125+0000","last_clean":"2023-11-24T08:10:41.413125+0000","last_became_active":"2023-11-24T08:10:40.391477+0000","last_became_peered":"2023-11-24T08:10:40.391477+0000","last_unstale":"2023-11-24T08:10:41.413125+0000","last_undegraded":"2023-11-24T08:10:41.413125+0000","last_fullsized":"2023-11-24T08:10:41.413125+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:06.464620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.552452+0000","last_change":"2023-11-24T08:10:41.410142+0000","last_active":"2023-11-24T08:10:43.552452+0000","last_peered":"2023-11-24T08:10:43.552452+0000","last_clean":"2023-11-24T08:10:43.552452+0000","last_became_active":"2023-11-24T08:10:41.409736+0000","last_became_peered":"2023-11-24T08:10:41.409736+0000","last_unstale":"2023-11-24T08:10:43.552452+0000","last_undegraded":"2023-11-24T08:10:43.552452+0000","last_fullsized":"2023-11-24T08:10:43.552452+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:26:39.663253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.423598+0000","last_change":"2023-11-24T08:10:40.400930+0000","last_active":"2023-11-24T08:10:41.423598+0000","last_peered":"2023-11-24T08:10:41.423598+0000","last_clean":"2023-11-24T08:10:41.423598+0000","last_became_active":"2023-11-24T08:10:40.400220+0000","last_became_peered":"2023-11-24T08:10:40.400220+0000","last_unstale":"2023-11-24T08:10:41.423598+0000","last_undegraded":"2023-11-24T08:10:41.423598+0000","last_fullsized":"2023-11-24T08:10:41.423598+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:50:19.777593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641163+0000","last_change":"2023-11-24T08:10:41.416316+0000","last_active":"2023-11-24T08:10:42.641163+0000","last_peered":"2023-11-24T08:10:42.641163+0000","last_clean":"2023-11-24T08:10:42.641163+0000","last_became_active":"2023-11-24T08:10:41.416038+0000","last_became_peered":"2023-11-24T08:10:41.416038+0000","last_unstale":"2023-11-24T08:10:42.641163+0000","last_undegraded":"2023-11-24T08:10:42.641163+0000","last_fullsized":"2023-11-24T08:10:42.641163+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:30:43.691758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417158+0000","last_change":"2023-11-24T08:10:40.394178+0000","last_active":"2023-11-24T08:10:41.417158+0000","last_peered":"2023-11-24T08:10:41.417158+0000","last_clean":"2023-11-24T08:10:41.417158+0000","last_became_active":"2023-11-24T08:10:40.393917+0000","last_became_peered":"2023-11-24T08:10:40.393917+0000","last_unstale":"2023-11-24T08:10:41.417158+0000","last_undegraded":"2023-11-24T08:10:41.417158+0000","last_fullsized":"2023-11-24T08:10:41.417158+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:29:54.802870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460437+0000","last_change":"2023-11-24T08:10:41.429543+0000","last_active":"2023-11-24T08:10:43.460437+0000","last_peered":"2023-11-24T08:10:43.460437+0000","last_clean":"2023-11-24T08:10:43.460437+0000","last_became_active":"2023-11-24T08:10:41.429272+0000","last_became_peered":"2023-11-24T08:10:41.429272+0000","last_unstale":"2023-11-24T08:10:43.460437+0000","last_undegraded":"2023-11-24T08:10:43.460437+0000","last_fullsized":"2023-11-24T08:10:43.460437+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:28.940501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.414400+0000","last_change":"2023-11-24T08:10:40.390840+0000","last_active":"2023-11-24T08:10:41.414400+0000","last_peered":"2023-11-24T08:10:41.414400+0000","last_clean":"2023-11-24T08:10:41.414400+0000","last_became_active":"2023-11-24T08:10:40.390315+0000","last_became_peered":"2023-11-24T08:10:40.390315+0000","last_unstale":"2023-11-24T08:10:41.414400+0000","last_undegraded":"2023-11-24T08:10:41.414400+0000","last_fullsized":"2023-11-24T08:10:41.414400+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:06.464620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460175+0000","last_change":"2023-11-24T08:10:41.434664+0000","last_active":"2023-11-24T08:10:43.460175+0000","last_peered":"2023-11-24T08:10:43.460175+0000","last_clean":"2023-11-24T08:10:43.460175+0000","last_became_active":"2023-11-24T08:10:41.426919+0000","last_became_peered":"2023-11-24T08:10:41.426919+0000","last_unstale":"2023-11-24T08:10:43.460175+0000","last_undegraded":"2023-11-24T08:10:43.460175+0000","last_fullsized":"2023-11-24T08:10:43.460175+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:01.261020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.396243+0000","last_change":"2023-11-24T08:10:40.390774+0000","last_active":"2023-11-24T08:10:41.396243+0000","last_peered":"2023-11-24T08:10:41.396243+0000","last_clean":"2023-11-24T08:10:41.396243+0000","last_became_active":"2023-11-24T08:10:40.390495+0000","last_became_peered":"2023-11-24T08:10:40.390495+0000","last_unstale":"2023-11-24T08:10:41.396243+0000","last_undegraded":"2023-11-24T08:10:41.396243+0000","last_fullsized":"2023-11-24T08:10:41.396243+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:50:14.655179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409909+0000","last_change":"2023-11-24T08:10:41.418194+0000","last_active":"2023-11-24T08:10:42.409909+0000","last_peered":"2023-11-24T08:10:42.409909+0000","last_clean":"2023-11-24T08:10:42.409909+0000","last_became_active":"2023-11-24T08:10:41.416986+0000","last_became_peered":"2023-11-24T08:10:41.416986+0000","last_unstale":"2023-11-24T08:10:42.409909+0000","last_undegraded":"2023-11-24T08:10:42.409909+0000","last_fullsized":"2023-11-24T08:10:42.409909+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:23.175828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.411720+0000","last_change":"2023-11-24T08:10:40.393357+0000","last_active":"2023-11-24T08:10:41.411720+0000","last_peered":"2023-11-24T08:10:41.411720+0000","last_clean":"2023-11-24T08:10:41.411720+0000","last_became_active":"2023-11-24T08:10:40.391265+0000","last_became_peered":"2023-11-24T08:10:40.391265+0000","last_unstale":"2023-11-24T08:10:41.411720+0000","last_undegraded":"2023-11-24T08:10:41.411720+0000","last_fullsized":"2023-11-24T08:10:41.411720+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:18.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641924+0000","last_change":"2023-11-24T08:10:41.427725+0000","last_active":"2023-11-24T08:10:42.641924+0000","last_peered":"2023-11-24T08:10:42.641924+0000","last_clean":"2023-11-24T08:10:42.641924+0000","last_became_active":"2023-11-24T08:10:41.427451+0000","last_became_peered":"2023-11-24T08:10:41.427451+0000","last_unstale":"2023-11-24T08:10:42.641924+0000","last_undegraded":"2023-11-24T08:10:42.641924+0000","last_fullsized":"2023-11-24T08:10:42.641924+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:17:28.940501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.403287+0000","last_change":"2023-11-24T08:10:40.382626+0000","last_active":"2023-11-24T08:10:41.403287+0000","last_peered":"2023-11-24T08:10:41.403287+0000","last_clean":"2023-11-24T08:10:41.403287+0000","last_became_active":"2023-11-24T08:10:40.381229+0000","last_became_peered":"2023-11-24T08:10:40.381229+0000","last_unstale":"2023-11-24T08:10:41.403287+0000","last_undegraded":"2023-11-24T08:10:41.403287+0000","last_fullsized":"2023-11-24T08:10:41.403287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:34.574303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.413359+0000","last_change":"2023-11-24T08:10:41.424059+0000","last_active":"2023-11-24T08:10:42.413359+0000","last_peered":"2023-11-24T08:10:42.413359+0000","last_clean":"2023-11-24T08:10:42.413359+0000","last_became_active":"2023-11-24T08:10:41.423565+0000","last_became_peered":"2023-11-24T08:10:41.423565+0000","last_unstale":"2023-11-24T08:10:42.413359+0000","last_undegraded":"2023-11-24T08:10:42.413359+0000","last_fullsized":"2023-11-24T08:10:42.413359+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:11:18.836423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.398130+0000","last_change":"2023-11-24T08:10:40.396039+0000","last_active":"2023-11-24T08:10:41.398130+0000","last_peered":"2023-11-24T08:10:41.398130+0000","last_clean":"2023-11-24T08:10:41.398130+0000","last_became_active":"2023-11-24T08:10:40.395680+0000","last_became_peered":"2023-11-24T08:10:40.395680+0000","last_unstale":"2023-11-24T08:10:41.398130+0000","last_undegraded":"2023-11-24T08:10:41.398130+0000","last_fullsized":"2023-11-24T08:10:41.398130+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:31.698363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409720+0000","last_change":"2023-11-24T08:10:41.432591+0000","last_active":"2023-11-24T08:10:42.409720+0000","last_peered":"2023-11-24T08:10:42.409720+0000","last_clean":"2023-11-24T08:10:42.409720+0000","last_became_active":"2023-11-24T08:10:41.432305+0000","last_became_peered":"2023-11-24T08:10:41.432305+0000","last_unstale":"2023-11-24T08:10:42.409720+0000","last_undegraded":"2023-11-24T08:10:42.409720+0000","last_fullsized":"2023-11-24T08:10:42.409720+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:16:28.486289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"47'3","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.430812+0000","last_change":"2023-11-24T08:10:40.400513+0000","last_active":"2023-11-24T08:10:41.430812+0000","last_peered":"2023-11-24T08:10:41.430812+0000","last_clean":"2023-11-24T08:10:41.430812+0000","last_became_active":"2023-11-24T08:10:40.399876+0000","last_became_peered":"2023-11-24T08:10:40.399876+0000","last_unstale":"2023-11-24T08:10:41.430812+0000","last_undegraded":"2023-11-24T08:10:41.430812+0000","last_fullsized":"2023-11-24T08:10:41.430812+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:28:56.907370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412803+0000","last_change":"2023-11-24T08:10:41.427142+0000","last_active":"2023-11-24T08:10:42.412803+0000","last_peered":"2023-11-24T08:10:42.412803+0000","last_clean":"2023-11-24T08:10:42.412803+0000","last_became_active":"2023-11-24T08:10:41.426847+0000","last_became_peered":"2023-11-24T08:10:41.426847+0000","last_unstale":"2023-11-24T08:10:42.412803+0000","last_undegraded":"2023-11-24T08:10:42.412803+0000","last_fullsized":"2023-11-24T08:10:42.412803+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:12.381653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"47'6","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.436940+0000","last_change":"2023-11-24T08:10:40.378220+0000","last_active":"2023-11-24T08:10:41.436940+0000","last_peered":"2023-11-24T08:10:41.436940+0000","last_clean":"2023-11-24T08:10:41.436940+0000","last_became_active":"2023-11-24T08:10:40.377915+0000","last_became_peered":"2023-11-24T08:10:40.377915+0000","last_unstale":"2023-11-24T08:10:41.436940+0000","last_undegraded":"2023-11-24T08:10:41.436940+0000","last_fullsized":"2023-11-24T08:10:41.436940+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:27:12.892493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.410905+0000","last_change":"2023-11-24T08:10:41.414931+0000","last_active":"2023-11-24T08:10:42.410905+0000","last_peered":"2023-11-24T08:10:42.410905+0000","last_clean":"2023-11-24T08:10:42.410905+0000","last_became_active":"2023-11-24T08:10:41.414621+0000","last_became_peered":"2023-11-24T08:10:41.414621+0000","last_unstale":"2023-11-24T08:10:42.410905+0000","last_undegraded":"2023-11-24T08:10:42.410905+0000","last_fullsized":"2023-11-24T08:10:42.410905+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:56:07.478213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.425599+0000","last_change":"2023-11-24T08:10:40.395756+0000","last_active":"2023-11-24T08:10:41.425599+0000","last_peered":"2023-11-24T08:10:41.425599+0000","last_clean":"2023-11-24T08:10:41.425599+0000","last_became_active":"2023-11-24T08:10:40.395493+0000","last_became_peered":"2023-11-24T08:10:40.395493+0000","last_unstale":"2023-11-24T08:10:41.425599+0000","last_undegraded":"2023-11-24T08:10:41.425599+0000","last_fullsized":"2023-11-24T08:10:41.425599+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:11.368061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460619+0000","last_change":"2023-11-24T08:10:41.422495+0000","last_active":"2023-11-24T08:10:43.460619+0000","last_peered":"2023-11-24T08:10:43.460619+0000","last_clean":"2023-11-24T08:10:43.460619+0000","last_became_active":"2023-11-24T08:10:41.422232+0000","last_became_peered":"2023-11-24T08:10:41.422232+0000","last_unstale":"2023-11-24T08:10:43.460619+0000","last_undegraded":"2023-11-24T08:10:43.460619+0000","last_fullsized":"2023-11-24T08:10:43.460619+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:42.167949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.416197+0000","last_change":"2023-11-24T08:10:40.397297+0000","last_active":"2023-11-24T08:10:41.416197+0000","last_peered":"2023-11-24T08:10:41.416197+0000","last_clean":"2023-11-24T08:10:41.416197+0000","last_became_active":"2023-11-24T08:10:40.397019+0000","last_became_peered":"2023-11-24T08:10:40.397019+0000","last_unstale":"2023-11-24T08:10:41.416197+0000","last_undegraded":"2023-11-24T08:10:41.416197+0000","last_fullsized":"2023-11-24T08:10:41.416197+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:18.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408655+0000","last_change":"2023-11-24T08:10:41.429888+0000","last_active":"2023-11-24T08:10:42.408655+0000","last_peered":"2023-11-24T08:10:42.408655+0000","last_clean":"2023-11-24T08:10:42.408655+0000","last_became_active":"2023-11-24T08:10:41.429609+0000","last_became_peered":"2023-11-24T08:10:41.429609+0000","last_unstale":"2023-11-24T08:10:42.408655+0000","last_undegraded":"2023-11-24T08:10:42.408655+0000","last_fullsized":"2023-11-24T08:10:42.408655+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:51:20.855433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.403023+0000","last_change":"2023-11-24T08:10:40.396293+0000","last_active":"2023-11-24T08:10:41.403023+0000","last_peered":"2023-11-24T08:10:41.403023+0000","last_clean":"2023-11-24T08:10:41.403023+0000","last_became_active":"2023-11-24T08:10:40.393238+0000","last_became_peered":"2023-11-24T08:10:40.393238+0000","last_unstale":"2023-11-24T08:10:41.403023+0000","last_undegraded":"2023-11-24T08:10:41.403023+0000","last_fullsized":"2023-11-24T08:10:41.403023+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:51.265083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412759+0000","last_change":"2023-11-24T08:10:41.422966+0000","last_active":"2023-11-24T08:10:42.412759+0000","last_peered":"2023-11-24T08:10:42.412759+0000","last_clean":"2023-11-24T08:10:42.412759+0000","last_became_active":"2023-11-24T08:10:41.422642+0000","last_became_peered":"2023-11-24T08:10:41.422642+0000","last_unstale":"2023-11-24T08:10:42.412759+0000","last_undegraded":"2023-11-24T08:10:42.412759+0000","last_fullsized":"2023-11-24T08:10:42.412759+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:04:50.101523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"47'8","reported_seq":20,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.437314+0000","last_change":"2023-11-24T08:10:40.391642+0000","last_active":"2023-11-24T08:10:41.437314+0000","last_peered":"2023-11-24T08:10:41.437314+0000","last_clean":"2023-11-24T08:10:41.437314+0000","last_became_active":"2023-11-24T08:10:40.389315+0000","last_became_peered":"2023-11-24T08:10:40.389315+0000","last_unstale":"2023-11-24T08:10:41.437314+0000","last_undegraded":"2023-11-24T08:10:41.437314+0000","last_fullsized":"2023-11-24T08:10:41.437314+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:50:19.777593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578735+0000","last_change":"2023-11-24T08:10:41.411050+0000","last_active":"2023-11-24T08:10:43.578735+0000","last_peered":"2023-11-24T08:10:43.578735+0000","last_clean":"2023-11-24T08:10:43.578735+0000","last_became_active":"2023-11-24T08:10:41.410799+0000","last_became_peered":"2023-11-24T08:10:41.410799+0000","last_unstale":"2023-11-24T08:10:43.578735+0000","last_undegraded":"2023-11-24T08:10:43.578735+0000","last_fullsized":"2023-11-24T08:10:43.578735+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:58:21.722380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417836+0000","last_change":"2023-11-24T08:10:40.387475+0000","last_active":"2023-11-24T08:10:41.417836+0000","last_peered":"2023-11-24T08:10:41.417836+0000","last_clean":"2023-11-24T08:10:41.417836+0000","last_became_active":"2023-11-24T08:10:40.386342+0000","last_became_peered":"2023-11-24T08:10:40.386342+0000","last_unstale":"2023-11-24T08:10:41.417836+0000","last_undegraded":"2023-11-24T08:10:41.417836+0000","last_fullsized":"2023-11-24T08:10:41.417836+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:50:15.430616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460933+0000","last_change":"2023-11-24T08:10:41.423141+0000","last_active":"2023-11-24T08:10:43.460933+0000","last_peered":"2023-11-24T08:10:43.460933+0000","last_clean":"2023-11-24T08:10:43.460933+0000","last_became_active":"2023-11-24T08:10:41.422861+0000","last_became_peered":"2023-11-24T08:10:41.422861+0000","last_unstale":"2023-11-24T08:10:43.460933+0000","last_undegraded":"2023-11-24T08:10:43.460933+0000","last_fullsized":"2023-11-24T08:10:43.460933+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:17:37.188403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.418770+0000","last_change":"2023-11-24T08:10:40.382415+0000","last_active":"2023-11-24T08:10:41.418770+0000","last_peered":"2023-11-24T08:10:41.418770+0000","last_clean":"2023-11-24T08:10:41.418770+0000","last_became_active":"2023-11-24T08:10:40.382090+0000","last_became_peered":"2023-11-24T08:10:40.382090+0000","last_unstale":"2023-11-24T08:10:41.418770+0000","last_undegraded":"2023-11-24T08:10:41.418770+0000","last_fullsized":"2023-11-24T08:10:41.418770+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:31:18.712376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.411535+0000","last_change":"2023-11-24T08:10:41.411739+0000","last_active":"2023-11-24T08:10:42.411535+0000","last_peered":"2023-11-24T08:10:42.411535+0000","last_clean":"2023-11-24T08:10:42.411535+0000","last_became_active":"2023-11-24T08:10:41.411478+0000","last_became_peered":"2023-11-24T08:10:41.411478+0000","last_unstale":"2023-11-24T08:10:42.411535+0000","last_undegraded":"2023-11-24T08:10:42.411535+0000","last_fullsized":"2023-11-24T08:10:42.411535+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:32:41.371090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"47'1","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.416456+0000","last_change":"2023-11-24T08:10:40.390983+0000","last_active":"2023-11-24T08:10:41.416456+0000","last_peered":"2023-11-24T08:10:41.416456+0000","last_clean":"2023-11-24T08:10:41.416456+0000","last_became_active":"2023-11-24T08:10:40.389721+0000","last_became_peered":"2023-11-24T08:10:40.389721+0000","last_unstale":"2023-11-24T08:10:41.416456+0000","last_undegraded":"2023-11-24T08:10:41.416456+0000","last_fullsized":"2023-11-24T08:10:41.416456+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:24:04.284926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.411889+0000","last_change":"2023-11-24T08:10:41.407461+0000","last_active":"2023-11-24T08:10:42.411889+0000","last_peered":"2023-11-24T08:10:42.411889+0000","last_clean":"2023-11-24T08:10:42.411889+0000","last_became_active":"2023-11-24T08:10:41.407137+0000","last_became_peered":"2023-11-24T08:10:41.407137+0000","last_unstale":"2023-11-24T08:10:42.411889+0000","last_undegraded":"2023-11-24T08:10:42.411889+0000","last_fullsized":"2023-11-24T08:10:42.411889+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:35:42.301586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.402508+0000","last_change":"2023-11-24T08:10:40.396743+0000","last_active":"2023-11-24T08:10:41.402508+0000","last_peered":"2023-11-24T08:10:41.402508+0000","last_clean":"2023-11-24T08:10:41.402508+0000","last_became_active":"2023-11-24T08:10:40.396292+0000","last_became_peered":"2023-11-24T08:10:40.396292+0000","last_unstale":"2023-11-24T08:10:41.402508+0000","last_undegraded":"2023-11-24T08:10:41.402508+0000","last_fullsized":"2023-11-24T08:10:41.402508+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:11:17.822830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.408950+0000","last_change":"2023-11-24T08:10:41.433540+0000","last_active":"2023-11-24T08:10:42.408950+0000","last_peered":"2023-11-24T08:10:42.408950+0000","last_clean":"2023-11-24T08:10:42.408950+0000","last_became_active":"2023-11-24T08:10:41.433249+0000","last_became_peered":"2023-11-24T08:10:41.433249+0000","last_unstale":"2023-11-24T08:10:42.408950+0000","last_undegraded":"2023-11-24T08:10:42.408950+0000","last_fullsized":"2023-11-24T08:10:42.408950+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:23:49.150477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417254+0000","last_change":"2023-11-24T08:10:40.394138+0000","last_active":"2023-11-24T08:10:41.417254+0000","last_peered":"2023-11-24T08:10:41.417254+0000","last_clean":"2023-11-24T08:10:41.417254+0000","last_became_active":"2023-11-24T08:10:40.393872+0000","last_became_peered":"2023-11-24T08:10:40.393872+0000","last_unstale":"2023-11-24T08:10:41.417254+0000","last_undegraded":"2023-11-24T08:10:41.417254+0000","last_fullsized":"2023-11-24T08:10:41.417254+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:41.154356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.551388+0000","last_change":"2023-11-24T08:10:41.409585+0000","last_active":"2023-11-24T08:10:43.551388+0000","last_peered":"2023-11-24T08:10:43.551388+0000","last_clean":"2023-11-24T08:10:43.551388+0000","last_became_active":"2023-11-24T08:10:41.407565+0000","last_became_peered":"2023-11-24T08:10:41.407565+0000","last_unstale":"2023-11-24T08:10:43.551388+0000","last_undegraded":"2023-11-24T08:10:43.551388+0000","last_fullsized":"2023-11-24T08:10:43.551388+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:54.209452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.392098+0000","last_change":"2023-11-24T08:10:40.393936+0000","last_active":"2023-11-24T08:10:41.392098+0000","last_peered":"2023-11-24T08:10:41.392098+0000","last_clean":"2023-11-24T08:10:41.392098+0000","last_became_active":"2023-11-24T08:10:40.393617+0000","last_became_peered":"2023-11-24T08:10:40.393617+0000","last_unstale":"2023-11-24T08:10:41.392098+0000","last_undegraded":"2023-11-24T08:10:41.392098+0000","last_fullsized":"2023-11-24T08:10:41.392098+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:06:17.056254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578132+0000","last_change":"2023-11-24T08:10:41.423108+0000","last_active":"2023-11-24T08:10:43.578132+0000","last_peered":"2023-11-24T08:10:43.578132+0000","last_clean":"2023-11-24T08:10:43.578132+0000","last_became_active":"2023-11-24T08:10:41.422850+0000","last_became_peered":"2023-11-24T08:10:41.422850+0000","last_unstale":"2023-11-24T08:10:43.578132+0000","last_undegraded":"2023-11-24T08:10:43.578132+0000","last_fullsized":"2023-11-24T08:10:43.578132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:27:48.503618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.399236+0000","last_change":"2023-11-24T08:10:40.392572+0000","last_active":"2023-11-24T08:10:41.399236+0000","last_peered":"2023-11-24T08:10:41.399236+0000","last_clean":"2023-11-24T08:10:41.399236+0000","last_became_active":"2023-11-24T08:10:40.392267+0000","last_became_peered":"2023-11-24T08:10:40.392267+0000","last_unstale":"2023-11-24T08:10:41.399236+0000","last_undegraded":"2023-11-24T08:10:41.399236+0000","last_fullsized":"2023-11-24T08:10:41.399236+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:53.195860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.459940+0000","last_change":"2023-11-24T08:10:41.413307+0000","last_active":"2023-11-24T08:10:43.459940+0000","last_peered":"2023-11-24T08:10:43.459940+0000","last_clean":"2023-11-24T08:10:43.459940+0000","last_became_active":"2023-11-24T08:10:41.413026+0000","last_became_peered":"2023-11-24T08:10:41.413026+0000","last_unstale":"2023-11-24T08:10:43.459940+0000","last_undegraded":"2023-11-24T08:10:43.459940+0000","last_fullsized":"2023-11-24T08:10:43.459940+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:36:31.790671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.415970+0000","last_change":"2023-11-24T08:10:40.397606+0000","last_active":"2023-11-24T08:10:41.415970+0000","last_peered":"2023-11-24T08:10:41.415970+0000","last_clean":"2023-11-24T08:10:41.415970+0000","last_became_active":"2023-11-24T08:10:40.397331+0000","last_became_peered":"2023-11-24T08:10:40.397331+0000","last_unstale":"2023-11-24T08:10:41.415970+0000","last_undegraded":"2023-11-24T08:10:41.415970+0000","last_fullsized":"2023-11-24T08:10:41.415970+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:51.581068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412422+0000","last_change":"2023-11-24T08:10:41.413102+0000","last_active":"2023-11-24T08:10:42.412422+0000","last_peered":"2023-11-24T08:10:42.412422+0000","last_clean":"2023-11-24T08:10:42.412422+0000","last_became_active":"2023-11-24T08:10:41.411422+0000","last_became_peered":"2023-11-24T08:10:41.411422+0000","last_unstale":"2023-11-24T08:10:42.412422+0000","last_undegraded":"2023-11-24T08:10:42.412422+0000","last_fullsized":"2023-11-24T08:10:42.412422+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:36:52.594661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"48'1","reported_seq":13,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409471+0000","last_change":"2023-11-24T08:10:40.390684+0000","last_active":"2023-11-24T08:10:42.409471+0000","last_peered":"2023-11-24T08:10:42.409471+0000","last_clean":"2023-11-24T08:10:42.409471+0000","last_became_active":"2023-11-24T08:10:40.389362+0000","last_became_peered":"2023-11-24T08:10:40.389362+0000","last_unstale":"2023-11-24T08:10:42.409471+0000","last_undegraded":"2023-11-24T08:10:42.409471+0000","last_fullsized":"2023-11-24T08:10:42.409471+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:15.532245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460622+0000","last_change":"2023-11-24T08:10:41.426539+0000","last_active":"2023-11-24T08:10:43.460622+0000","last_peered":"2023-11-24T08:10:43.460622+0000","last_clean":"2023-11-24T08:10:43.460622+0000","last_became_active":"2023-11-24T08:10:41.426124+0000","last_became_peered":"2023-11-24T08:10:41.426124+0000","last_unstale":"2023-11-24T08:10:43.460622+0000","last_undegraded":"2023-11-24T08:10:43.460622+0000","last_fullsized":"2023-11-24T08:10:43.460622+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:12.381653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.422174+0000","last_change":"2023-11-24T08:10:40.400533+0000","last_active":"2023-11-24T08:10:41.422174+0000","last_peered":"2023-11-24T08:10:41.422174+0000","last_clean":"2023-11-24T08:10:41.422174+0000","last_became_active":"2023-11-24T08:10:40.400241+0000","last_became_peered":"2023-11-24T08:10:40.400241+0000","last_unstale":"2023-11-24T08:10:41.422174+0000","last_undegraded":"2023-11-24T08:10:41.422174+0000","last_fullsized":"2023-11-24T08:10:41.422174+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:48:15.532245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412881+0000","last_change":"2023-11-24T08:10:41.430166+0000","last_active":"2023-11-24T08:10:42.412881+0000","last_peered":"2023-11-24T08:10:42.412881+0000","last_clean":"2023-11-24T08:10:42.412881+0000","last_became_active":"2023-11-24T08:10:41.429887+0000","last_became_peered":"2023-11-24T08:10:41.429887+0000","last_unstale":"2023-11-24T08:10:42.412881+0000","last_undegraded":"2023-11-24T08:10:42.412881+0000","last_fullsized":"2023-11-24T08:10:42.412881+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:25:42.167949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.388387+0000","last_change":"2023-11-24T08:10:40.391406+0000","last_active":"2023-11-24T08:10:41.388387+0000","last_peered":"2023-11-24T08:10:41.388387+0000","last_clean":"2023-11-24T08:10:41.388387+0000","last_became_active":"2023-11-24T08:10:40.391073+0000","last_became_peered":"2023-11-24T08:10:40.391073+0000","last_unstale":"2023-11-24T08:10:41.388387+0000","last_undegraded":"2023-11-24T08:10:41.388387+0000","last_fullsized":"2023-11-24T08:10:41.388387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:22.162235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409066+0000","last_change":"2023-11-24T08:10:41.416681+0000","last_active":"2023-11-24T08:10:42.409066+0000","last_peered":"2023-11-24T08:10:42.409066+0000","last_clean":"2023-11-24T08:10:42.409066+0000","last_became_active":"2023-11-24T08:10:41.416399+0000","last_became_peered":"2023-11-24T08:10:41.416399+0000","last_unstale":"2023-11-24T08:10:42.409066+0000","last_undegraded":"2023-11-24T08:10:42.409066+0000","last_fullsized":"2023-11-24T08:10:42.409066+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:54:27.465599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417937+0000","last_change":"2023-11-24T08:10:40.400318+0000","last_active":"2023-11-24T08:10:41.417937+0000","last_peered":"2023-11-24T08:10:41.417937+0000","last_clean":"2023-11-24T08:10:41.417937+0000","last_became_active":"2023-11-24T08:10:40.400006+0000","last_became_peered":"2023-11-24T08:10:40.400006+0000","last_unstale":"2023-11-24T08:10:41.417937+0000","last_undegraded":"2023-11-24T08:10:41.417937+0000","last_fullsized":"2023-11-24T08:10:41.417937+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:16.638357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.409292+0000","last_change":"2023-11-24T08:10:41.432402+0000","last_active":"2023-11-24T08:10:42.409292+0000","last_peered":"2023-11-24T08:10:42.409292+0000","last_clean":"2023-11-24T08:10:42.409292+0000","last_became_active":"2023-11-24T08:10:41.432035+0000","last_became_peered":"2023-11-24T08:10:41.432035+0000","last_unstale":"2023-11-24T08:10:42.409292+0000","last_undegraded":"2023-11-24T08:10:42.409292+0000","last_fullsized":"2023-11-24T08:10:42.409292+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:06:36.606640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.401563+0000","last_change":"2023-11-24T08:10:40.387069+0000","last_active":"2023-11-24T08:10:41.401563+0000","last_peered":"2023-11-24T08:10:41.401563+0000","last_clean":"2023-11-24T08:10:41.401563+0000","last_became_active":"2023-11-24T08:10:40.386462+0000","last_became_peered":"2023-11-24T08:10:40.386462+0000","last_unstale":"2023-11-24T08:10:41.401563+0000","last_undegraded":"2023-11-24T08:10:41.401563+0000","last_fullsized":"2023-11-24T08:10:41.401563+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:53.195860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.578829+0000","last_change":"2023-11-24T08:10:41.410791+0000","last_active":"2023-11-24T08:10:43.578829+0000","last_peered":"2023-11-24T08:10:43.578829+0000","last_clean":"2023-11-24T08:10:43.578829+0000","last_became_active":"2023-11-24T08:10:41.410506+0000","last_became_peered":"2023-11-24T08:10:41.410506+0000","last_unstale":"2023-11-24T08:10:43.578829+0000","last_undegraded":"2023-11-24T08:10:43.578829+0000","last_fullsized":"2023-11-24T08:10:43.578829+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:25.388885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.388896+0000","last_change":"2023-11-24T08:10:40.394410+0000","last_active":"2023-11-24T08:10:41.388896+0000","last_peered":"2023-11-24T08:10:41.388896+0000","last_clean":"2023-11-24T08:10:41.388896+0000","last_became_active":"2023-11-24T08:10:40.394009+0000","last_became_peered":"2023-11-24T08:10:40.394009+0000","last_unstale":"2023-11-24T08:10:41.388896+0000","last_undegraded":"2023-11-24T08:10:41.388896+0000","last_fullsized":"2023-11-24T08:10:41.388896+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:07.631892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.641132+0000","last_change":"2023-11-24T08:10:41.411977+0000","last_active":"2023-11-24T08:10:42.641132+0000","last_peered":"2023-11-24T08:10:42.641132+0000","last_clean":"2023-11-24T08:10:42.641132+0000","last_became_active":"2023-11-24T08:10:41.411633+0000","last_became_peered":"2023-11-24T08:10:41.411633+0000","last_unstale":"2023-11-24T08:10:42.641132+0000","last_undegraded":"2023-11-24T08:10:42.641132+0000","last_fullsized":"2023-11-24T08:10:42.641132+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:17.651949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.412009+0000","last_change":"2023-11-24T08:10:40.385164+0000","last_active":"2023-11-24T08:10:41.412009+0000","last_peered":"2023-11-24T08:10:41.412009+0000","last_clean":"2023-11-24T08:10:41.412009+0000","last_became_active":"2023-11-24T08:10:40.384523+0000","last_became_peered":"2023-11-24T08:10:40.384523+0000","last_unstale":"2023-11-24T08:10:41.412009+0000","last_undegraded":"2023-11-24T08:10:41.412009+0000","last_fullsized":"2023-11-24T08:10:41.412009+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:16.638357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":11,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:43.460887+0000","last_change":"2023-11-24T08:10:41.421154+0000","last_active":"2023-11-24T08:10:43.460887+0000","last_peered":"2023-11-24T08:10:43.460887+0000","last_clean":"2023-11-24T08:10:43.460887+0000","last_became_active":"2023-11-24T08:10:41.420904+0000","last_became_peered":"2023-11-24T08:10:41.420904+0000","last_unstale":"2023-11-24T08:10:43.460887+0000","last_undegraded":"2023-11-24T08:10:43.460887+0000","last_fullsized":"2023-11-24T08:10:43.460887+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":46,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:40.350420+0000","last_clean_scrub_stamp":"2023-11-24T08:10:40.350420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:52.278676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"47'2","reported_seq":14,"reported_epoch":47,"state":"active+clean","last_fresh":"2023-11-24T08:10:41.417967+0000","last_change":"2023-11-24T08:10:40.396648+0000","last_active":"2023-11-24T08:10:41.417967+0000","last_peered":"2023-11-24T08:10:41.417967+0000","last_clean":"2023-11-24T08:10:41.417967+0000","last_became_active":"2023-11-24T08:10:40.395757+0000","last_became_peered":"2023-11-24T08:10:40.395757+0000","last_unstale":"2023-11-24T08:10:41.417967+0000","last_undegraded":"2023-11-24T08:10:41.417967+0000","last_fullsized":"2023-11-24T08:10:41.417967+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:10:39.336827+0000","last_clean_scrub_stamp":"2023-11-24T08:10:39.336827+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:57:25.960984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":187,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-24T08:10:42.412225+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:10:42.412225+0000","last_peered":"2023-11-24T08:10:42.412225+0000","last_clean":"2023-11-24T08:10:42.412225+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:10:42.412225+0000","last_undegraded":"2023-11-24T08:10:42.412225+0000","last_fullsized":"2023-11-24T08:10:42.412225+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542173,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32312,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339528,"statfs":{"total":96636764160,"available":96603676672,"internally_reserved":0,"allocated":1159168,"data_stored":818992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.018,"15min":1.0129999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.014,"15min":0.99299999999999999},"min":{"1min":0.65300000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.70399999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.141,"15min":1.129},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.323},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.179,"15min":1.167},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2250000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.90400000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.093,"15min":1.0589999999999999},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.0620000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.2150000000000001,"15min":1.206},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0699999999999998,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":0.96599999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.25,"15min":1.222},"min":{"1min":0.97999999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.151}]},{"osd":4,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.2330000000000001,"15min":1.2110000000000001},"min":{"1min":0.95599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.113},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3959999999999999,"15min":1.3779999999999999},"min":{"1min":1.0920000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.2909999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.4399999999999999,"15min":1.4390000000000001},"min":{"1min":1.0449999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.931,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.395},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.371,"15min":1.355},"min":{"1min":1.0149999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.0149999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.494,"15min":1.4790000000000001},"min":{"1min":1.1890000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.482},"min":{"1min":1.016,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.8450000000000002,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.3620000000000001}]}]},{"osd":6,"up_from":14,"seq":60129542172,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31736,"kb_used_data":556,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340104,"statfs":{"total":96636764160,"available":96604266496,"internally_reserved":0,"allocated":569344,"data_stored":229206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.091,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6220000000000001,"5min":2.085,"15min":2.085},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2390000000000001,"15min":1.2450000000000001},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.766,"5min":1.794,"15min":1.794},"last":1.282}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.121,"15min":1.127},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.837,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.087},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.77400000000000002,"5min":0.749,"15min":0.749},"max":{"1min":1.615,"5min":1.98,"15min":1.98},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7729999999999999,"5min":1.778,"15min":1.778},"last":1.21},{"interface":"front","average":{"1min":1.105,"5min":1.1859999999999999,"15min":1.2},"min":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7649999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.0409999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3340000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7010000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.409},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4399999999999999,"15min":1.448},"min":{"1min":0.83299999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.948,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3240000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.359,"15min":1.3740000000000001},"min":{"1min":0.92200000000000004,"5min":0.622,"15min":0.622},"max":{"1min":1.9750000000000001,"5min":2.343,"15min":2.343},"last":1.159},{"interface":"front","average":{"1min":1.444,"5min":1.571,"15min":1.5920000000000001},"min":{"1min":0.79800000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9099999999999999,"5min":2.661,"15min":2.661},"last":1.5129999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.6859999999999999,"15min":1.728},"min":{"1min":0.88400000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.012,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.635,"15min":1.6659999999999999},"min":{"1min":0.70699999999999996,"5min":0.505,"15min":0.505},"max":{"1min":2.2610000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.375}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.7709999999999999,"15min":1.7949999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.3140000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.7010000000000001},{"interface":"front","average":{"1min":1.573,"5min":1.7669999999999999,"15min":1.7989999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.109,"5min":2.657,"15min":2.657},"last":1.464}]}]},{"osd":5,"up_from":14,"seq":60129542171,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31732,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340108,"statfs":{"total":96636764160,"available":96604270592,"internally_reserved":0,"allocated":565248,"data_stored":228604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.042,"15min":1.022},"min":{"1min":0.63600000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1639999999999999,"15min":1.1679999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.83999999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.878},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1850000000000001,"15min":1.1970000000000001},"min":{"1min":0.70199999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5469999999999999,"5min":1.722,"15min":1.722},"last":0.79200000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.76500000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.9179999999999999,"5min":2.085,"15min":2.085},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.3620000000000001,"15min":1.3720000000000001},"min":{"1min":0.748,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.75,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.748}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4330000000000001,"15min":1.4450000000000001},"min":{"1min":0.85599999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7130000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.395,"5min":1.512,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9510000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.214}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":0.91800000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7849999999999999,"5min":3.577,"15min":3.577},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.694,"15min":1.7190000000000001},"min":{"1min":1.133,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.9750000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.28}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6100000000000001,"15min":1.617},"min":{"1min":0.98299999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.121,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.546,"15min":1.54},"min":{"1min":1.1279999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.996,"5min":2.512,"15min":2.512},"last":1.1279999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.7130000000000001,"15min":1.726},"min":{"1min":1.2070000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.897,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.413},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.74,"15min":1.748},"min":{"1min":1.3220000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1989999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.337}]}]},{"osd":4,"up_from":14,"seq":60129542172,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31740,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340100,"statfs":{"total":96636764160,"available":96604262400,"internally_reserved":0,"allocated":573440,"data_stored":230146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.747,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.524,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.88},{"interface":"front","average":{"1min":1.077,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.69599999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.167}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.135,"15min":1.1479999999999999},"min":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4790000000000001,"5min":1.859,"15min":1.859},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.171,"15min":1.157},"min":{"1min":0.83399999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.4370000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.871,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8759999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.80800000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.004,"5min":2.25,"15min":2.25},"last":1.5},{"interface":"front","average":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4199999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0899999999999999,"5min":2.746,"15min":2.746},"last":1.276}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.319,"15min":1.325},"min":{"1min":0.82399999999999995,"5min":0.627,"15min":0.627},"max":{"1min":1.7350000000000001,"5min":1.923,"15min":1.923},"last":1.119},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4530000000000001,"15min":1.458},"min":{"1min":0.93100000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.145,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.661}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.444,"15min":1.452},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.946,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.554},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5609999999999999,"15min":1.5649999999999999},"min":{"1min":0.84799999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9239999999999999,"5min":2.641,"15min":2.641},"last":1.369}]},{"osd":7,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5169999999999999,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.3769999999999998,"5min":2.548,"15min":2.548},"last":1.611},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.77100000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1989999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.8520000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542172,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27580,"kb_used_data":560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344260,"statfs":{"total":96636764160,"available":96608522240,"internally_reserved":0,"allocated":573440,"data_stored":229172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":6.6879999999999997,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":8.0530000000000008,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":3.7309999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":2.9220000000000002,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.268}]},{"osd":2,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.252,"5min":1.252,"15min":1.252},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":5.7400000000000002,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.4159999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4950000000000001,"5min":1.4950000000000001,"15min":1.4950000000000001},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":8.8409999999999993,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"min":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":3.8620000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":6.2729999999999997,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.246},{"interface":"front","average":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":3.339,"5min":3.339,"15min":3.339},"last":1.6180000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":9.0579999999999998,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.6910000000000001},{"interface":"front","average":{"1min":1.5760000000000001,"5min":1.5760000000000001,"15min":1.5760000000000001},"min":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":4.4800000000000004,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.4570000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:09:45 2023","interfaces":[{"interface":"back","average":{"1min":1.71,"5min":1.71,"15min":1.71},"min":{"1min":1.048,"5min":1.048,"15min":1.048},"max":{"1min":5.1399999999999997,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.7370000000000001},{"interface":"front","average":{"1min":2.1080000000000001,"5min":2.1080000000000001,"15min":2.1080000000000001},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":16.952999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.8420000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542172,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27584,"kb_used_data":564,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344256,"statfs":{"total":96636764160,"available":96608518144,"internally_reserved":0,"allocated":577536,"data_stored":230168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.78900000000000003,"15min":0.76700000000000002},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85999999999999999,"15min":0.874},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.4339999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.97199999999999998,"15min":0.96999999999999997},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.466,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.157},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.3560000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.752,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5960000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.095},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2150000000000001,"15min":1.2190000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.2729999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.4179999999999999,"15min":1.45},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.895,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.28,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.208}]},{"osd":6,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4039999999999999,"15min":1.423},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8440000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.401},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":1.482}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4650000000000001,"15min":1.4870000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.9219999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4350000000000001,"15min":1.446},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9910000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.4670000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542172,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":552,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":565248,"data_stored":228604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.175,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.125,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.54200000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85899999999999999,"15min":0.85499999999999998},"min":{"1min":0.46700000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.52,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0209999999999999,"15min":1.01},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4219999999999999,"5min":1.621,"15min":1.621},"last":0.85499999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.157,"15min":1.163},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5509999999999999,"5min":1.907,"15min":1.907},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.999,"5min":1.048,"15min":1.056},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.069,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0700000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.91700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.7270000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3380000000000001,"15min":1.335},"min":{"1min":0.86299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8280000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2809999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.625,"5min":2.661,"15min":2.661},"last":1.022},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.3300000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6279999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.1000000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3580000000000001,"15min":1.361},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.337,"15min":1.333},"min":{"1min":0.98299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.704,"5min":2.105,"15min":2.105},"last":0.98299999999999998}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4139999999999999,"15min":1.4059999999999999},"min":{"1min":1.0569999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.0339999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3580000000000001,"15min":1.365},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7709999999999999,"5min":2.008,"15min":2.008},"last":1.3740000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542172,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28156,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343684,"statfs":{"total":96636764160,"available":96607932416,"internally_reserved":0,"allocated":1159168,"data_stored":818936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2909999999999999,"5min":1.403,"15min":1.403},"last":1.105},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.494,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2150000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.44,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92600000000000005,"15min":0.93400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.335,"5min":1.986,"15min":1.986},"last":1.0649999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0649999999999999,"15min":1.077},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.704,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.323},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95499999999999996,"15min":0.97399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.585,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.53600000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.163,"15min":1.1699999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.288,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7849999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4159999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.228},"min":{"1min":0.747,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7270000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":0.86099999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5860000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.2370000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.288,"15min":1.2789999999999999},"min":{"1min":0.755,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.9139999999999999,"5min":2.149,"15min":2.149},"last":1.371}]},{"osd":7,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.85399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7969999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.423,"15min":1.425},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.0600000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2829999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:10:51.339 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:10:51.339 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:10:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.600+0000 7f95aa167640 1 Processor -- start 2023-11-24T08:10:51.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.600+0000 7f95aa167640 1 -- start start 2023-11-24T08:10:51.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.601+0000 7f95aa167640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f95a4141b30 0x7f95a406c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.601+0000 7f95aa167640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a40ccdf0 con 0x7f95a40cede0 2023-11-24T08:10:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.601+0000 7f95aa167640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a415f470 con 0x7f95a415ef00 2023-11-24T08:10:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.601+0000 7f95aa167640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a415f750 con 0x7f95a4141b30 2023-11-24T08:10:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f95a9165640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f95a4141b30 0x7f95a406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f95a9966640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f95a40cede0 0x7f95a40cf1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47320/0 (socket says 172.21.15.106:47320) 2023-11-24T08:10:51.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f95a9966640 1 -- 172.21.15.106:0/1483545710 learned_addr learned my addr 172.21.15.106:0/1483545710 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:51.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f959bfff640 1 -- 172.21.15.106:0/1483545710 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614107538 0 0) 0x7f95a40ccdf0 con 0x7f95a40cede0 2023-11-24T08:10:51.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f959bfff640 1 -- 172.21.15.106:0/1483545710 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9590003520 con 0x7f95a40cede0 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f95a9165640 1 -- 172.21.15.106:0/1483545710 >> v1:172.21.15.156:6789/0 conn(0x7f95a415ef00 legacy=0x7f95a4071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f95a9165640 1 -- 172.21.15.106:0/1483545710 >> v1:172.21.15.106:6789/0 conn(0x7f95a40cede0 legacy=0x7f95a40cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.602+0000 7f95a9165640 1 -- 172.21.15.106:0/1483545710 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a415fa30 con 0x7f95a4141b30 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.604+0000 7f95a9165640 1 --2- 172.21.15.106:0/1483545710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a4141b30 0x7f95a406c770 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95a00097a0 tx=0x7f95a002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d038bb693861bd9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.604+0000 7f959bfff640 1 -- 172.21.15.106:0/1483545710 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a002e070 con 0x7f95a4141b30 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.604+0000 7f959bfff640 1 -- 172.21.15.106:0/1483545710 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a00315a0 con 0x7f95a4141b30 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.604+0000 7f959bfff640 1 -- 172.21.15.106:0/1483545710 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0031910 con 0x7f95a4141b30 2023-11-24T08:10:51.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.604+0000 7f95aa167640 1 -- 172.21.15.106:0/1483545710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a4141b30 msgr2=0x7f95a406c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.604+0000 7f95aa167640 1 --2- 172.21.15.106:0/1483545710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a4141b30 0x7f95a406c770 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f95a00097a0 tx=0x7f95a002b840 comp rx=0 tx=0).stop 2023-11-24T08:10:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.605+0000 7f95aa167640 1 -- 172.21.15.106:0/1483545710 shutdown_connections 2023-11-24T08:10:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.605+0000 7f95aa167640 1 --2- 172.21.15.106:0/1483545710 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a4141b30 0x7f95a406c770 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.605+0000 7f95aa167640 1 -- 172.21.15.106:0/1483545710 >> 172.21.15.106:0/1483545710 conn(0x7f95a406a500 msgr2=0x7f95a406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.605+0000 7f95aa167640 1 -- 172.21.15.106:0/1483545710 shutdown_connections 2023-11-24T08:10:51.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.605+0000 7f95aa167640 1 -- 172.21.15.106:0/1483545710 wait complete. 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.606+0000 7f95aa167640 1 Processor -- start 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.607+0000 7f95aa167640 1 -- start start 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.607+0000 7f95aa167640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95a40cede0 0x7f95a40c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95aa167640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95a4141b30 0x7f95a415ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95aa167640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 0x7f95a41408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95aa167640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a4142140 con 0x7f95a415ef00 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95aa167640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a4142420 con 0x7f95a4141b30 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95aa167640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a4142700 con 0x7f95a40cede0 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 0x7f95a41408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 0x7f95a41408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42594/0 (socket says 172.21.15.106:42594) 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 -- 172.21.15.106:0/1421585634 learned_addr learned my addr 172.21.15.106:0/1421585634 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9165640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95a40cede0 0x7f95a40c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a8964640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95a4141b30 0x7f95a415ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 -- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95a40cede0 msgr2=0x7f95a40c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95a40cede0 0x7f95a40c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 -- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95a4141b30 msgr2=0x7f95a415ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95a4141b30 0x7f95a415ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a9966640 1 -- 172.21.15.106:0/1421585634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a0009400 con 0x7f95a415ef00 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.608+0000 7f95a8964640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95a4141b30 0x7f95a415ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.609+0000 7f95a9966640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 0x7f95a41408b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f959c00caa0 tx=0x7f959c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.609+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00ac00 con 0x7f95a415ef00 2023-11-24T08:10:51.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.609+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00ad60 con 0x7f95a415ef00 2023-11-24T08:10:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.609+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c012620 con 0x7f95a415ef00 2023-11-24T08:10:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.609+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a41429e0 con 0x7f95a415ef00 2023-11-24T08:10:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.609+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a40c5390 con 0x7f95a415ef00 2023-11-24T08:10:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.610+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a4067990 con 0x7f95a415ef00 2023-11-24T08:10:51.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.611+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f959c007070 con 0x7f95a415ef00 2023-11-24T08:10:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.612+0000 7f9599ffb640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f958806aed0 0x7f958806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.612+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f959c089730 con 0x7f95a415ef00 2023-11-24T08:10:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.612+0000 7f95a9165640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f958806aed0 0x7f958806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:51.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.613+0000 7f95a9165640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f958806aed0 0x7f958806d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95a0036000 tx=0x7f95a0008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:51.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.617+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f959c0546f0 con 0x7f95a415ef00 2023-11-24T08:10:51.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.874+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f95a404e840 con 0x7f95a415ef00 2023-11-24T08:10:51.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.880+0000 7f9599ffb640 1 -- 172.21.15.106:0/1421585634 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f959c057d10 con 0x7f95a415ef00 2023-11-24T08:10:51.882 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2023-11-24T08:10:51.882 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:10:51.882 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2023-11-24T08:10:51.882 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2023-11-24T08:10:51.882 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:10:51.883 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f958806aed0 msgr2=0x7f958806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f958806aed0 0x7f958806d390 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f95a0036000 tx=0x7f95a0008030 comp rx=0 tx=0).stop 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 msgr2=0x7f95a41408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 0x7f95a41408b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f959c00caa0 tx=0x7f959c00cf60 comp rx=0 tx=0).stop 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 shutdown_connections 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f95a415ef00 0x7f95a41408b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f95a4141b30 0x7f95a415ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f958806aed0 0x7f958806d390 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 --2- 172.21.15.106:0/1421585634 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f95a40cede0 0x7f95a40c4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 >> 172.21.15.106:0/1421585634 conn(0x7f95a406a500 msgr2=0x7f95a406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 shutdown_connections 2023-11-24T08:10:51.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:51.883+0000 7f95aa167640 1 -- 172.21.15.106:0/1421585634 wait complete. 2023-11-24T08:10:51.904 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:10:52.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.108+0000 7f7fd942e640 1 Processor -- start 2023-11-24T08:10:52.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.108+0000 7f7fd942e640 1 -- start start 2023-11-24T08:10:52.112 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd942e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7fd413d120 0x7f7fd415af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.112 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd942e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7fd406d6c0 0x7f7fd406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.112 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd942e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 0x7f7fd40b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd942e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd406e060 con 0x7f7fd413d120 2023-11-24T08:10:52.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd942e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd413f9e0 con 0x7f7fd406d6c0 2023-11-24T08:10:52.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd942e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd413fcc0 con 0x7f7fd40b0a70 2023-11-24T08:10:52.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7fd413d120 0x7f7fd415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7fd413d120 0x7f7fd415af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42598/0 (socket says 172.21.15.106:42598) 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd3fff640 1 -- 172.21.15.106:0/3269645160 learned_addr learned my addr 172.21.15.106:0/3269645160 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd3fff640 1 -- 172.21.15.106:0/3269645160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 msgr2=0x7f7fd40b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd3fff640 1 --2- 172.21.15.106:0/3269645160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 0x7f7fd40b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd3fff640 1 -- 172.21.15.106:0/3269645160 >> v2:172.21.15.156:3300/0 conn(0x7f7fd406d6c0 msgr2=0x7f7fd406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.110+0000 7f7fd37fe640 1 --2- 172.21.15.106:0/3269645160 >> v2:172.21.15.156:3300/0 conn(0x7f7fd406d6c0 0x7f7fd406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.111+0000 7f7fd3fff640 1 --2- 172.21.15.106:0/3269645160 >> v2:172.21.15.156:3300/0 conn(0x7f7fd406d6c0 0x7f7fd406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.111+0000 7f7fd3fff640 1 -- 172.21.15.106:0/3269645160 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd413ffa0 con 0x7f7fd413d120 2023-11-24T08:10:52.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.111+0000 7f7fd37fe640 1 --2- 172.21.15.106:0/3269645160 >> v2:172.21.15.156:3300/0 conn(0x7f7fd406d6c0 0x7f7fd406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:52.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.112+0000 7f7fd3fff640 1 --2- 172.21.15.106:0/3269645160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd415af40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4072a70 tx=0x7f7fc402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae9e201fe428698e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.112+0000 7f7fd2ffd640 1 -- 172.21.15.106:0/3269645160 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc402c9e0 con 0x7f7fd413d120 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.112+0000 7f7fd2ffd640 1 -- 172.21.15.106:0/3269645160 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc402cb40 con 0x7f7fd413d120 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.112+0000 7f7fd2ffd640 1 -- 172.21.15.106:0/3269645160 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc40343f0 con 0x7f7fd413d120 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.112+0000 7f7fd942e640 1 -- 172.21.15.106:0/3269645160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 msgr2=0x7f7fd415af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.112+0000 7f7fd942e640 1 --2- 172.21.15.106:0/3269645160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd415af40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4072a70 tx=0x7f7fc402be40 comp rx=0 tx=0).stop 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 -- 172.21.15.106:0/3269645160 shutdown_connections 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 --2- 172.21.15.106:0/3269645160 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 0x7f7fd40b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 --2- 172.21.15.106:0/3269645160 >> v2:172.21.15.156:3300/0 conn(0x7f7fd406d6c0 0x7f7fd406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 --2- 172.21.15.106:0/3269645160 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd415af40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 -- 172.21.15.106:0/3269645160 >> 172.21.15.106:0/3269645160 conn(0x7f7fd40cb830 msgr2=0x7f7fd40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:52.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 -- 172.21.15.106:0/3269645160 shutdown_connections 2023-11-24T08:10:52.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.113+0000 7f7fd942e640 1 -- 172.21.15.106:0/3269645160 wait complete. 2023-11-24T08:10:52.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.114+0000 7f7fd942e640 1 Processor -- start 2023-11-24T08:10:52.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.114+0000 7f7fd942e640 1 -- start start 2023-11-24T08:10:52.118 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.115+0000 7f7fd942e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fd406d6c0 0x7f7fd415c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.115+0000 7f7fd942e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 0x7f7fd415c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.115+0000 7f7fd942e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd4159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.115+0000 7f7fd942e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415bed0 con 0x7f7fd413d120 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.115+0000 7f7fd942e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415d5b0 con 0x7f7fd406d6c0 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.115+0000 7f7fd942e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd415b600 con 0x7f7fd40b0a70 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd4159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd4159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42612/0 (socket says 172.21.15.106:42612) 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 -- 172.21.15.106:0/635641661 learned_addr learned my addr 172.21.15.106:0/635641661 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 -- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 msgr2=0x7f7fd415c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 0x7f7fd415c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 -- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fd406d6c0 msgr2=0x7f7fd415c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd3fff640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fd406d6c0 0x7f7fd415c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fd406d6c0 0x7f7fd415c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 -- 172.21.15.106:0/635641661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009b30 con 0x7f7fd413d120 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.116+0000 7f7fd8c2d640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd4159210 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00b410 tx=0x7f7fcc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.117+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011830 con 0x7f7fd413d120 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.117+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fcc0119b0 con 0x7f7fd413d120 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.117+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc011d60 con 0x7f7fd413d120 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.117+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd415d2d0 con 0x7f7fd413d120 2023-11-24T08:10:52.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.117+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd415baf0 con 0x7f7fd413d120 2023-11-24T08:10:52.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.118+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd415c010 con 0x7f7fd413d120 2023-11-24T08:10:52.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.119+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f7fcc01a400 con 0x7f7fd413d120 2023-11-24T08:10:52.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.120+0000 7f7fd0ff9640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fb406aed0 0x7f7fb406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.120+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7fcc0898b0 con 0x7f7fd413d120 2023-11-24T08:10:52.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.120+0000 7f7fd3fff640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fb406aed0 0x7f7fb406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.120+0000 7f7fd3fff640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fb406aed0 0x7f7fb406d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4035000 tx=0x7f7fc4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:52.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.124+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7fd415c010 con 0x7f7fd413d120 2023-11-24T08:10:52.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.339+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fd415c010 con 0x7f7fd413d120 2023-11-24T08:10:52.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.340+0000 7f7fd0ff9640 1 -- 172.21.15.106:0/635641661 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f7fd415c010 con 0x7f7fd413d120 2023-11-24T08:10:52.342 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:10:52.342 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:10:40.829930+0000","modified":"2023-11-24T08:10:43.906826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1634060253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1634060253},{"type":"v1","addr":"172.21.15.106:6837","nonce":1634060253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:10:52.342 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 25 2023-11-24T08:10:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fb406aed0 msgr2=0x7f7fb406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fb406aed0 0x7f7fb406d390 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7fc4035000 tx=0x7f7fc4030000 comp rx=0 tx=0).stop 2023-11-24T08:10:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 msgr2=0x7f7fd4159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd4159210 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00b410 tx=0x7f7fcc00b8d0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 shutdown_connections 2023-11-24T08:10:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fd413d120 0x7f7fd4159210 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7fb406aed0 0x7f7fb406d390 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fd40b0a70 0x7f7fd415c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 --2- 172.21.15.106:0/635641661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fd406d6c0 0x7f7fd415c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 >> 172.21.15.106:0/635641661 conn(0x7f7fd40cb830 msgr2=0x7f7fd40af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 shutdown_connections 2023-11-24T08:10:52.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.343+0000 7f7fd942e640 1 -- 172.21.15.106:0/635641661 wait complete. 2023-11-24T08:10:52.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:10:40.829930+0000', 'modified': '2023-11-24T08:10:43.906826+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5562': {'gid': 5562, 'name': 'a', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/1634060253', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 1634060253}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1634060253}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:10:52.766 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:10:52.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T08:10:52.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.988+0000 7f5a250c6640 1 Processor -- start 2023-11-24T08:10:52.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.989+0000 7f5a250c6640 1 -- start start 2023-11-24T08:10:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.990+0000 7f5a250c6640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a200ccbd0 0x7f5a201660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.990+0000 7f5a250c6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 0x7f5a20162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.990+0000 7f5a250c6640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a201655f0 con 0x7f5a20162540 2023-11-24T08:10:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.990+0000 7f5a250c6640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a200cc380 con 0x7f5a200ccbd0 2023-11-24T08:10:52.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.990+0000 7f5a250c6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a20162b30 con 0x7f5a200ccfb0 2023-11-24T08:10:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.991+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 0x7f5a20162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.991+0000 7f5a1f59e640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7f5a20162540 0x7f5a200c9650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:37012/0 (socket says 172.21.15.106:37012) 2023-11-24T08:10:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.991+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 0x7f5a20162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:55536/0 (socket says 172.21.15.106:55536) 2023-11-24T08:10:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.991+0000 7f5a1f59e640 1 -- 172.21.15.106:0/2294671121 learned_addr learned my addr 172.21.15.106:0/2294671121 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.991+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/2294671121 >> v2:172.21.15.156:3300/0 conn(0x7f5a200ccbd0 0x7f5a201660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.992+0000 7f5a1e59c640 1 -- 172.21.15.106:0/2294671121 >> v2:172.21.15.156:3300/0 conn(0x7f5a200ccbd0 msgr2=0x7f5a201660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.992+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/2294671121 >> v2:172.21.15.156:3300/0 conn(0x7f5a200ccbd0 0x7f5a201660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.992+0000 7f5a1e59c640 1 -- 172.21.15.106:0/2294671121 >> v1:172.21.15.156:6789/0 conn(0x7f5a20162540 legacy=0x7f5a200c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.992+0000 7f5a1e59c640 1 -- 172.21.15.106:0/2294671121 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a20162e10 con 0x7f5a200ccfb0 2023-11-24T08:10:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.992+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/2294671121 >> v2:172.21.15.156:3300/0 conn(0x7f5a200ccbd0 0x7f5a201660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:10:52.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.993+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/2294671121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 0x7f5a20162000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009dc0 tx=0x7f5a0802be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e742b09cf9ab23bc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:52.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a1dd9b640 1 -- 172.21.15.106:0/2294671121 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0802cc30 con 0x7f5a200ccfb0 2023-11-24T08:10:52.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a1dd9b640 1 -- 172.21.15.106:0/2294671121 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0802cd90 con 0x7f5a200ccfb0 2023-11-24T08:10:52.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a1dd9b640 1 -- 172.21.15.106:0/2294671121 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08034660 con 0x7f5a200ccfb0 2023-11-24T08:10:52.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a250c6640 1 -- 172.21.15.106:0/2294671121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 msgr2=0x7f5a20162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:52.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a250c6640 1 --2- 172.21.15.106:0/2294671121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 0x7f5a20162000 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009dc0 tx=0x7f5a0802be80 comp rx=0 tx=0).stop 2023-11-24T08:10:52.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a250c6640 1 -- 172.21.15.106:0/2294671121 shutdown_connections 2023-11-24T08:10:52.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a250c6640 1 --2- 172.21.15.106:0/2294671121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a200ccfb0 0x7f5a20162000 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a250c6640 1 --2- 172.21.15.106:0/2294671121 >> v2:172.21.15.156:3300/0 conn(0x7f5a200ccbd0 0x7f5a201660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:52.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.994+0000 7f5a250c6640 1 -- 172.21.15.106:0/2294671121 >> 172.21.15.106:0/2294671121 conn(0x7f5a200b0c50 msgr2=0x7f5a200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:52.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.995+0000 7f5a250c6640 1 -- 172.21.15.106:0/2294671121 shutdown_connections 2023-11-24T08:10:52.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.995+0000 7f5a250c6640 1 -- 172.21.15.106:0/2294671121 wait complete. 2023-11-24T08:10:52.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.996+0000 7f5a250c6640 1 Processor -- start 2023-11-24T08:10:52.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.997+0000 7f5a250c6640 1 -- start start 2023-11-24T08:10:52.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.997+0000 7f5a250c6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a200ccbd0 0x7f5a20161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.997+0000 7f5a250c6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 0x7f5a20164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.997+0000 7f5a250c6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a20162540 0x7f5a20071510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:52.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a250c6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2009ca00 con 0x7f5a200ccfb0 2023-11-24T08:10:52.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a250c6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200999d0 con 0x7f5a200ccbd0 2023-11-24T08:10:52.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a250c6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a200b4bb0 con 0x7f5a20162540 2023-11-24T08:10:53.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 0x7f5a20164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 0x7f5a20164e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42622/0 (socket says 172.21.15.106:42622) 2023-11-24T08:10:53.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 -- 172.21.15.106:0/817682844 learned_addr learned my addr 172.21.15.106:0/817682844 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a20162540 0x7f5a20071510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a200ccbd0 0x7f5a20161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 -- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a20162540 msgr2=0x7f5a20071510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a20162540 0x7f5a20071510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 -- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a200ccbd0 msgr2=0x7f5a20161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a200ccbd0 0x7f5a20161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1e59c640 1 -- 172.21.15.106:0/817682844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a10009400 con 0x7f5a200ccfb0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.998+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a20162540 0x7f5a20071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.999+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 0x7f5a20164e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d60 tx=0x7f5a08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.999+0000 7f5a177fe640 1 -- 172.21.15.106:0/817682844 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08040650 con 0x7f5a200ccfb0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.999+0000 7f5a177fe640 1 -- 172.21.15.106:0/817682844 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a080407d0 con 0x7f5a200ccfb0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.999+0000 7f5a177fe640 1 -- 172.21.15.106:0/817682844 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a08040b80 con 0x7f5a200ccfb0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.999+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a08009a70 con 0x7f5a200ccfb0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:52.999+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a20161800 con 0x7f5a200ccfb0 2023-11-24T08:10:53.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.001+0000 7f5a177fe640 1 -- 172.21.15.106:0/817682844 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f5a08040ce0 con 0x7f5a200ccfb0 2023-11-24T08:10:53.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.002+0000 7f5a177fe640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a0406aed0 0x7f5a0406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.002+0000 7f5a177fe640 1 -- 172.21.15.106:0/817682844 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a080ac430 con 0x7f5a200ccfb0 2023-11-24T08:10:53.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.003+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a0406aed0 0x7f5a0406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.003+0000 7f5a27c58740 1 Processor -- start 2023-11-24T08:10:53.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.003+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a0406aed0 0x7f5a0406d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009bc0 tx=0x7f5a10008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.003+0000 7f5a27c58740 1 -- start start 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc64ea0 0x55c45fc65280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fca8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a27c58740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fbc3880 con 0x55c45fc645e0 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fbd2ae0 con 0x55c45fc5fba0 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45fbd01d0 con 0x55c45fc64ea0 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a1f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fca8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a1f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fca8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42632/0 (socket says 172.21.15.106:42632) 2023-11-24T08:10:53.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.004+0000 7f5a1f59e640 1 -- 172.21.15.106:0/2152279733 learned_addr learned my addr 172.21.15.106:0/2152279733 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:53.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc64ea0 0x55c45fc65280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1f59e640 1 -- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc64ea0 msgr2=0x55c45fc65280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc64ea0 0x55c45fc65280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1f59e640 1 -- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 msgr2=0x55c45fc640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc640a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1f59e640 1 -- 172.21.15.106:0/2152279733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45fb93860 con 0x55c45fc645e0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fca8560 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a18001610 tx=0x7f5a18001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f81a26cd19eff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a14ff9640 1 -- 172.21.15.106:0/2152279733 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800b070 con 0x55c45fc645e0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a14ff9640 1 -- 172.21.15.106:0/2152279733 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a18005cc0 con 0x55c45fc645e0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.005+0000 7f5a14ff9640 1 -- 172.21.15.106:0/2152279733 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a180072f0 con 0x55c45fc645e0 2023-11-24T08:10:53.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 -- 172.21.15.106:0/2152279733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 msgr2=0x55c45fca8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fca8560 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f5a18001610 tx=0x7f5a18001ad0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 -- 172.21.15.106:0/2152279733 shutdown_connections 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fca8560 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 --2- 172.21.15.106:0/2152279733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc64ea0 0x55c45fc65280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 -- 172.21.15.106:0/2152279733 >> 172.21.15.106:0/2152279733 conn(0x55c45fc26c70 msgr2=0x55c45fcaa990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 -- 172.21.15.106:0/2152279733 shutdown_connections 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.006+0000 7f5a27c58740 1 -- 172.21.15.106:0/2152279733 wait complete. 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.007+0000 7f5a27c58740 1 -- start start 2023-11-24T08:10:53.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.007+0000 7f5a27c58740 1 Processor -- start 2023-11-24T08:10:53.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc5fba0 0x55c45fc8e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fc90b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fcb0750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fbcbac0 con 0x55c45fc645e0 2023-11-24T08:10:53.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fb682f0 con 0x55c45fc64ea0 2023-11-24T08:10:53.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45fb85c70 con 0x55c45fc5fba0 2023-11-24T08:10:53.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a27c58740 10 client.0 _ll_register_callbacks cb 0x55c45fb8a3b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:10:53.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fc90b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a1e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fc90b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42642/0 (socket says 172.21.15.106:42642) 2023-11-24T08:10:53.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.008+0000 7f5a1e59c640 1 -- 172.21.15.106:0/3952420294 learned_addr learned my addr 172.21.15.106:0/3952420294 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:53.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc5fba0 0x55c45fc8e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1e59c640 1 -- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc5fba0 msgr2=0x55c45fc8e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc5fba0 0x55c45fc8e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fcb0750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1e59c640 1 -- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 msgr2=0x55c45fcb0750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fcb0750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1e59c640 1 -- 172.21.15.106:0/3952420294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a201655f0 con 0x55c45fc645e0 2023-11-24T08:10:53.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a1e59c640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fc90b80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009dc0 tx=0x7f5a0807b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2339bcb3ba410dd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3952420294 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0807f160 con 0x55c45fc645e0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3952420294 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a08083500 con 0x55c45fc645e0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.009+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c45fbc4aa0 con 0x55c45fc645e0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3952420294 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a080ac0c0 con 0x55c45fc645e0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 10 client.5712 fetch_fsmap learned FSMap version 25 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c45fbca140 con 0x55c45fc645e0 2023-11-24T08:10:53.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3952420294 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c45fbc4aa0 con 0x55c45fc645e0 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3952420294 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a080aad20 con 0x55c45fc645e0 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 10 client.5712 fetch_fsmap finished waiting for FSMap version 25 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 10 client.5712 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 20 client.5712 populate_metadata read hostname 'smithi106' 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fc8ee20 0x55c45fcb1f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 4 client.5712 mds_command: new command op to 5562 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 --> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c45fbc47c0 con 0x55c45fc8ee20 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.010+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fc8ee20 0x55c45fcb1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.011+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fc8ee20 0x55c45fcb1f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c4867d3ba2670 server_cookie=e25e21291a314b2f in_seq=0 out_seq=0 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.011+0000 7f5a0dffb640 10 client.5712 ms_handle_connect on v2:172.21.15.106:6836/1634060253 2023-11-24T08:10:53.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3952420294 <== mds.0 v2:172.21.15.106:6836/1634060253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c45fbc47c0 con 0x55c45fc8ee20 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a0dffb640 10 client.5712 handle_command_reply: tid=0 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 client.5712 shutdown 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 msgr2=0x55c45fc90b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fc90b80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009dc0 tx=0x7f5a0807b980 comp rx=0 tx=0).stop 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 shutdown_connections 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fc8ee20 0x55c45fcb1f80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fcb0750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc645e0 0x55c45fc90b80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3952420294 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc5fba0 0x55c45fc8e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 >> 172.21.15.106:0/3952420294 conn(0x55c45fc26c70 msgr2=0x55c45fca8320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a0dffb640 0 client.5712 ms_handle_reset on v2:172.21.15.106:6836/1634060253 2023-11-24T08:10:53.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 shutdown_connections 2023-11-24T08:10:53.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 1 -- 172.21.15.106:0/3952420294 wait complete. 2023-11-24T08:10:53.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.012+0000 7f5a27c58740 20 client.5712 trim_cache size 0 max 16384 2023-11-24T08:10:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.029+0000 7f5a27c58740 1 Processor -- start 2023-11-24T08:10:53.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.029+0000 7f5a27c58740 1 -- start start 2023-11-24T08:10:53.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 0x55c45fc90bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 0x55c45fc60030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fc5eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a27c58740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45f92ef20 con 0x55c45fc5fba0 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fbcda40 con 0x55c45fc64ea0 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45fc914a0 con 0x55c45fc645e0 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 0x55c45fc90bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 0x55c45fc90bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42644/0 (socket says 172.21.15.106:42644) 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 -- 172.21.15.106:0/3073568773 learned_addr learned my addr 172.21.15.106:0/3073568773 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 -- 172.21.15.106:0/3073568773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 msgr2=0x55c45fc60030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 0x55c45fc60030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 -- 172.21.15.106:0/3073568773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 msgr2=0x55c45fc5eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fc5eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 -- 172.21.15.106:0/3073568773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45fb67b80 con 0x55c45fc5fba0 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.030+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 0x55c45fc90bc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55c45fcaecd0 tx=0x7f5a10014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04556026f10e639 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a157fa640 1 -- 172.21.15.106:0/3073568773 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a10027070 con 0x55c45fc5fba0 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a157fa640 1 -- 172.21.15.106:0/3073568773 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a10022850 con 0x55c45fc5fba0 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a157fa640 1 -- 172.21.15.106:0/3073568773 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a100229c0 con 0x55c45fc5fba0 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 -- 172.21.15.106:0/3073568773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 msgr2=0x55c45fc90bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 0x55c45fc90bc0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x55c45fcaecd0 tx=0x7f5a10014000 comp rx=0 tx=0).stop 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 -- 172.21.15.106:0/3073568773 shutdown_connections 2023-11-24T08:10:53.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc64ea0 0x55c45fc5eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 0x55c45fc60030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3073568773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc5fba0 0x55c45fc90bc0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 -- 172.21.15.106:0/3073568773 >> 172.21.15.106:0/3073568773 conn(0x55c45fc26c70 msgr2=0x55c45fcb10a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 -- 172.21.15.106:0/3073568773 shutdown_connections 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.031+0000 7f5a27c58740 1 -- 172.21.15.106:0/3073568773 wait complete. 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.032+0000 7f5a27c58740 1 -- start start 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.032+0000 7f5a27c58740 1 Processor -- start 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc5ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 0x55c45fcbccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 0x55c45fcbd230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fcb1fe0 con 0x55c45fc64ea0 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45fc855c0 con 0x55c45fc5fba0 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45fca0320 con 0x55c45fc645e0 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a27c58740 10 client.0 _ll_register_callbacks cb 0x7f5a0802cc40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.033+0000 7f5a1f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 0x55c45fcbd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 0x55c45fcbd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42660/0 (socket says 172.21.15.106:42660) 2023-11-24T08:10:53.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 -- 172.21.15.106:0/3853259550 learned_addr learned my addr 172.21.15.106:0/3853259550 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:53.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 -- 172.21.15.106:0/3853259550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 msgr2=0x55c45fcbccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 0x55c45fcbccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 -- 172.21.15.106:0/3853259550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 msgr2=0x55c45fc5ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc5ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 -- 172.21.15.106:0/3853259550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45fb8d170 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a1f59e640 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 0x55c45fcbd230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005880 tx=0x7f5a180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a53ad7505f32e3cc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a0dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3853259550 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18007d30 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3853259550 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5a18001d20 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.034+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a18005cc0 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3853259550 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a1800d940 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 10 client.5718 fetch_fsmap learned FSMap version 25 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c45fb73f40 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3853259550 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a18005cc0 con 0x55c45fc64ea0 2023-11-24T08:10:53.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3853259550 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 25) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a1801d040 con 0x55c45fc64ea0 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 10 client.5718 fetch_fsmap finished waiting for FSMap version 25 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 10 client.5718 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 20 client.5718 populate_metadata read hostname 'smithi106' 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fd0be30 0x55c45fd0c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 4 client.5718 mds_command: new command op to 5562 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 --> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c45f8cc6c0 con 0x55c45fd0be30 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.035+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fd0be30 0x55c45fd0c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a1ed9d640 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fd0be30 0x55c45fd0c210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7468bb2dbe1064f3 server_cookie=f35e4b8919ecbc5f in_seq=0 out_seq=0 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a0dffb640 10 client.5718 ms_handle_connect on v2:172.21.15.106:6836/1634060253 2023-11-24T08:10:53.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a0dffb640 1 -- 172.21.15.106:0/3853259550 <== mds.0 v2:172.21.15.106:6836/1634060253 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55c45f8cc6c0 con 0x55c45fd0be30 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a0dffb640 10 client.5718 handle_command_reply: tid=0 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 client.5718 shutdown 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 msgr2=0x55c45fcbd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 0x55c45fcbd230 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005880 tx=0x7f5a180057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 shutdown_connections 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:6836/1634060253,v1:172.21.15.106:6837/1634060253] conn(0x55c45fd0be30 0x55c45fd0c210 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x55c45fc64ea0 0x55c45fcbd230 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x55c45fc645e0 0x55c45fcbccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 --2- 172.21.15.106:0/3853259550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45fc5fba0 0x55c45fc5ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a0dffb640 0 client.5718 ms_handle_reset on v2:172.21.15.106:6836/1634060253 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.036+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 >> 172.21.15.106:0/3853259550 conn(0x55c45fc26c70 msgr2=0x55c45fc60150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 shutdown_connections 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a27c58740 1 -- 172.21.15.106:0/3853259550 wait complete. 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a27c58740 20 client.5718 trim_cache size 0 max 16384 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d1a897c2-15ce-4272-ad66-4b3723970cb1", 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 5562, 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2023-11-24T08:10:53.044 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 11.652844204999999, 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 25, 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 48, 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 48, 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 17.884913832999999 2023-11-24T08:10:53.045 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a0406aed0 msgr2=0x7f5a0406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a0406aed0 0x7f5a0406d390 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5a10009bc0 tx=0x7f5a10008040 comp rx=0 tx=0).stop 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 msgr2=0x7f5a20164e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 0x7f5a20164e40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5a08009d60 tx=0x7f5a08004a40 comp rx=0 tx=0).stop 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 shutdown_connections 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a0406aed0 0x7f5a0406d390 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a20162540 0x7f5a20071510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a200ccfb0 0x7f5a20164e40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 --2- 172.21.15.106:0/817682844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a200ccbd0 0x7f5a20161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.037+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 >> 172.21.15.106:0/817682844 conn(0x7f5a200b0c50 msgr2=0x7f5a200ca550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.038+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 shutdown_connections 2023-11-24T08:10:53.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.038+0000 7f5a250c6640 1 -- 172.21.15.106:0/817682844 wait complete. 2023-11-24T08:10:53.055 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:10:53.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.251+0000 7f4165fb0640 1 Processor -- start 2023-11-24T08:10:53.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.252+0000 7f4165fb0640 1 -- start start 2023-11-24T08:10:53.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4165fb0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4160157c70 0x7f4160161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4165fb0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f416015e390 0x7f41600cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4165fb0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41600ce0b0 con 0x7f41600cf450 2023-11-24T08:10:53.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4165fb0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41600cfe40 con 0x7f4160157c70 2023-11-24T08:10:53.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4165fb0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4160161960 con 0x7f416015e390 2023-11-24T08:10:53.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4156ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f41600cf450 0x7f416015dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:47326/0 (socket says 172.21.15.106:47326) 2023-11-24T08:10:53.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.253+0000 7f4156ffd640 1 -- 172.21.15.106:0/1410289828 learned_addr learned my addr 172.21.15.106:0/1410289828 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:53.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415ffff640 1 --2- 172.21.15.106:0/1410289828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f416015e390 0x7f41600cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415f7fe640 1 --2- 172.21.15.106:0/1410289828 >> v2:172.21.15.156:3300/0 conn(0x7f4160157c70 0x7f4160161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415effd640 1 -- 172.21.15.106:0/1410289828 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 12729240 0 0) 0x7f41600ce0b0 con 0x7f41600cf450 2023-11-24T08:10:53.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415effd640 1 -- 172.21.15.106:0/1410289828 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4140003520 con 0x7f41600cf450 2023-11-24T08:10:53.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415effd640 1 -- 172.21.15.106:0/1410289828 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3696810448 0 0) 0x7f4140003520 con 0x7f41600cf450 2023-11-24T08:10:53.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415ffff640 1 -- 172.21.15.106:0/1410289828 >> v2:172.21.15.156:3300/0 conn(0x7f4160157c70 msgr2=0x7f4160161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:53.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415ffff640 1 --2- 172.21.15.106:0/1410289828 >> v2:172.21.15.156:3300/0 conn(0x7f4160157c70 0x7f4160161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415ffff640 1 -- 172.21.15.106:0/1410289828 >> v1:172.21.15.106:6789/0 conn(0x7f41600cf450 legacy=0x7f416015dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.254+0000 7f415ffff640 1 -- 172.21.15.106:0/1410289828 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416015e8d0 con 0x7f416015e390 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.256+0000 7f415ffff640 1 --2- 172.21.15.106:0/1410289828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f416015e390 0x7f41600cf900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4158009d90 tx=0x7f415802bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8af9ee36115b69b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.256+0000 7f415effd640 1 -- 172.21.15.106:0/1410289828 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4148001670 con 0x7f416015e390 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.256+0000 7f415effd640 1 -- 172.21.15.106:0/1410289828 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415802c810 con 0x7f416015e390 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.256+0000 7f415effd640 1 -- 172.21.15.106:0/1410289828 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415802cb80 con 0x7f416015e390 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 -- 172.21.15.106:0/1410289828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f416015e390 msgr2=0x7f41600cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1410289828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f416015e390 0x7f41600cf900 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4158009d90 tx=0x7f415802bce0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 -- 172.21.15.106:0/1410289828 shutdown_connections 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1410289828 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f416015e390 0x7f41600cf900 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1410289828 >> v2:172.21.15.156:3300/0 conn(0x7f4160157c70 0x7f4160161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 -- 172.21.15.106:0/1410289828 >> 172.21.15.106:0/1410289828 conn(0x7f416006a920 msgr2=0x7f416006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 -- 172.21.15.106:0/1410289828 shutdown_connections 2023-11-24T08:10:53.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.257+0000 7f4165fb0640 1 -- 172.21.15.106:0/1410289828 wait complete. 2023-11-24T08:10:53.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.259+0000 7f4165fb0640 1 Processor -- start 2023-11-24T08:10:53.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.259+0000 7f4165fb0640 1 -- start start 2023-11-24T08:10:53.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f4165fb0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41600cf450 0x7f41601f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f4165fb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4160157c70 0x7f41600ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f4165fb0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 0x7f41600aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f4165fb0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416009cb70 con 0x7f416015e390 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f4165fb0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41601f65c0 con 0x7f4160157c70 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f4165fb0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416009bff0 con 0x7f41600cf450 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f415ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 0x7f41600aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f415ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 0x7f41600aebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:42662/0 (socket says 172.21.15.106:42662) 2023-11-24T08:10:53.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.260+0000 7f415ffff640 1 -- 172.21.15.106:0/1177913733 learned_addr learned my addr 172.21.15.106:0/1177913733 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:53.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415f7fe640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41600cf450 0x7f41601f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f4156ffd640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4160157c70 0x7f41600ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415ffff640 1 -- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41600cf450 msgr2=0x7f41601f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415ffff640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41600cf450 0x7f41601f6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415ffff640 1 -- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4160157c70 msgr2=0x7f41600ae6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415ffff640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4160157c70 0x7f41600ae6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415ffff640 1 -- 172.21.15.106:0/1177913733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4158009a70 con 0x7f416015e390 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f4156ffd640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4160157c70 0x7f41600ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415ffff640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 0x7f41600aebf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f415802bd20 tx=0x7f4158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4158009450 con 0x7f416015e390 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4158035ce0 con 0x7f416015e390 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415803e650 con 0x7f416015e390 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.261+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416009c2d0 con 0x7f416015e390 2023-11-24T08:10:53.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.262+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416009c890 con 0x7f416015e390 2023-11-24T08:10:53.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.263+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41600cdf70 con 0x7f416015e390 2023-11-24T08:10:53.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.264+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f415803b050 con 0x7f416015e390 2023-11-24T08:10:53.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.265+0000 7f415cff9640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f414406af00 0x7f414406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:53.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.265+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41580ab180 con 0x7f416015e390 2023-11-24T08:10:53.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.265+0000 7f415f7fe640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f414406af00 0x7f414406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:53.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.266+0000 7f415f7fe640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f414406af00 0x7f414406d3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4150009ad0 tx=0x7f4150008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:53.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.269+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4158076240 con 0x7f416015e390 2023-11-24T08:10:53.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.505+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4160067470 con 0x7f416015e390 2023-11-24T08:10:53.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.506+0000 7f415cff9640 1 -- 172.21.15.106:0/1177913733 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4158079860 con 0x7f416015e390 2023-11-24T08:10:53.508 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:10:53.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f414406af00 msgr2=0x7f414406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f414406af00 0x7f414406d3c0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4150009ad0 tx=0x7f4150008040 comp rx=0 tx=0).stop 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 msgr2=0x7f41600aebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 0x7f41600aebf0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f415802bd20 tx=0x7f4158004a40 comp rx=0 tx=0).stop 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 shutdown_connections 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f416015e390 0x7f41600aebf0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4160157c70 0x7f41600ae6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f414406af00 0x7f414406d3c0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 --2- 172.21.15.106:0/1177913733 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f41600cf450 0x7f41601f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:53.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 >> 172.21.15.106:0/1177913733 conn(0x7f416006a920 msgr2=0x7f416006d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:53.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 shutdown_connections 2023-11-24T08:10:53.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:53.509+0000 7f4165fb0640 1 -- 172.21.15.106:0/1177913733 wait complete. 2023-11-24T08:10:53.529 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:10:53.529 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:10:53.529 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:10:53.530 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:10:53.530 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:53.530 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:10:53.530 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:10:53.530 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:10:53.530 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:10:53.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:53.531 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:10:53.584 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:10:53.584 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:10:53.584 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:10:53.584 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.584 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:10:53.584 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:53.585 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42263sec preferred_lft 42263sec 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a407:f7ff:fe84:63d0/64 scope link 2023-11-24T08:10:53.586 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9079:d8ff:febb:c08c/64 scope link 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::20e3:f2ff:fe93:487e/64 scope link 2023-11-24T08:10:53.587 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:53.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:53.588 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:10:53.679 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:10:53.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:53.683 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:10:53.738 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:10:53.738 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:10:53.739 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:10:53.739 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:10:53.739 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:10:53.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:53.739 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:10:53.739 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:10:53.739 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:10:53.740 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:10:53.740 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:10:53.740 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:10:53.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:53.926 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:10:53.976 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:10:53.976 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:10:53.977 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:10:53.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:53.977 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2023-11-24T08:10:54.062 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:54.062 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:54.119 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:10:54.119 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:54.119 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:54.172 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:10:54.172 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:54.172 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:54.312 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:54.312 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:54.345 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:54.346 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:54.346 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:10:54.346 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:10:54.346 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:10:54.347 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-24T08:10:54.347 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:10:54.347 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:10:54.347 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:10:54.347 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:10:54.350 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:10:54.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:54.351 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:10:54.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:54.434 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:10:54.497 INFO:teuthology.orchestra.run.smithi106.stdout:client4670 2023-11-24T08:10:54.498 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:10:54.498 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:54.547 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:10:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.805+0000 7f3f6ebea640 1 Processor -- start 2023-11-24T08:10:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.805+0000 7f3f6ebea640 1 -- start start 2023-11-24T08:10:54.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f6ebea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f6815c7f0 0x7f3f6806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f6ebea640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f680cf450 0x7f3f680cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f6ebea640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f680cece0 con 0x7f3f680cdfb0 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f6ebea640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f68156010 con 0x7f3f680cf450 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f6ebea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f6806ce30 con 0x7f3f6815c7f0 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f5ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f680cf450 0x7f3f680cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f5ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3f680cf450 0x7f3f680cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54572/0 (socket says 172.21.15.106:54572) 2023-11-24T08:10:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1554746048 learned_addr learned my addr 172.21.15.106:0/1554746048 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.807+0000 7f3f6c986640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f6815c7f0 0x7f3f6806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.808+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1554746048 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f6815c7f0 msgr2=0x7f3f6806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.808+0000 7f3f5ffff640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f6815c7f0 0x7f3f6806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.808+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1554746048 >> v1:172.21.15.156:6789/0 conn(0x7f3f680cdfb0 legacy=0x7f3f680ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.808+0000 7f3f5ffff640 1 -- 172.21.15.106:0/1554746048 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6806d040 con 0x7f3f680cf450 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.808+0000 7f3f6c986640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f6815c7f0 0x7f3f6806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.809+0000 7f3f5ffff640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f680cf8d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009870 tx=0x7f3f5802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fcbb2e44dc906 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.809+0000 7f3f5f7fe640 1 -- 172.21.15.106:0/1554746048 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58031640 con 0x7f3f680cf450 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.809+0000 7f3f5f7fe640 1 -- 172.21.15.106:0/1554746048 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f580317a0 con 0x7f3f680cf450 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.809+0000 7f3f5f7fe640 1 -- 172.21.15.106:0/1554746048 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58031b30 con 0x7f3f680cf450 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.809+0000 7f3f6ebea640 1 -- 172.21.15.106:0/1554746048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 msgr2=0x7f3f680cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.809+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f680cf8d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f3f58009870 tx=0x7f3f5802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.810+0000 7f3f6ebea640 1 -- 172.21.15.106:0/1554746048 shutdown_connections 2023-11-24T08:10:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.810+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f680cf8d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.810+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/1554746048 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f6815c7f0 0x7f3f6806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.810+0000 7f3f6ebea640 1 -- 172.21.15.106:0/1554746048 >> 172.21.15.106:0/1554746048 conn(0x7f3f6806a920 msgr2=0x7f3f6806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.810+0000 7f3f6ebea640 1 -- 172.21.15.106:0/1554746048 shutdown_connections 2023-11-24T08:10:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.810+0000 7f3f6ebea640 1 -- 172.21.15.106:0/1554746048 wait complete. 2023-11-24T08:10:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.811+0000 7f3f6ebea640 1 Processor -- start 2023-11-24T08:10:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.812+0000 7f3f6ebea640 1 -- start start 2023-11-24T08:10:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.812+0000 7f3f6ebea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f680cdfb0 0x7f3f68154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.812+0000 7f3f6ebea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f68157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.812+0000 7f3f6ebea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6815c7f0 0x7f3f681f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f6ebea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68140290 con 0x7f3f680cf450 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f6ebea640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68159000 con 0x7f3f6815c7f0 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f6ebea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f68158760 con 0x7f3f680cdfb0 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f68157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f68157d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54574/0 (socket says 172.21.15.106:54574) 2023-11-24T08:10:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 -- 172.21.15.106:0/421797504 learned_addr learned my addr 172.21.15.106:0/421797504 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f6c986640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f680cdfb0 0x7f3f68154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 -- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f680cdfb0 msgr2=0x7f3f68154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f680cdfb0 0x7f3f68154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 -- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6815c7f0 msgr2=0x7f3f681f27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6815c7f0 0x7f3f681f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.813+0000 7f3f5ffff640 1 -- 172.21.15.106:0/421797504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580094d0 con 0x7f3f680cf450 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.814+0000 7f3f5ffff640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f68157d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3f5802b930 tx=0x7f3f580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.814+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f5803ac60 con 0x7f3f680cf450 2023-11-24T08:10:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.814+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f5803ade0 con 0x7f3f680cf450 2023-11-24T08:10:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.814+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f58042630 con 0x7f3f680cf450 2023-11-24T08:10:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.814+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f68158a40 con 0x7f3f680cf450 2023-11-24T08:10:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.814+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f68154fd0 con 0x7f3f680cf450 2023-11-24T08:10:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.815+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f68140060 con 0x7f3f680cf450 2023-11-24T08:10:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.816+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f3f5802f050 con 0x7f3f680cf450 2023-11-24T08:10:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.817+0000 7f3f5d7fa640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3f5406af00 0x7f3f5406d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.817+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f580ab2b0 con 0x7f3f680cf450 2023-11-24T08:10:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.817+0000 7f3f6c986640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3f5406af00 0x7f3f5406d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.818+0000 7f3f6c986640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3f5406af00 0x7f3f5406d3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b840 tx=0x7f3f64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:54.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:54.822+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3f68140060 con 0x7f3f680cf450 2023-11-24T08:10:55.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.050+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3f68140060 con 0x7f3f680cf450 2023-11-24T08:10:55.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.051+0000 7f3f5d7fa640 1 -- 172.21.15.106:0/421797504 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3f68140060 con 0x7f3f680cf450 2023-11-24T08:10:55.052 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:10:55.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.053+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3f5406af00 msgr2=0x7f3f5406d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:55.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.053+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3f5406af00 0x7f3f5406d3c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400b840 tx=0x7f3f64008040 comp rx=0 tx=0).stop 2023-11-24T08:10:55.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.053+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 msgr2=0x7f3f68157d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.053+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f68157d80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3f5802b930 tx=0x7f3f580046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 shutdown_connections 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f6815c7f0 0x7f3f681f27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3f680cf450 0x7f3f68157d80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3f5406af00 0x7f3f5406d3c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 --2- 172.21.15.106:0/421797504 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3f680cdfb0 0x7f3f68154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 >> 172.21.15.106:0/421797504 conn(0x7f3f6806a920 msgr2=0x7f3f680735d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 shutdown_connections 2023-11-24T08:10:55.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:55.054+0000 7f3f6ebea640 1 -- 172.21.15.106:0/421797504 wait complete. 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:10:55.075 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:10:55.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.076 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:10:55.138 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:10:55.138 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:10:55.138 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:10:55.139 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.139 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:10:55.139 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.140 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:55.140 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:55.140 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:10:55.140 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:55.140 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:55.141 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:10:55.141 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:10:55.141 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:55.141 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:10:55.141 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:55.141 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42277sec preferred_lft 42277sec 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::1831:a6ff:fe65:552d/64 scope link 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.142 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::683b:9fff:fe67:7152/64 scope link 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether a2:a3:a7:89:e5:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a0a3:a7ff:fe89:e507/64 scope link 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:10:55.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.143 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:10:55.238 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:10:55.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.243 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:10:55.297 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:10:55.298 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:10:55.298 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:10:55.298 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:10:55.298 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:10:55.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.299 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:10:55.299 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:10:55.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:10:55.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:10:55.299 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:10:55.299 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:10:55.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.472 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:10:55.522 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:10:55.523 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:10:55.524 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:10:55.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.524 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-24T08:10:55.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:55.620 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:55.676 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:10:55.676 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:10:55.676 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:55.728 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:10:55.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.728 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:55.868 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:55.868 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:55.903 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:10:55.903 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:10:55.903 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:10:55.904 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 1@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:10:55.904 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:10:55.906 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-24T08:10:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:10:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:10:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:10:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:10:55.910 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:10:55.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.910 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:10:55.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:10:55.989 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:10:56.043 INFO:teuthology.orchestra.run.smithi156.stdout:client5733 2023-11-24T08:10:56.043 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:10:56.043 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:10:56.092 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:10:56.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.318+0000 7f068a133640 1 Processor -- start 2023-11-24T08:10:56.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.318+0000 7f068a133640 1 -- start start 2023-11-24T08:10:56.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f068a133640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f068415c7f0 0x7f068406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f068a133640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cf450 0x7f06840cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f068a133640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06840cdfb0 0x7f06840ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f068a133640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0684156010 con 0x7f06840cdfb0 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f068a133640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068406ce30 con 0x7f068415c7f0 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f068a133640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06840cfe10 con 0x7f06840cf450 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f0683fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06840cdfb0 0x7f06840ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f0683fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f06840cdfb0 0x7f06840ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54584/0 (socket says 172.21.15.106:54584) 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.320+0000 7f0683fff640 1 -- 172.21.15.106:0/86005762 learned_addr learned my addr 172.21.15.106:0/86005762 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f0683fff640 1 -- 172.21.15.106:0/86005762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cf450 msgr2=0x7f06840cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f06837fe640 1 --2- 172.21.15.106:0/86005762 >> v2:172.21.15.156:3300/0 conn(0x7f068415c7f0 0x7f068406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f067affd640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cf450 0x7f06840cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:56.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f0683fff640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cf450 0x7f06840cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f0683fff640 1 -- 172.21.15.106:0/86005762 >> v2:172.21.15.156:3300/0 conn(0x7f068415c7f0 msgr2=0x7f068406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:56.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f0683fff640 1 --2- 172.21.15.106:0/86005762 >> v2:172.21.15.156:3300/0 conn(0x7f068415c7f0 0x7f068406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f0683fff640 1 -- 172.21.15.106:0/86005762 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06840cece0 con 0x7f06840cdfb0 2023-11-24T08:10:56.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f06837fe640 1 --2- 172.21.15.106:0/86005762 >> v2:172.21.15.156:3300/0 conn(0x7f068415c7f0 0x7f068406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:56.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.321+0000 7f067affd640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cf450 0x7f06840cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:56.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.322+0000 7f0683fff640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cdfb0 0x7f06840ce440 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f06840d0130 tx=0x7f067c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72720f0772f6d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:56.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f0682ffd640 1 -- 172.21.15.106:0/86005762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c031870 con 0x7f06840cdfb0 2023-11-24T08:10:56.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f0682ffd640 1 -- 172.21.15.106:0/86005762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f067c0319d0 con 0x7f06840cdfb0 2023-11-24T08:10:56.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f0682ffd640 1 -- 172.21.15.106:0/86005762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067c031d60 con 0x7f06840cdfb0 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 -- 172.21.15.106:0/86005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cdfb0 msgr2=0x7f06840ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cdfb0 0x7f06840ce440 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f06840d0130 tx=0x7f067c02b910 comp rx=0 tx=0).stop 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 -- 172.21.15.106:0/86005762 shutdown_connections 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cdfb0 0x7f06840ce440 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 --2- 172.21.15.106:0/86005762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cf450 0x7f06840cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 --2- 172.21.15.106:0/86005762 >> v2:172.21.15.156:3300/0 conn(0x7f068415c7f0 0x7f068406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.326+0000 7f068a133640 1 -- 172.21.15.106:0/86005762 >> 172.21.15.106:0/86005762 conn(0x7f068406a920 msgr2=0x7f068406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:56.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.327+0000 7f068a133640 1 -- 172.21.15.106:0/86005762 shutdown_connections 2023-11-24T08:10:56.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.327+0000 7f068a133640 1 -- 172.21.15.106:0/86005762 wait complete. 2023-11-24T08:10:56.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.328+0000 7f068a133640 1 Processor -- start 2023-11-24T08:10:56.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.329+0000 7f068a133640 1 -- start start 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.329+0000 7f068a133640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cdfb0 0x7f0684148370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.329+0000 7f068a133640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 0x7f0684146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f068a133640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f068415c7f0 0x7f06841469d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f068a133640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0684146750 con 0x7f06840cf450 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f068a133640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06841472d0 con 0x7f068415c7f0 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f068a133640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06841475b0 con 0x7f06840cdfb0 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 0x7f0684146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:56.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 0x7f0684146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54586/0 (socket says 172.21.15.106:54586) 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 -- 172.21.15.106:0/3664018244 learned_addr learned my addr 172.21.15.106:0/3664018244 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f06837fe640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cdfb0 0x7f0684148370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 -- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cdfb0 msgr2=0x7f0684148370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cdfb0 0x7f0684148370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 -- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f068415c7f0 msgr2=0x7f06841469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f068415c7f0 0x7f06841469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.330+0000 7f067affd640 1 -- 172.21.15.106:0/3664018244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f067c0094d0 con 0x7f06840cf450 2023-11-24T08:10:56.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.331+0000 7f067affd640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 0x7f0684146210 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c7c0 tx=0x7f066c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:56.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.331+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c007a30 con 0x7f06840cf450 2023-11-24T08:10:56.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.331+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f066c007b90 con 0x7f06840cf450 2023-11-24T08:10:56.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.331+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f066c00ce70 con 0x7f06840cf450 2023-11-24T08:10:56.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.331+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0684147890 con 0x7f06840cf450 2023-11-24T08:10:56.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.331+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0684147e50 con 0x7f06840cf450 2023-11-24T08:10:56.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.332+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0684148070 con 0x7f06840cf450 2023-11-24T08:10:56.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.333+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f066c0113f0 con 0x7f06840cf450 2023-11-24T08:10:56.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.335+0000 7f0680ff9640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f066406aed0 0x7f066406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:56.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.335+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f066c0899d0 con 0x7f06840cf450 2023-11-24T08:10:56.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.335+0000 7f06837fe640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f066406aed0 0x7f066406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:56.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.336+0000 7f06837fe640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f066406aed0 0x7f066406d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0674007510 tx=0x7f06740074a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:56.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.339+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f066c054880 con 0x7f06840cf450 2023-11-24T08:10:56.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:56.568+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0684067470 con 0x7f06840cf450 2023-11-24T08:10:57.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.333+0000 7f0680ff9640 1 -- 172.21.15.106:0/3664018244 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 136+0+0 (secure 0 0 0) 0x7f066c057ea0 con 0x7f06840cf450 2023-11-24T08:10:57.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f066406aed0 msgr2=0x7f066406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f066406aed0 0x7f066406d390 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f0674007510 tx=0x7f06740074a0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 msgr2=0x7f0684146210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 0x7f0684146210 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f066c00c7c0 tx=0x7f066c00cc80 comp rx=0 tx=0).stop 2023-11-24T08:10:57.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 shutdown_connections 2023-11-24T08:10:57.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f068415c7f0 0x7f06841469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f06840cf450 0x7f0684146210 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f066406aed0 0x7f066406d390 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 --2- 172.21.15.106:0/3664018244 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f06840cdfb0 0x7f0684148370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 >> 172.21.15.106:0/3664018244 conn(0x7f068406a920 msgr2=0x7f06840aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:57.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.337+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 shutdown_connections 2023-11-24T08:10:57.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.338+0000 7f068a133640 1 -- 172.21.15.106:0/3664018244 wait complete. 2023-11-24T08:10:57.360 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:10:57.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.562+0000 7f19fae2b640 1 Processor -- start 2023-11-24T08:10:57.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.563+0000 7f19fae2b640 1 -- start start 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.564+0000 7f19fae2b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19f40cfb90 0x7f19f4161a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.564+0000 7f19fae2b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19f4161fa0 0x7f19f406ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.564+0000 7f19fae2b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f406f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.564+0000 7f19fae2b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40cff70 con 0x7f19f4161fa0 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.564+0000 7f19fae2b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40afb20 con 0x7f19f40cfb90 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.564+0000 7f19fae2b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41612f0 con 0x7f19f40b2400 2023-11-24T08:10:57.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19f4161fa0 0x7f19f406ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:57.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19f4161fa0 0x7f19f406ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54600/0 (socket says 172.21.15.106:54600) 2023-11-24T08:10:57.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 -- 172.21.15.106:0/1309293566 learned_addr learned my addr 172.21.15.106:0/1309293566 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 -- 172.21.15.106:0/1309293566 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 msgr2=0x7f19f406f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f93c8640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f406f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f406f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 -- 172.21.15.106:0/1309293566 >> v2:172.21.15.156:3300/0 conn(0x7f19f40cfb90 msgr2=0x7f19f4161a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f8bc7640 1 --2- 172.21.15.106:0/1309293566 >> v2:172.21.15.156:3300/0 conn(0x7f19f40cfb90 0x7f19f4161a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 --2- 172.21.15.106:0/1309293566 >> v2:172.21.15.156:3300/0 conn(0x7f19f40cfb90 0x7f19f4161a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f3fff640 1 -- 172.21.15.106:0/1309293566 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f406a850 con 0x7f19f4161fa0 2023-11-24T08:10:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.565+0000 7f19f93c8640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f406f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.566+0000 7f19f3fff640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4161fa0 0x7f19f406ee50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f19f4070d30 tx=0x7f19dc02b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=519fb0a04f8c8a38 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19f37fe640 1 -- 172.21.15.106:0/1309293566 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc030850 con 0x7f19f4161fa0 2023-11-24T08:10:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19f37fe640 1 -- 172.21.15.106:0/1309293566 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc0309b0 con 0x7f19f4161fa0 2023-11-24T08:10:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19f37fe640 1 -- 172.21.15.106:0/1309293566 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc030d40 con 0x7f19f4161fa0 2023-11-24T08:10:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 -- 172.21.15.106:0/1309293566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4161fa0 msgr2=0x7f19f406ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4161fa0 0x7f19f406ee50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f19f4070d30 tx=0x7f19dc02b990 comp rx=0 tx=0).stop 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 -- 172.21.15.106:0/1309293566 shutdown_connections 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f406f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 --2- 172.21.15.106:0/1309293566 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4161fa0 0x7f19f406ee50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 --2- 172.21.15.106:0/1309293566 >> v2:172.21.15.156:3300/0 conn(0x7f19f40cfb90 0x7f19f4161a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 -- 172.21.15.106:0/1309293566 >> 172.21.15.106:0/1309293566 conn(0x7f19f40999d0 msgr2=0x7f19f40b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 -- 172.21.15.106:0/1309293566 shutdown_connections 2023-11-24T08:10:57.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.567+0000 7f19fae2b640 1 -- 172.21.15.106:0/1309293566 wait complete. 2023-11-24T08:10:57.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.569+0000 7f19fae2b640 1 Processor -- start 2023-11-24T08:10:57.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.569+0000 7f19fae2b640 1 -- start start 2023-11-24T08:10:57.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19fae2b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f40c9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19fae2b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f40cfb90 0x7f19f40b2de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19fae2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 0x7f19f40ca9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19fae2b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40c79f0 con 0x7f19f4155680 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19fae2b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40b3630 con 0x7f19f40cfb90 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19fae2b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40b3910 con 0x7f19f40b2400 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19f93c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 0x7f19f40ca9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19f93c8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 0x7f19f40ca9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54606/0 (socket says 172.21.15.106:54606) 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.570+0000 7f19f93c8640 1 -- 172.21.15.106:0/969017999 learned_addr learned my addr 172.21.15.106:0/969017999 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f93c8640 1 -- 172.21.15.106:0/969017999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 msgr2=0x7f19f40c9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f93c8640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f40c9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f93c8640 1 -- 172.21.15.106:0/969017999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f40cfb90 msgr2=0x7f19f40b2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f93c8640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f40cfb90 0x7f19f40b2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f93c8640 1 -- 172.21.15.106:0/969017999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19dc0094d0 con 0x7f19f4155680 2023-11-24T08:10:57.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f93c8640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 0x7f19f40ca9f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c7f0 tx=0x7f19ec00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:57.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec007820 con 0x7f19f4155680 2023-11-24T08:10:57.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec007980 con 0x7f19f4155680 2023-11-24T08:10:57.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40c7c40 con 0x7f19f4155680 2023-11-24T08:10:57.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec007d30 con 0x7f19f4155680 2023-11-24T08:10:57.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.571+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40b3320 con 0x7f19f4155680 2023-11-24T08:10:57.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.572+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f40c8160 con 0x7f19f4155680 2023-11-24T08:10:57.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.573+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f19ec0113f0 con 0x7f19f4155680 2023-11-24T08:10:57.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.574+0000 7f19f17fa640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19d406aed0 0x7f19d406d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:57.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.574+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19ec089b60 con 0x7f19f4155680 2023-11-24T08:10:57.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.575+0000 7f19f8bc7640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19d406aed0 0x7f19d406d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:57.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.575+0000 7f19f8bc7640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19d406aed0 0x7f19d406d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f19e4007e60 tx=0x7f19e4007390 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:57.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.579+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19f40c8160 con 0x7f19f4155680 2023-11-24T08:10:57.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.778+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f19f4161540 con 0x7f19d406aed0 2023-11-24T08:10:57.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.779+0000 7f19f17fa640 1 -- 172.21.15.106:0/969017999 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f19e4007a50 con 0x7f19d406aed0 2023-11-24T08:10:57.781 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:10:57.781 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:10:57.781 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2023-11-24T08:10:57.781 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:10:57.782 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19d406aed0 msgr2=0x7f19d406d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19d406aed0 0x7f19d406d390 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f19e4007e60 tx=0x7f19e4007390 comp rx=0 tx=0).stop 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 msgr2=0x7f19f40ca9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 0x7f19f40ca9f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00c7f0 tx=0x7f19ec00ccb0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 shutdown_connections 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19f4155680 0x7f19f40ca9f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19f40cfb90 0x7f19f40b2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19d406aed0 0x7f19d406d390 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 --2- 172.21.15.106:0/969017999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19f40b2400 0x7f19f40c9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 >> 172.21.15.106:0/969017999 conn(0x7f19f40999d0 msgr2=0x7f19f415f620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:57.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 shutdown_connections 2023-11-24T08:10:57.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:57.782+0000 7f19fae2b640 1 -- 172.21.15.106:0/969017999 wait complete. 2023-11-24T08:10:57.801 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:10:58.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.015+0000 7fc893ee0640 1 Processor -- start 2023-11-24T08:10:58.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.015+0000 7fc893ee0640 1 -- start start 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc893ee0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc88c15b440 0x7fc88c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc893ee0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc88c0cbf20 0x7fc88c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc893ee0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c15eea0 con 0x7fc88c0c57e0 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc893ee0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c0cc3d0 con 0x7fc88c0cbf20 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc893ee0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c15b820 con 0x7fc88c15b440 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc89147b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc88c0cbf20 0x7fc88c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc89247d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc88c0c57e0 0x7fc88c0c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46362/0 (socket says 172.21.15.106:46362) 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.017+0000 7fc89247d640 1 -- 172.21.15.106:0/4058805240 learned_addr learned my addr 172.21.15.106:0/4058805240 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:58.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.018+0000 7fc891c7c640 1 --2- 172.21.15.106:0/4058805240 >> v2:172.21.15.156:3300/0 conn(0x7fc88c15b440 0x7fc88c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.018+0000 7fc89147b640 1 -- 172.21.15.106:0/4058805240 >> v2:172.21.15.156:3300/0 conn(0x7fc88c15b440 msgr2=0x7fc88c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:58.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.018+0000 7fc89147b640 1 --2- 172.21.15.106:0/4058805240 >> v2:172.21.15.156:3300/0 conn(0x7fc88c15b440 0x7fc88c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.018+0000 7fc89147b640 1 -- 172.21.15.106:0/4058805240 >> v1:172.21.15.106:6789/0 conn(0x7fc88c0c57e0 legacy=0x7fc88c0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.018+0000 7fc89147b640 1 -- 172.21.15.106:0/4058805240 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c06d600 con 0x7fc88c0cbf20 2023-11-24T08:10:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.019+0000 7fc89147b640 1 --2- 172.21.15.106:0/4058805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0c52a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc88c06e770 tx=0x7fc87c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9466fea59a4c8a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.019+0000 7fc890c7a640 1 -- 172.21.15.106:0/4058805240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c02e070 con 0x7fc88c0cbf20 2023-11-24T08:10:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.019+0000 7fc890c7a640 1 -- 172.21.15.106:0/4058805240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c0315a0 con 0x7fc88c0cbf20 2023-11-24T08:10:58.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.019+0000 7fc890c7a640 1 -- 172.21.15.106:0/4058805240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c031930 con 0x7fc88c0cbf20 2023-11-24T08:10:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 -- 172.21.15.106:0/4058805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 msgr2=0x7fc88c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 --2- 172.21.15.106:0/4058805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0c52a0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc88c06e770 tx=0x7fc87c02b840 comp rx=0 tx=0).stop 2023-11-24T08:10:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 -- 172.21.15.106:0/4058805240 shutdown_connections 2023-11-24T08:10:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 --2- 172.21.15.106:0/4058805240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0c52a0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 --2- 172.21.15.106:0/4058805240 >> v2:172.21.15.156:3300/0 conn(0x7fc88c15b440 0x7fc88c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 -- 172.21.15.106:0/4058805240 >> 172.21.15.106:0/4058805240 conn(0x7fc88c0b0c50 msgr2=0x7fc88c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 -- 172.21.15.106:0/4058805240 shutdown_connections 2023-11-24T08:10:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.020+0000 7fc893ee0640 1 -- 172.21.15.106:0/4058805240 wait complete. 2023-11-24T08:10:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.022+0000 7fc893ee0640 1 Processor -- start 2023-11-24T08:10:58.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.022+0000 7fc893ee0640 1 -- start start 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc893ee0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc88c0c57e0 0x7fc88c0ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc893ee0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc893ee0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc88c15b440 0x7fc88c0ae970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc893ee0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b3f80 con 0x7fc88c0cbf20 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc893ee0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b4260 con 0x7fc88c0c57e0 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc893ee0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c0b4540 con 0x7fc88c15b440 2023-11-24T08:10:58.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54624/0 (socket says 172.21.15.106:54624) 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 -- 172.21.15.106:0/2260079215 learned_addr learned my addr 172.21.15.106:0/2260079215 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc891c7c640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc88c0c57e0 0x7fc88c0ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 -- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc88c15b440 msgr2=0x7fc88c0ae970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc88c15b440 0x7fc88c0ae970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 -- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc88c0c57e0 msgr2=0x7fc88c0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc88c0c57e0 0x7fc88c0ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.023+0000 7fc89147b640 1 -- 172.21.15.106:0/2260079215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87c009400 con 0x7fc88c0cbf20 2023-11-24T08:10:58.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.024+0000 7fc89147b640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0ad6e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc87c000c00 tx=0x7fc87c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.024+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c032900 con 0x7fc88c0cbf20 2023-11-24T08:10:58.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.024+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87c032a80 con 0x7fc88c0cbf20 2023-11-24T08:10:58.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.024+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87c032e30 con 0x7fc88c0cbf20 2023-11-24T08:10:58.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.024+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0b4820 con 0x7fc88c0cbf20 2023-11-24T08:10:58.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.024+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c154320 con 0x7fc88c0cbf20 2023-11-24T08:10:58.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.025+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c04e8b0 con 0x7fc88c0cbf20 2023-11-24T08:10:58.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.026+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7fc87c04d3f0 con 0x7fc88c0cbf20 2023-11-24T08:10:58.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.027+0000 7fc8827fc640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc87006aed0 0x7fc87006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.027+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc87c0ab3d0 con 0x7fc88c0cbf20 2023-11-24T08:10:58.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.027+0000 7fc891c7c640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc87006aed0 0x7fc87006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.028+0000 7fc891c7c640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc87006aed0 0x7fc87006d390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc888006fd0 tx=0x7fc888008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.031+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc87c076310 con 0x7fc88c0cbf20 2023-11-24T08:10:58.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.222+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc88c06dc80 con 0x7fc88c0cbf20 2023-11-24T08:10:58.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.223+0000 7fc8827fc640 1 -- 172.21.15.106:0/2260079215 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc87c07ee60 con 0x7fc88c0cbf20 2023-11-24T08:10:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.225+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc87006aed0 msgr2=0x7fc87006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.225+0000 7fc893ee0640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc87006aed0 0x7fc87006d390 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fc888006fd0 tx=0x7fc888008040 comp rx=0 tx=0).stop 2023-11-24T08:10:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.225+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 msgr2=0x7fc88c0ad6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.225+0000 7fc893ee0640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0ad6e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc87c000c00 tx=0x7fc87c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 shutdown_connections 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc87006aed0 0x7fc87006d390 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc88c15b440 0x7fc88c0ae970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc88c0cbf20 0x7fc88c0ad6e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 --2- 172.21.15.106:0/2260079215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc88c0c57e0 0x7fc88c0ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 >> 172.21.15.106:0/2260079215 conn(0x7fc88c0b0c50 msgr2=0x7fc88c0c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 shutdown_connections 2023-11-24T08:10:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.226+0000 7fc893ee0640 1 -- 172.21.15.106:0/2260079215 wait complete. 2023-11-24T08:10:58.246 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp8086 2023-11-24T08:10:58.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.457+0000 7f2a2d290640 1 Processor -- start 2023-11-24T08:10:58.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.457+0000 7f2a2d290640 1 -- start start 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.459+0000 7f2a2d290640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a2815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.459+0000 7f2a2d290640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a2815ec90 con 0x7f2a280c49f0 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.459+0000 7f2a2d290640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a280cd530 con 0x7f2a2806de10 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.459+0000 7f2a2d290640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c4ed0 con 0x7f2a28072160 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.459+0000 7f2a1e7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2a280c49f0 0x7f2a2806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:46374/0 (socket says 172.21.15.106:46374) 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.459+0000 7f2a1e7fc640 1 -- 172.21.15.106:0/3996630708 learned_addr learned my addr 172.21.15.106:0/3996630708 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:10:58.461 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a2815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535302270 0 0) 0x7f2a10001110 con 0x7f2a280c49f0 2023-11-24T08:10:58.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a14003520 con 0x7f2a280c49f0 2023-11-24T08:10:58.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1952179412 0 0) 0x7f2a14003520 con 0x7f2a280c49f0 2023-11-24T08:10:58.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 msgr2=0x7f2a2815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:58.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a2815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 >> v1:172.21.15.156:6789/0 conn(0x7f2a2806de10 legacy=0x7f2a280705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280c51b0 con 0x7f2a280c49f0 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2a10001800 con 0x7f2a280c49f0 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 >> v1:172.21.15.106:6789/0 conn(0x7f2a280c49f0 legacy=0x7f2a2806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a140039a0 0x7f2a14003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a140047c0 0x7f2a14004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c4ed0 con 0x7f2a140039a0 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1400bab0 con 0x7f2a140047c0 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1400bc70 con 0x7f2a28072160 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.460+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a140039a0 0x7f2a14003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a277fe640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a26ffd640 1 -- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 msgr2=0x7f2a140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:58.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a1e7fc640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a140047c0 0x7f2a14004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a140051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a26ffd640 1 -- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a140047c0 msgr2=0x7f2a14004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a140047c0 0x7f2a14004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a26ffd640 1 -- 172.21.15.106:0/3996630708 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a14003520 con 0x7f2a140039a0 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a277fe640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a28072160 0x7f2a140051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.461+0000 7f2a1e7fc640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a140047c0 0x7f2a14004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.462+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a140039a0 0x7f2a14003dc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ba30 tx=0x7f2a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fae0b7e38317037 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.462+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18032630 con 0x7f2a140039a0 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.462+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a18032790 con 0x7f2a140039a0 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.462+0000 7f2a267fc640 1 -- 172.21.15.106:0/3996630708 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18032b20 con 0x7f2a140039a0 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a2d290640 1 -- 172.21.15.106:0/3996630708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a140039a0 msgr2=0x7f2a14003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a2d290640 1 --2- 172.21.15.106:0/3996630708 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a140039a0 0x7f2a14003dc0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ba30 tx=0x7f2a140057f0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a277fe640 1 -- 172.21.15.106:0/3996630708 reap_dead start 2023-11-24T08:10:58.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a2d290640 1 -- 172.21.15.106:0/3996630708 shutdown_connections 2023-11-24T08:10:58.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a2d290640 1 -- 172.21.15.106:0/3996630708 >> 172.21.15.106:0/3996630708 conn(0x7f2a280b0c50 msgr2=0x7f2a280b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:58.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a2d290640 1 -- 172.21.15.106:0/3996630708 shutdown_connections 2023-11-24T08:10:58.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.463+0000 7f2a2d290640 1 -- 172.21.15.106:0/3996630708 wait complete. 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.465+0000 7f2a2d290640 1 Processor -- start 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.465+0000 7f2a2d290640 1 -- start start 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a2d290640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a140039a0 0x7f2a280c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a2d290640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 0x7f2a2815c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a2d290640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a28158b10 0x7f2a280aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a2d290640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c90d0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a2d290640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c7a10 con 0x7f2a28158b10 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a2d290640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2815cd70 con 0x7f2a140039a0 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 0x7f2a2815c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 0x7f2a2815c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54642/0 (socket says 172.21.15.106:54642) 2023-11-24T08:10:58.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 -- 172.21.15.106:0/1856703871 learned_addr learned my addr 172.21.15.106:0/1856703871 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 -- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a140039a0 msgr2=0x7f2a280c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a140039a0 0x7f2a280c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a140039a0 0x7f2a280c8b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a277fe640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a28158b10 0x7f2a280aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 -- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a28158b10 msgr2=0x7f2a280aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a28158b10 0x7f2a280aebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.466+0000 7f2a1e7fc640 1 -- 172.21.15.106:0/1856703871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280c92c0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.467+0000 7f2a1e7fc640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 0x7f2a2815c830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000d8e0 tx=0x7f2a1000dda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.467+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100074e0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.467+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a10007640 con 0x7f2a280c7ea0 2023-11-24T08:10:58.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.467+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100049e0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.467+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a280c73c0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.467+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a280c8300 con 0x7f2a280c7ea0 2023-11-24T08:10:58.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.468+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a280c7bc0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.469+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f2a10004b40 con 0x7f2a280c7ea0 2023-11-24T08:10:58.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.470+0000 7f2a1ffff640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a0806af00 0x7f2a0806d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.470+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a10088ec0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.470+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a0806af00 0x7f2a0806d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.471+0000 7f2a26ffd640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a0806af00 0x7f2a0806d3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ba00 tx=0x7f2a1803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.474+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2a280c7bc0 con 0x7f2a280c7ea0 2023-11-24T08:10:58.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.674+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a280c5a50 con 0x7f2a0806af00 2023-11-24T08:10:58.686 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:58.685+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:58.686 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:58.685+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:58.686 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:58.685+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:58.687 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:58.685+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:58.687 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:10:58.685+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:10:58.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.707+0000 7f2a1ffff640 1 -- 172.21.15.106:0/1856703871 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2a18032980 con 0x7f2a0806af00 2023-11-24T08:10:58.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a0806af00 msgr2=0x7f2a0806d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a0806af00 0x7f2a0806d3c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2a1800ba00 tx=0x7f2a1803d040 comp rx=0 tx=0).stop 2023-11-24T08:10:58.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 msgr2=0x7f2a2815c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 0x7f2a2815c830 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2a1000d8e0 tx=0x7f2a1000dda0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 shutdown_connections 2023-11-24T08:10:58.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a28158b10 0x7f2a280aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2a280c7ea0 0x7f2a2815c830 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2a0806af00 0x7f2a0806d3c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 --2- 172.21.15.106:0/1856703871 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2a140039a0 0x7f2a280c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 >> 172.21.15.106:0/1856703871 conn(0x7f2a280b0c50 msgr2=0x7f2a280ac190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:58.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 shutdown_connections 2023-11-24T08:10:58.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.709+0000 7f2a2d290640 1 -- 172.21.15.106:0/1856703871 wait complete. 2023-11-24T08:10:58.727 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol5699 subvol_grp8086 2023-11-24T08:10:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.925+0000 7f719147c640 1 Processor -- start 2023-11-24T08:10:58.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.926+0000 7f719147c640 1 -- start start 2023-11-24T08:10:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.927+0000 7f719147c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f718c0b2400 0x7f718c06a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.927+0000 7f719147c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f718c0af400 0x7f718c06aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.927+0000 7f719147c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f718c0cfed0 con 0x7f718c15d920 2023-11-24T08:10:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.927+0000 7f719147c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c0caae0 con 0x7f718c0af400 2023-11-24T08:10:58.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.927+0000 7f719147c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f718c0cadc0 con 0x7f718c0b2400 2023-11-24T08:10:58.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f718c0af400 0x7f718c06aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f718c0af400 0x7f718c06aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54676/0 (socket says 172.21.15.106:54676) 2023-11-24T08:10:58.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 -- 172.21.15.106:0/40156282 learned_addr learned my addr 172.21.15.106:0/40156282 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:58.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718a7fc640 1 --2- 172.21.15.106:0/40156282 >> v2:172.21.15.156:3300/0 conn(0x7f718c0b2400 0x7f718c06a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 -- 172.21.15.106:0/40156282 >> v2:172.21.15.156:3300/0 conn(0x7f718c0b2400 msgr2=0x7f718c06a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:58.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 --2- 172.21.15.106:0/40156282 >> v2:172.21.15.156:3300/0 conn(0x7f718c0b2400 0x7f718c06a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 -- 172.21.15.106:0/40156282 >> v1:172.21.15.156:6789/0 conn(0x7f718c15d920 legacy=0x7f718c0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:58.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.928+0000 7f718b7fe640 1 -- 172.21.15.106:0/40156282 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718c0cb0a0 con 0x7f718c0af400 2023-11-24T08:10:58.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.929+0000 7f718b7fe640 1 --2- 172.21.15.106:0/40156282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0af400 0x7f718c06aa40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f718c0ce420 tx=0x7f718402bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679a45ba644849ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f7189ffb640 1 -- 172.21.15.106:0/40156282 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718402cb20 con 0x7f718c0af400 2023-11-24T08:10:58.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f7189ffb640 1 -- 172.21.15.106:0/40156282 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718402cc80 con 0x7f718c0af400 2023-11-24T08:10:58.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f7189ffb640 1 -- 172.21.15.106:0/40156282 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71840343f0 con 0x7f718c0af400 2023-11-24T08:10:58.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 -- 172.21.15.106:0/40156282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0af400 msgr2=0x7f718c06aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 --2- 172.21.15.106:0/40156282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0af400 0x7f718c06aa40 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f718c0ce420 tx=0x7f718402bf80 comp rx=0 tx=0).stop 2023-11-24T08:10:58.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 -- 172.21.15.106:0/40156282 shutdown_connections 2023-11-24T08:10:58.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 --2- 172.21.15.106:0/40156282 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0af400 0x7f718c06aa40 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 --2- 172.21.15.106:0/40156282 >> v2:172.21.15.156:3300/0 conn(0x7f718c0b2400 0x7f718c06a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 -- 172.21.15.106:0/40156282 >> 172.21.15.106:0/40156282 conn(0x7f718c0999d0 msgr2=0x7f718c0b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:58.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 -- 172.21.15.106:0/40156282 shutdown_connections 2023-11-24T08:10:58.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.930+0000 7f719147c640 1 -- 172.21.15.106:0/40156282 wait complete. 2023-11-24T08:10:58.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.932+0000 7f719147c640 1 Processor -- start 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.932+0000 7f719147c640 1 -- start start 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.933+0000 7f719147c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718c0af400 0x7f718c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.933+0000 7f719147c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 0x7f718c1596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.933+0000 7f719147c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718c15d920 0x7f718c15c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.933+0000 7f719147c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0c8b40 con 0x7f718c0b2400 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.933+0000 7f719147c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0c8e20 con 0x7f718c0af400 2023-11-24T08:10:58.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.933+0000 7f719147c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718c0c9100 con 0x7f718c15d920 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 0x7f718c1596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 0x7f718c1596e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54680/0 (socket says 172.21.15.106:54680) 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 -- 172.21.15.106:0/1236726013 learned_addr learned my addr 172.21.15.106:0/1236726013 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 -- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718c15d920 msgr2=0x7f718c15c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718c15d920 0x7f718c15c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718affd640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718c0af400 0x7f718c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 -- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718c0af400 msgr2=0x7f718c0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:58.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718c0af400 0x7f718c0c8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:58.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 -- 172.21.15.106:0/1236726013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7184009b30 con 0x7f718c0b2400 2023-11-24T08:10:58.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718affd640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718c0af400 0x7f718c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:58.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.934+0000 7f718a7fc640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 0x7f718c1596e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f718000d800 tx=0x7f718000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.935+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000b6c0 con 0x7f718c0b2400 2023-11-24T08:10:58.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.935+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f718000b820 con 0x7f718c0b2400 2023-11-24T08:10:58.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.935+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f718000bbb0 con 0x7f718c0b2400 2023-11-24T08:10:58.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.935+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f718c0c93e0 con 0x7f718c0b2400 2023-11-24T08:10:58.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.935+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f718c0c9be0 con 0x7f718c0b2400 2023-11-24T08:10:58.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.936+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f718c067470 con 0x7f718c0b2400 2023-11-24T08:10:58.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.937+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 6) v1 ==== 89282+0+0 (secure 0 0 0) 0x7f718000bd10 con 0x7f718c0b2400 2023-11-24T08:10:58.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.938+0000 7f71737fe640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f717806aed0 0x7f717806d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:58.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.938+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7180089b10 con 0x7f718c0b2400 2023-11-24T08:10:58.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.938+0000 7f718affd640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f717806aed0 0x7f717806d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:58.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.939+0000 7f718affd640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f717806aed0 0x7f717806d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f717c00a830 tx=0x7f717c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:58.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:58.942+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7180054c60 con 0x7f718c0b2400 2023-11-24T08:10:59.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.148+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol5699", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7f718c0c8f90 con 0x7f717806aed0 2023-11-24T08:10:59.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.196+0000 7f71737fe640 1 -- 172.21.15.106:0/1236726013 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f718c0c8f90 con 0x7f717806aed0 2023-11-24T08:10:59.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f717806aed0 msgr2=0x7f717806d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f717806aed0 0x7f717806d390 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f717c00a830 tx=0x7f717c008040 comp rx=0 tx=0).stop 2023-11-24T08:10:59.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 msgr2=0x7f718c1596e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 0x7f718c1596e0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f718000d800 tx=0x7f718000dcc0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 shutdown_connections 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f717806aed0 0x7f717806d390 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.198+0000 7f719147c640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f718c15d920 0x7f718c15c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.199+0000 7f719147c640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f718c0b2400 0x7f718c1596e0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.199+0000 7f719147c640 1 --2- 172.21.15.106:0/1236726013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f718c0af400 0x7f718c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.199+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 >> 172.21.15.106:0/1236726013 conn(0x7f718c0999d0 msgr2=0x7f718c0ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.199+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 shutdown_connections 2023-11-24T08:10:59.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.199+0000 7f719147c640 1 -- 172.21.15.106:0/1236726013 wait complete. 2023-11-24T08:10:59.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol5699 subvol_snap1401 subvol_grp8086 2023-11-24T08:10:59.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.437+0000 7f3cba3ef640 1 Processor -- start 2023-11-24T08:10:59.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.437+0000 7f3cba3ef640 1 -- start start 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.438+0000 7f3cba3ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb415b1d0 0x7f3cb4071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.438+0000 7f3cba3ef640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cb40b1250 0x7f3cb415edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.438+0000 7f3cba3ef640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cb40ca0e0 con 0x7f3cb4070550 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.438+0000 7f3cba3ef640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb4072800 con 0x7f3cb40b1250 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.438+0000 7f3cba3ef640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb40c97b0 con 0x7f3cb415b1d0 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.438+0000 7f3cb9bee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cb40b1250 0x7f3cb415edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb9bee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3cb40b1250 0x7f3cb415edd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54682/0 (socket says 172.21.15.106:54682) 2023-11-24T08:10:59.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb9bee640 1 -- 172.21.15.106:0/2826082878 learned_addr learned my addr 172.21.15.106:0/2826082878 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb9bee640 1 -- 172.21.15.106:0/2826082878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb415b1d0 msgr2=0x7f3cb4071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb93ed640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb415b1d0 0x7f3cb4071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb9bee640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb415b1d0 0x7f3cb4071f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb9bee640 1 -- 172.21.15.106:0/2826082878 >> v1:172.21.15.156:6789/0 conn(0x7f3cb4070550 legacy=0x7f3cb40b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb9bee640 1 -- 172.21.15.106:0/2826082878 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb40c9a90 con 0x7f3cb40b1250 2023-11-24T08:10:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.439+0000 7f3cb93ed640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb415b1d0 0x7f3cb4071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:10:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.440+0000 7f3cb9bee640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb415edd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009870 tx=0x7f3cac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5dca6037abe8f3f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cb3fff640 1 -- 172.21.15.106:0/2826082878 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac031640 con 0x7f3cb40b1250 2023-11-24T08:10:59.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cb3fff640 1 -- 172.21.15.106:0/2826082878 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cac0317a0 con 0x7f3cb40b1250 2023-11-24T08:10:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cb3fff640 1 -- 172.21.15.106:0/2826082878 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cac031b30 con 0x7f3cb40b1250 2023-11-24T08:10:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2826082878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 msgr2=0x7f3cb415edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb415edd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3cac009870 tx=0x7f3cac02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2826082878 shutdown_connections 2023-11-24T08:10:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb415edd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2826082878 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb415b1d0 0x7f3cb4071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.441+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2826082878 >> 172.21.15.106:0/2826082878 conn(0x7f3cb40cbb40 msgr2=0x7f3cb40cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.442+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2826082878 shutdown_connections 2023-11-24T08:10:59.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.442+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2826082878 wait complete. 2023-11-24T08:10:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.443+0000 7f3cba3ef640 1 Processor -- start 2023-11-24T08:10:59.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.443+0000 7f3cba3ef640 1 -- start start 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.444+0000 7f3cba3ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb4070550 0x7f3cb41f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.444+0000 7f3cba3ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb40b3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.444+0000 7f3cba3ef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb415b1d0 0x7f3cb415cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.444+0000 7f3cba3ef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb409ab90 con 0x7f3cb40b1250 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.444+0000 7f3cba3ef640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb409ae70 con 0x7f3cb415b1d0 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.444+0000 7f3cba3ef640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb41f4000 con 0x7f3cb4070550 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb40b3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb40b3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54690/0 (socket says 172.21.15.106:54690) 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 -- 172.21.15.106:0/2677862617 learned_addr learned my addr 172.21.15.106:0/2677862617 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:59.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb93ed640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb4070550 0x7f3cb41f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb9bee640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb415b1d0 0x7f3cb415cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 -- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb4070550 msgr2=0x7f3cb41f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb4070550 0x7f3cb41f3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 -- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb415b1d0 msgr2=0x7f3cb415cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb415b1d0 0x7f3cb415cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.445+0000 7f3cb8bec640 1 -- 172.21.15.106:0/2677862617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0094d0 con 0x7f3cb40b1250 2023-11-24T08:10:59.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.446+0000 7f3cb8bec640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb40b3f50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000c430 tx=0x7f3ca000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:59.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.446+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011790 con 0x7f3cb40b1250 2023-11-24T08:10:59.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.446+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca00118f0 con 0x7f3cb40b1250 2023-11-24T08:10:59.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.446+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0011ca0 con 0x7f3cb40b1250 2023-11-24T08:10:59.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.446+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb4099f10 con 0x7f3cb40b1250 2023-11-24T08:10:59.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.446+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb409a4d0 con 0x7f3cb40b1250 2023-11-24T08:10:59.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.447+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb409acd0 con 0x7f3cb40b1250 2023-11-24T08:10:59.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.448+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f3ca001c460 con 0x7f3cb40b1250 2023-11-24T08:10:59.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.449+0000 7f3cb1ffb640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3c9806af60 0x7f3c9806d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.449+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ca0089740 con 0x7f3cb40b1250 2023-11-24T08:10:59.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.449+0000 7f3cb93ed640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3c9806af60 0x7f3c9806d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.450+0000 7f3cb93ed640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3c9806af60 0x7f3c9806d420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8007510 tx=0x7f3ca80074a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:59.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.454+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3cb409acd0 con 0x7f3cb40b1250 2023-11-24T08:10:59.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.676+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol5699", "snap_name": "subvol_snap1401", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cb40ad950 con 0x7f3c9806af60 2023-11-24T08:10:59.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.706+0000 7f3cb1ffb640 1 -- 172.21.15.106:0/2677862617 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3cb40ad950 con 0x7f3c9806af60 2023-11-24T08:10:59.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3c9806af60 msgr2=0x7f3c9806d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3c9806af60 0x7f3c9806d420 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8007510 tx=0x7f3ca80074a0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 msgr2=0x7f3cb40b3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb40b3f50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000c430 tx=0x7f3ca000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 shutdown_connections 2023-11-24T08:10:59.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3cb415b1d0 0x7f3cb415cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3cb40b1250 0x7f3cb40b3f50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3c9806af60 0x7f3c9806d420 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 --2- 172.21.15.106:0/2677862617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3cb4070550 0x7f3cb41f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 >> 172.21.15.106:0/2677862617 conn(0x7f3cb40cbb40 msgr2=0x7f3cb40ac6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 shutdown_connections 2023-11-24T08:10:59.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.709+0000 7f3cba3ef640 1 -- 172.21.15.106:0/2677862617 wait complete. 2023-11-24T08:10:59.730 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot metadata set cephfs subvol5699 subvol_snap1401 key value subvol_grp8086 2023-11-24T08:10:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.941+0000 7f551a37d640 1 Processor -- start 2023-11-24T08:10:59.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.942+0000 7f551a37d640 1 -- start start 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.943+0000 7f551a37d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55140af000 0x7f55140b3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.943+0000 7f551a37d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5514161060 0x7f55141614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.943+0000 7f551a37d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f551415df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.943+0000 7f551a37d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55140af3e0 con 0x7f55140af000 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.943+0000 7f551a37d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5514161a10 con 0x7f5514161060 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.943+0000 7f551a37d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551406dca0 con 0x7f551415da70 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55140af000 0x7f55140b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55140af000 0x7f55140b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54706/0 (socket says 172.21.15.106:54706) 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 -- 172.21.15.106:0/3812733741 learned_addr learned my addr 172.21.15.106:0/3812733741 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:59.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f55137fe640 1 --2- 172.21.15.106:0/3812733741 >> v2:172.21.15.156:3300/0 conn(0x7f5514161060 0x7f55141614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f551891a640 1 --2- 172.21.15.106:0/3812733741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f551415df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 -- 172.21.15.106:0/3812733741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 msgr2=0x7f551415df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 --2- 172.21.15.106:0/3812733741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f551415df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 -- 172.21.15.106:0/3812733741 >> v2:172.21.15.156:3300/0 conn(0x7f5514161060 msgr2=0x7f55141614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:10:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 --2- 172.21.15.106:0/3812733741 >> v2:172.21.15.156:3300/0 conn(0x7f5514161060 0x7f55141614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.944+0000 7f5513fff640 1 -- 172.21.15.106:0/3812733741 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551406df40 con 0x7f55140af000 2023-11-24T08:10:59.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.945+0000 7f5513fff640 1 --2- 172.21.15.106:0/3812733741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f55140b3330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f55040098a0 tx=0x7f550402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15d34077694aa62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:59.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.946+0000 7f5512ffd640 1 -- 172.21.15.106:0/3812733741 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504031640 con 0x7f55140af000 2023-11-24T08:10:59.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.946+0000 7f5512ffd640 1 -- 172.21.15.106:0/3812733741 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55040317a0 con 0x7f55140af000 2023-11-24T08:10:59.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.946+0000 7f5512ffd640 1 -- 172.21.15.106:0/3812733741 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504031b30 con 0x7f55140af000 2023-11-24T08:10:59.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.946+0000 7f551a37d640 1 -- 172.21.15.106:0/3812733741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 msgr2=0x7f55140b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.946+0000 7f551a37d640 1 --2- 172.21.15.106:0/3812733741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f55140b3330 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f55040098a0 tx=0x7f550402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 -- 172.21.15.106:0/3812733741 shutdown_connections 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 --2- 172.21.15.106:0/3812733741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f551415df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 --2- 172.21.15.106:0/3812733741 >> v2:172.21.15.156:3300/0 conn(0x7f5514161060 0x7f55141614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 --2- 172.21.15.106:0/3812733741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f55140b3330 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 -- 172.21.15.106:0/3812733741 >> 172.21.15.106:0/3812733741 conn(0x7f55140cf900 msgr2=0x7f55140cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 -- 172.21.15.106:0/3812733741 shutdown_connections 2023-11-24T08:10:59.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.947+0000 7f551a37d640 1 -- 172.21.15.106:0/3812733741 wait complete. 2023-11-24T08:10:59.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.948+0000 7f551a37d640 1 Processor -- start 2023-11-24T08:10:59.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 -- start start 2023-11-24T08:10:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f5514148320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f5514155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5514161060 0x7f5514158570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55141570c0 con 0x7f55140af000 2023-11-24T08:10:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5514148860 con 0x7f5514161060 2023-11-24T08:10:59.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.949+0000 7f551a37d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551415f0e0 con 0x7f551415da70 2023-11-24T08:10:59.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f5514148320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f5514148320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:54712/0 (socket says 172.21.15.106:54712) 2023-11-24T08:10:59.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 -- 172.21.15.106:0/3699028166 learned_addr learned my addr 172.21.15.106:0/3699028166 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 -- 172.21.15.106:0/3699028166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 msgr2=0x7f5514155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f5514155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 -- 172.21.15.106:0/3699028166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5514161060 msgr2=0x7f5514158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5514161060 0x7f5514158570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 -- 172.21.15.106:0/3699028166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55040094d0 con 0x7f55140af000 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.950+0000 7f5513fff640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f5514148320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5504000c00 tx=0x7f55040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.951+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550403ac60 con 0x7f55140af000 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.951+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550403adc0 con 0x7f55140af000 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.951+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5504044630 con 0x7f55140af000 2023-11-24T08:10:59.953 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.951+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5514156de0 con 0x7f55140af000 2023-11-24T08:10:59.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.951+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551415ed70 con 0x7f55140af000 2023-11-24T08:10:59.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.952+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551415e8f0 con 0x7f55140af000 2023-11-24T08:10:59.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.953+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f550402f050 con 0x7f55140af000 2023-11-24T08:10:59.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.955+0000 7f5510ff9640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f550006af90 0x7f550006d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:10:59.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.955+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55040ab6c0 con 0x7f55140af000 2023-11-24T08:10:59.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.955+0000 7f55137fe640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f550006af90 0x7f550006d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:10:59.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.955+0000 7f55137fe640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f550006af90 0x7f550006d450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5508006e20 tx=0x7f5508008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:10:59.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:10:59.959+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f551415e8f0 con 0x7f55140af000 2023-11-24T08:11:00.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.189+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot metadata set", "vol_name": "cephfs", "sub_name": "subvol5699", "snap_name": "subvol_snap1401", "key_name": "key", "value": "value", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7f551406beb0 con 0x7f550006af90 2023-11-24T08:11:00.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.215+0000 7f5510ff9640 1 -- 172.21.15.106:0/3699028166 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f551406beb0 con 0x7f550006af90 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f550006af90 msgr2=0x7f550006d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f550006af90 0x7f550006d450 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5508006e20 tx=0x7f5508008040 comp rx=0 tx=0).stop 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 msgr2=0x7f5514148320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f5514148320 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f5504000c00 tx=0x7f55040046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 shutdown_connections 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5514161060 0x7f5514158570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f550006af90 0x7f550006d450 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f551415da70 0x7f5514155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 --2- 172.21.15.106:0/3699028166 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55140af000 0x7f5514148320 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:00.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 >> 172.21.15.106:0/3699028166 conn(0x7f55140cf900 msgr2=0x7f55140ac9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:00.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 shutdown_connections 2023-11-24T08:11:00.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:00.218+0000 7f551a37d640 1 -- 172.21.15.106:0/3699028166 wait complete. 2023-11-24T08:11:00.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:00.238 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp8086/subvol5699/.meta ./volumes/subvol_grp8086/subvol5699/.meta.stale_snap_section) 2023-11-24T08:11:05.198 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol5699 subvol_snap1401 subvol_grp8086 2023-11-24T08:11:05.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.423+0000 7feb10d5c640 1 Processor -- start 2023-11-24T08:11:05.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.423+0000 7feb10d5c640 1 -- start start 2023-11-24T08:11:05.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.424+0000 7feb10d5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c06e150 0x7feb0c143b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.424+0000 7feb10d5c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb0c0b1e00 0x7feb0c0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.424+0000 7feb10d5c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c144050 con 0x7feb0c1434a0 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.424+0000 7feb10d5c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c06b010 con 0x7feb0c0b1e00 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.424+0000 7feb10d5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0b27d0 con 0x7feb0c06e150 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.425+0000 7feb0b7fe640 1 --1- >> v1:172.21.15.156:6789/0 conn(0x7feb0c1434a0 0x7feb0c0ad470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:36032/0 (socket says 172.21.15.106:36032) 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.425+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3599132640 learned_addr learned my addr 172.21.15.106:0/3599132640 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:05.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.425+0000 7feb0affd640 1 --2- 172.21.15.106:0/3599132640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c06e150 0x7feb0c143b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.425+0000 7feb0bfff640 1 --2- 172.21.15.106:0/3599132640 >> v2:172.21.15.156:3300/0 conn(0x7feb0c0b1e00 0x7feb0c0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.426+0000 7feb0affd640 1 -- 172.21.15.106:0/3599132640 >> v2:172.21.15.156:3300/0 conn(0x7feb0c0b1e00 msgr2=0x7feb0c0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.426+0000 7feb0affd640 1 --2- 172.21.15.106:0/3599132640 >> v2:172.21.15.156:3300/0 conn(0x7feb0c0b1e00 0x7feb0c0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.426+0000 7feb0affd640 1 -- 172.21.15.106:0/3599132640 >> v1:172.21.15.156:6789/0 conn(0x7feb0c1434a0 legacy=0x7feb0c0ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.426+0000 7feb0affd640 1 -- 172.21.15.106:0/3599132640 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c143880 con 0x7feb0c06e150 2023-11-24T08:11:05.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.426+0000 7feb0a7fc640 1 -- 172.21.15.106:0/3599132640 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3817009225 0 0) 0x7feafc001490 con 0x7feb0c1434a0 2023-11-24T08:11:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.428+0000 7feb0affd640 1 --2- 172.21.15.106:0/3599132640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c06e150 0x7feb0c143b10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feaf40097d0 tx=0x7feaf402b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26e944391d578eb0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:05.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.428+0000 7feb0a7fc640 1 -- 172.21.15.106:0/3599132640 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4031800 con 0x7feb0c06e150 2023-11-24T08:11:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.428+0000 7feb0a7fc640 1 -- 172.21.15.106:0/3599132640 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4031960 con 0x7feb0c06e150 2023-11-24T08:11:05.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.428+0000 7feb0a7fc640 1 -- 172.21.15.106:0/3599132640 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4031cf0 con 0x7feb0c06e150 2023-11-24T08:11:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 -- 172.21.15.106:0/3599132640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c06e150 msgr2=0x7feb0c143b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3599132640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c06e150 0x7feb0c143b10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feaf40097d0 tx=0x7feaf402b8a0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 -- 172.21.15.106:0/3599132640 shutdown_connections 2023-11-24T08:11:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3599132640 >> v2:172.21.15.156:3300/0 conn(0x7feb0c0b1e00 0x7feb0c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3599132640 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c06e150 0x7feb0c143b10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 -- 172.21.15.106:0/3599132640 >> 172.21.15.106:0/3599132640 conn(0x7feb0c0999d0 msgr2=0x7feb0c0b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 -- 172.21.15.106:0/3599132640 shutdown_connections 2023-11-24T08:11:05.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.429+0000 7feb10d5c640 1 -- 172.21.15.106:0/3599132640 wait complete. 2023-11-24T08:11:05.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.432+0000 7feb10d5c640 1 Processor -- start 2023-11-24T08:11:05.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.432+0000 7feb10d5c640 1 -- start start 2023-11-24T08:11:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb10d5c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 0x7feb0c13df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb10d5c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0b1e00 0x7feb0c0cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:05.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb10d5c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c1434a0 0x7feb0c06b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb10d5c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f0ce0 con 0x7feb0c06e150 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb10d5c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f1260 con 0x7feb0c0b1e00 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb10d5c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0ce470 con 0x7feb0c1434a0 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.433+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 0x7feb0c13df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 0x7feb0c13df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41328/0 (socket says 172.21.15.106:41328) 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3727560685 learned_addr learned my addr 172.21.15.106:0/3727560685 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3727560685 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c1434a0 msgr2=0x7feb0c06b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c1434a0 0x7feb0c06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3727560685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0b1e00 msgr2=0x7feb0c0cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0b1e00 0x7feb0c0cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 -- 172.21.15.106:0/3727560685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4009400 con 0x7feb0c06e150 2023-11-24T08:11:05.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb0b7fe640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 0x7feb0c13df70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c7c0 tx=0x7feafc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00a640 con 0x7feb0c06e150 2023-11-24T08:11:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00a7c0 con 0x7feb0c06e150 2023-11-24T08:11:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.434+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00ab70 con 0x7feb0c06e150 2023-11-24T08:11:05.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.435+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0ce750 con 0x7feb0c06e150 2023-11-24T08:11:05.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.435+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1f0ef0 con 0x7feb0c06e150 2023-11-24T08:11:05.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.436+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c04e840 con 0x7feb0c06e150 2023-11-24T08:11:05.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.437+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7feafc00acd0 con 0x7feb0c06e150 2023-11-24T08:11:05.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.438+0000 7feb03fff640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feaec06af60 0x7feaec06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:05.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.438+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feafc088bb0 con 0x7feb0c06e150 2023-11-24T08:11:05.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.438+0000 7feb0affd640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feaec06af60 0x7feaec06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:05.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.439+0000 7feb0affd640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feaec06af60 0x7feaec06d420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feaf4036000 tx=0x7feaf4030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:05.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.443+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feafc053b40 con 0x7feb0c06e150 2023-11-24T08:11:05.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.668+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol5699", "snap_name": "subvol_snap1401", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0c06ddf0 con 0x7feaec06af60 2023-11-24T08:11:05.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.737+0000 7feb03fff640 1 -- 172.21.15.106:0/3727560685 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7feaf40317d0 con 0x7feaec06af60 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feaec06af60 msgr2=0x7feaec06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feaec06af60 0x7feaec06d420 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7feaf4036000 tx=0x7feaf4030000 comp rx=0 tx=0).stop 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 msgr2=0x7feb0c13df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 0x7feb0c13df70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7feafc00c7c0 tx=0x7feafc00cc80 comp rx=0 tx=0).stop 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 shutdown_connections 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feaec06af60 0x7feaec06d420 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb0c1434a0 0x7feb0c06b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb0c0b1e00 0x7feb0c0cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 --2- 172.21.15.106:0/3727560685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb0c06e150 0x7feb0c13df70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 >> 172.21.15.106:0/3727560685 conn(0x7feb0c0999d0 msgr2=0x7feb0c15c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 shutdown_connections 2023-11-24T08:11:05.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:05.740+0000 7feb10d5c640 1 -- 172.21.15.106:0/3727560685 wait complete. 2023-11-24T08:11:05.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:05.763 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp -p ./volumes/subvol_grp8086/subvol5699/.meta.stale_snap_section ./volumes/subvol_grp8086/subvol5699/.meta) 2023-11-24T08:11:05.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:05.880 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap1401 ./volumes/subvol_grp8086/subvol5699/.meta) 2023-11-24T08:11:05.969 INFO:teuthology.orchestra.run.smithi106.stdout:[SNAP_METADATA_subvol_snap1401] 2023-11-24T08:11:05.974 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol5699 subvol_grp8086 2023-11-24T08:11:06.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.186+0000 7ffb97d6f640 1 Processor -- start 2023-11-24T08:11:06.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.186+0000 7ffb97d6f640 1 -- start start 2023-11-24T08:11:06.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.187+0000 7ffb97d6f640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ffb900cf450 0x7ffb900cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.187+0000 7ffb97d6f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffb900cdfb0 0x7ffb900ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.187+0000 7ffb97d6f640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb900cece0 con 0x7ffb9015c7f0 2023-11-24T08:11:06.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.187+0000 7ffb97d6f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb900cfe40 con 0x7ffb900cdfb0 2023-11-24T08:11:06.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.187+0000 7ffb97d6f640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb90156010 con 0x7ffb900cf450 2023-11-24T08:11:06.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffb900cdfb0 0x7ffb900ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffb900cdfb0 0x7ffb900ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41338/0 (socket says 172.21.15.106:41338) 2023-11-24T08:11:06.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 -- 172.21.15.106:0/4078646427 learned_addr learned my addr 172.21.15.106:0/4078646427 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:06.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9530a640 1 --2- 172.21.15.106:0/4078646427 >> v2:172.21.15.156:3300/0 conn(0x7ffb900cf450 0x7ffb900cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 -- 172.21.15.106:0/4078646427 >> v2:172.21.15.156:3300/0 conn(0x7ffb900cf450 msgr2=0x7ffb900cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:06.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 --2- 172.21.15.106:0/4078646427 >> v2:172.21.15.156:3300/0 conn(0x7ffb900cf450 0x7ffb900cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 -- 172.21.15.106:0/4078646427 >> v1:172.21.15.156:6789/0 conn(0x7ffb9015c7f0 legacy=0x7ffb9006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:06.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9630c640 1 -- 172.21.15.106:0/4078646427 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb9006d040 con 0x7ffb900cdfb0 2023-11-24T08:11:06.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.188+0000 7ffb9530a640 1 --2- 172.21.15.106:0/4078646427 >> v2:172.21.15.156:3300/0 conn(0x7ffb900cf450 0x7ffb900cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:06.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.190+0000 7ffb9630c640 1 --2- 172.21.15.106:0/4078646427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb900cdfb0 0x7ffb900ce3e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffb90070410 tx=0x7ffb8802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bed48227e80dfc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.190+0000 7ffb94b09640 1 -- 172.21.15.106:0/4078646427 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8802ca80 con 0x7ffb900cdfb0 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.190+0000 7ffb94b09640 1 -- 172.21.15.106:0/4078646427 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8802cbe0 con 0x7ffb900cdfb0 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.190+0000 7ffb94b09640 1 -- 172.21.15.106:0/4078646427 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb880343f0 con 0x7ffb900cdfb0 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 -- 172.21.15.106:0/4078646427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb900cdfb0 msgr2=0x7ffb900ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/4078646427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb900cdfb0 0x7ffb900ce3e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7ffb90070410 tx=0x7ffb8802bee0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 -- 172.21.15.106:0/4078646427 shutdown_connections 2023-11-24T08:11:06.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/4078646427 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb900cdfb0 0x7ffb900ce3e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/4078646427 >> v2:172.21.15.156:3300/0 conn(0x7ffb900cf450 0x7ffb900cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 -- 172.21.15.106:0/4078646427 >> 172.21.15.106:0/4078646427 conn(0x7ffb9006a920 msgr2=0x7ffb9006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 -- 172.21.15.106:0/4078646427 shutdown_connections 2023-11-24T08:11:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.191+0000 7ffb97d6f640 1 -- 172.21.15.106:0/4078646427 wait complete. 2023-11-24T08:11:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.193+0000 7ffb97d6f640 1 Processor -- start 2023-11-24T08:11:06.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.193+0000 7ffb97d6f640 1 -- start start 2023-11-24T08:11:06.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.194+0000 7ffb97d6f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb900cdfb0 0x7ffb9015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.194+0000 7ffb97d6f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb900cf450 0x7ffb9015eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.194+0000 7ffb97d6f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 0x7ffb9015f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.194+0000 7ffb97d6f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900b2550 con 0x7ffb9015c7f0 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.194+0000 7ffb97d6f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900b27a0 con 0x7ffb900cf450 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.194+0000 7ffb97d6f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb900b2a80 con 0x7ffb900cdfb0 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 0x7ffb9015f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 0x7ffb9015f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41352/0 (socket says 172.21.15.106:41352) 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 -- 172.21.15.106:0/1177064770 learned_addr learned my addr 172.21.15.106:0/1177064770 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9530a640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb900cf450 0x7ffb9015eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb95b0b640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb900cdfb0 0x7ffb9015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 -- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb900cdfb0 msgr2=0x7ffb9015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb900cdfb0 0x7ffb9015b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 -- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb900cf450 msgr2=0x7ffb9015eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb900cf450 0x7ffb9015eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 -- 172.21.15.106:0/1177064770 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb88009b30 con 0x7ffb9015c7f0 2023-11-24T08:11:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.195+0000 7ffb9630c640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 0x7ffb9015f400 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffb88036040 tx=0x7ffb88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.196+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb88009510 con 0x7ffb9015c7f0 2023-11-24T08:11:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.196+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb8803e450 con 0x7ffb9015c7f0 2023-11-24T08:11:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.196+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb8803e800 con 0x7ffb9015c7f0 2023-11-24T08:11:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.196+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb900b2d60 con 0x7ffb9015c7f0 2023-11-24T08:11:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.196+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb900b3320 con 0x7ffb9015c7f0 2023-11-24T08:11:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.197+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb900b3510 con 0x7ffb9015c7f0 2023-11-24T08:11:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.198+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7ffb8802cd50 con 0x7ffb9015c7f0 2023-11-24T08:11:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.199+0000 7ffb867fc640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffb7406af60 0x7ffb7406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.199+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb880ac2d0 con 0x7ffb9015c7f0 2023-11-24T08:11:06.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.199+0000 7ffb95b0b640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffb7406af60 0x7ffb7406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.200+0000 7ffb95b0b640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffb7406af60 0x7ffb7406d420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00a750 tx=0x7ffb8c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.204+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffb88077270 con 0x7ffb9015c7f0 2023-11-24T08:11:06.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.423+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol5699", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb9006d600 con 0x7ffb7406af60 2023-11-24T08:11:06.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.505+0000 7ffb867fc640 1 -- 172.21.15.106:0/1177064770 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+705 (secure 0 0 0) 0x7ffb8c00a380 con 0x7ffb7406af60 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "atime": "2023-11-24 08:10:59", 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "bytes_pcent": "undefined", 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "bytes_quota": "infinite", 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "bytes_used": 0, 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "created_at": "2023-11-24 08:10:59", 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "ctime": "2023-11-24 08:10:59", 2023-11-24T08:11:06.509 INFO:teuthology.orchestra.run.smithi106.stdout: "data_pool": "cephfs_data", 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "features": [ 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "snapshot-clone", 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "snapshot-autoprotect", 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "snapshot-retention" 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "gid": 0, 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "mode": 16877, 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "mon_addrs": [ 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "172.21.15.106:6789", 2023-11-24T08:11:06.510 INFO:teuthology.orchestra.run.smithi106.stdout: "172.21.15.156:6789", 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: "172.21.15.156:6790" 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: ], 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: "mtime": "2023-11-24 08:10:59", 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: "path": "/volumes/subvol_grp8086/subvol5699/b4941f0d-5ada-4968-92ee-06282fd823f1", 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: "pool_namespace": "", 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "complete", 2023-11-24T08:11:06.511 INFO:teuthology.orchestra.run.smithi106.stdout: "type": "subvolume", 2023-11-24T08:11:06.512 INFO:teuthology.orchestra.run.smithi106.stdout: "uid": 0 2023-11-24T08:11:06.512 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:11:06.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffb7406af60 msgr2=0x7ffb7406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffb7406af60 0x7ffb7406d420 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ffb8c00a750 tx=0x7ffb8c008040 comp rx=0 tx=0).stop 2023-11-24T08:11:06.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 msgr2=0x7ffb9015f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 0x7ffb9015f400 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffb88036040 tx=0x7ffb88004a40 comp rx=0 tx=0).stop 2023-11-24T08:11:06.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 shutdown_connections 2023-11-24T08:11:06.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffb9015c7f0 0x7ffb9015f400 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb900cf450 0x7ffb9015eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffb7406af60 0x7ffb7406d420 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 --2- 172.21.15.106:0/1177064770 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffb900cdfb0 0x7ffb9015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 >> 172.21.15.106:0/1177064770 conn(0x7ffb9006a920 msgr2=0x7ffb900af2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:06.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 shutdown_connections 2023-11-24T08:11:06.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.509+0000 7ffb97d6f640 1 -- 172.21.15.106:0/1177064770 wait complete. 2023-11-24T08:11:06.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:06.534 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo grep SNAP_METADATA_subvol_snap1401 ./volumes/subvol_grp8086/subvol5699/.meta) 2023-11-24T08:11:06.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:11:06.638 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol5699 subvol_grp8086 2023-11-24T08:11:06.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.850+0000 7f5213fff640 1 Processor -- start 2023-11-24T08:11:06.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.850+0000 7f5213fff640 1 -- start start 2023-11-24T08:11:06.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.851+0000 7f5213fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f52140d0af0 0x7f521406ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.851+0000 7f5213fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52140cff80 con 0x7f52140999d0 2023-11-24T08:11:06.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.851+0000 7f5213fff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52140d0410 con 0x7f5214070110 2023-11-24T08:11:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.851+0000 7f5213fff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52140b0240 con 0x7f52140d0af0 2023-11-24T08:11:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.852+0000 7f5212ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f52140999d0 0x7f5214099db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59198/0 (socket says 172.21.15.106:59198) 2023-11-24T08:11:06.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.852+0000 7f5212ffd640 1 -- 172.21.15.106:0/4169456697 learned_addr learned my addr 172.21.15.106:0/4169456697 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:06.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.852+0000 7f52137fe640 1 --2- 172.21.15.106:0/4169456697 >> v2:172.21.15.156:3300/0 conn(0x7f52140d0af0 0x7f521406ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.852+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133725722 0 0) 0x7f52140cff80 con 0x7f52140999d0 2023-11-24T08:11:06.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.852+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5200003520 con 0x7f52140999d0 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1916006121 0 0) 0x7f5200003520 con 0x7f52140999d0 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 >> v2:172.21.15.156:3300/0 conn(0x7f52140d0af0 msgr2=0x7f521406ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 --2- 172.21.15.106:0/4169456697 >> v2:172.21.15.156:3300/0 conn(0x7f52140d0af0 0x7f521406ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 >> v1:172.21.15.156:6789/0 conn(0x7f5214070110 legacy=0x7f521406e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140b3680 con 0x7f52140999d0 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f52040019f0 con 0x7f52140999d0 2023-11-24T08:11:06.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 >> v1:172.21.15.106:6789/0 conn(0x7f52140999d0 legacy=0x7f5214099db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52000039c0 0x7f5200003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f5200004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52000051d0 0x7f520000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140b0240 con 0x7f52000047c0 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520000c120 con 0x7f52000039c0 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520000c2e0 con 0x7f52000051d0 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f52127fc640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f5200004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.853+0000 7f5212ffd640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52000039c0 0x7f5200003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52137fe640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52000051d0 0x7f520000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52127fc640 1 -- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52000051d0 msgr2=0x7f520000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52127fc640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f52000051d0 0x7f520000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52127fc640 1 -- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52000039c0 msgr2=0x7f5200003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52127fc640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52000039c0 0x7f5200003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52127fc640 1 -- 172.21.15.106:0/4169456697 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5200003520 con 0x7f52000047c0 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.854+0000 7f52137fe640 1 -- 172.21.15.106:0/4169456697 reap_dead start 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.855+0000 7f52127fc640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f5200004c40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5214070580 tx=0x7f52000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b311a4334a229642 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.855+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc0052d0 con 0x7f52000047c0 2023-11-24T08:11:06.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.855+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51fc02e050 con 0x7f52000047c0 2023-11-24T08:11:06.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.855+0000 7f5211ffb640 1 -- 172.21.15.106:0/4169456697 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51fc003830 con 0x7f52000047c0 2023-11-24T08:11:06.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.855+0000 7f5213fff640 1 -- 172.21.15.106:0/4169456697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 msgr2=0x7f5200004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.855+0000 7f5213fff640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f5200004c40 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5214070580 tx=0x7f52000057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.856+0000 7f5213fff640 1 -- 172.21.15.106:0/4169456697 shutdown_connections 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.856+0000 7f5213fff640 1 --2- 172.21.15.106:0/4169456697 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f5200004c40 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.856+0000 7f5213fff640 1 -- 172.21.15.106:0/4169456697 >> 172.21.15.106:0/4169456697 conn(0x7f52140b2ac0 msgr2=0x7f52140b2ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.856+0000 7f5213fff640 1 -- 172.21.15.106:0/4169456697 shutdown_connections 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.856+0000 7f5213fff640 1 -- 172.21.15.106:0/4169456697 wait complete. 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.857+0000 7f5213fff640 1 Processor -- start 2023-11-24T08:11:06.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 -- start start 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f52140b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5214157560 0x7f52140c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52140c7230 0x7f52140c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c6da0 con 0x7f52000047c0 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c64b0 con 0x7f52140c7230 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.858+0000 7f5213fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52140c5a50 con 0x7f5214157560 2023-11-24T08:11:06.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f52140b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f52140b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41370/0 (socket says 172.21.15.106:41370) 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 -- 172.21.15.106:0/3967819988 learned_addr learned my addr 172.21.15.106:0/3967819988 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f52137fe640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52140c7230 0x7f52140c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 -- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5214157560 msgr2=0x7f52140c5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f52127fc640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5214157560 0x7f52140c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5214157560 0x7f52140c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 -- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52140c7230 msgr2=0x7f52140c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52140c7230 0x7f52140c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 -- 172.21.15.106:0/3967819988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52140c6750 con 0x7f52000047c0 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f52137fe640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52140c7230 0x7f52140c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.859+0000 7f5212ffd640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f52140b53e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5204007a10 tx=0x7f520400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.860+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f520400ee50 con 0x7f52000047c0 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.860+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204001990 con 0x7f52000047c0 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.860+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204013660 con 0x7f52000047c0 2023-11-24T08:11:06.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.860+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52140c6a30 con 0x7f52000047c0 2023-11-24T08:11:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.860+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52141579e0 con 0x7f52000047c0 2023-11-24T08:11:06.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.861+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521404e840 con 0x7f52000047c0 2023-11-24T08:11:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.862+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7f52040137c0 con 0x7f52000047c0 2023-11-24T08:11:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.863+0000 7f520bfff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f51f406af60 0x7f51f406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.863+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5204089ed0 con 0x7f52000047c0 2023-11-24T08:11:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.863+0000 7f52127fc640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f51f406af60 0x7f51f406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:06.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.864+0000 7f52127fc640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f51f406af60 0x7f51f406d420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51fc033000 tx=0x7f51fc009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:06.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:06.867+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5204054dd0 con 0x7f52000047c0 2023-11-24T08:11:07.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.085+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol5699", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7f52140aff40 con 0x7f51f406af60 2023-11-24T08:11:07.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.121+0000 7f520bfff640 1 -- 172.21.15.106:0/3967819988 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f51fc0054d0 con 0x7f51f406af60 2023-11-24T08:11:07.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f51f406af60 msgr2=0x7f51f406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f51f406af60 0x7f51f406d420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f51fc033000 tx=0x7f51fc009030 comp rx=0 tx=0).stop 2023-11-24T08:11:07.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 msgr2=0x7f52140b53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f52140b53e0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5204007a10 tx=0x7f520400e530 comp rx=0 tx=0).stop 2023-11-24T08:11:07.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 shutdown_connections 2023-11-24T08:11:07.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52140c7230 0x7f52140c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f51f406af60 0x7f51f406d420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5214157560 0x7f52140c5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 --2- 172.21.15.106:0/3967819988 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52000047c0 0x7f52140b53e0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 >> 172.21.15.106:0/3967819988 conn(0x7f52140b2ac0 msgr2=0x7f52140b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:07.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 shutdown_connections 2023-11-24T08:11:07.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.124+0000 7f5213fff640 1 -- 172.21.15.106:0/3967819988 wait complete. 2023-11-24T08:11:07.130 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.128+0000 7f67ad1c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.130 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.128+0000 7f67ad1c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.131 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.128+0000 7f67ad1c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.131 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.128+0000 7f67ad1c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.131 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.128+0000 7f67ad1c5640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.147 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp8086 2023-11-24T08:11:07.153 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.150+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.154 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.150+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.154 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.150+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.154 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.150+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.154 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:07.150+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:07.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.331+0000 7fc3f7c44640 1 Processor -- start 2023-11-24T08:11:07.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.331+0000 7fc3f7c44640 1 -- start start 2023-11-24T08:11:07.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.333+0000 7fc3f7c44640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 0x7fc3f00cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.333+0000 7fc3f7c44640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f00cc3a0 con 0x7fc3f00cbf20 2023-11-24T08:11:07.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.333+0000 7fc3f7c44640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3f015eea0 con 0x7fc3f00c59b0 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.333+0000 7fc3f7c44640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015b820 con 0x7fc3f015b440 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.333+0000 7fc3f51df640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc3f00cbf20 0x7fc3f00c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59212/0 (socket says 172.21.15.106:59212) 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.333+0000 7fc3f51df640 1 -- 172.21.15.106:0/256344273 learned_addr learned my addr 172.21.15.106:0/256344273 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706641529 0 0) 0x7fc3e0001110 con 0x7fc3f00cbf20 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3dc003520 con 0x7fc3f00cbf20 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f59e0640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 0x7fc3f00cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3081454384 0 0) 0x7fc3dc003520 con 0x7fc3f00cbf20 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 msgr2=0x7fc3f00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 0x7fc3f00cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 >> v1:172.21.15.156:6789/0 conn(0x7fc3f00c59b0 legacy=0x7fc3f006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00abf30 con 0x7fc3f00cbf20 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc3e0001800 con 0x7fc3f00cbf20 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 >> v1:172.21.15.106:6789/0 conn(0x7fc3f00cbf20 legacy=0x7fc3f00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3dc0039a0 0x7fc3dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 0x7fc3dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3dc004d00 0x7fc3dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f015b820 con 0x7fc3dc004d00 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc00bb60 con 0x7fc3dc0039a0 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.334+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3dc00bd20 con 0x7fc3f015b440 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f61e1640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3dc004d00 0x7fc3dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f59e0640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3dc0039a0 0x7fc3dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f51df640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 0x7fc3dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f61e1640 1 -- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 msgr2=0x7fc3dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:07.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f61e1640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f015b440 0x7fc3dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f61e1640 1 -- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3dc0039a0 msgr2=0x7fc3dc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:07.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f61e1640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3dc0039a0 0x7fc3dc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.335+0000 7fc3f61e1640 1 -- 172.21.15.106:0/256344273 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc003520 con 0x7fc3dc004d00 2023-11-24T08:11:07.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.336+0000 7fc3f61e1640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3dc004d00 0x7fc3dc005180 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8001e50 tx=0x7fc3dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dd72d029c3da742 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:07.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.336+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8003b70 con 0x7fc3dc004d00 2023-11-24T08:11:07.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.336+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e8003cd0 con 0x7fc3dc004d00 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.336+0000 7fc3f49de640 1 -- 172.21.15.106:0/256344273 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8005310 con 0x7fc3dc004d00 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f7c44640 1 -- 172.21.15.106:0/256344273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3dc004d00 msgr2=0x7fc3dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/256344273 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3dc004d00 0x7fc3dc005180 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8001e50 tx=0x7fc3dc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f61e1640 1 -- 172.21.15.106:0/256344273 reap_dead start 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f7c44640 1 -- 172.21.15.106:0/256344273 shutdown_connections 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f7c44640 1 -- 172.21.15.106:0/256344273 >> 172.21.15.106:0/256344273 conn(0x7fc3f00b0c50 msgr2=0x7fc3f00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f7c44640 1 -- 172.21.15.106:0/256344273 shutdown_connections 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.337+0000 7fc3f7c44640 1 -- 172.21.15.106:0/256344273 wait complete. 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.339+0000 7fc3f7c44640 1 Processor -- start 2023-11-24T08:11:07.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.339+0000 7fc3f7c44640 1 -- start start 2023-11-24T08:11:07.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f7c44640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 0x7fc3f009a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f7c44640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f009a920 0x7fc3f006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f7c44640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f00cdea0 0x7fc3f006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f7c44640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f009ba20 con 0x7fc3f01569f0 2023-11-24T08:11:07.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f7c44640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f009ada0 con 0x7fc3f009a920 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f7c44640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f009b080 con 0x7fc3f00cdea0 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f59e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 0x7fc3f009a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f59e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 0x7fc3f009a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41438/0 (socket says 172.21.15.106:41438) 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.340+0000 7fc3f59e0640 1 -- 172.21.15.106:0/365388692 learned_addr learned my addr 172.21.15.106:0/365388692 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f61e1640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f00cdea0 0x7fc3f006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f51df640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f009a920 0x7fc3f006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f59e0640 1 -- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f00cdea0 msgr2=0x7fc3f006d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f59e0640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f00cdea0 0x7fc3f006d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f59e0640 1 -- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f009a920 msgr2=0x7fc3f006cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f59e0640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f009a920 0x7fc3f006cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f59e0640 1 -- 172.21.15.106:0/365388692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f009b360 con 0x7fc3f01569f0 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f61e1640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f00cdea0 0x7fc3f006d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f59e0640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 0x7fc3f009a3e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00f580 tx=0x7fc3ec00fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec014870 con 0x7fc3f01569f0 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3ec0149d0 con 0x7fc3f01569f0 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec014d80 con 0x7fc3f01569f0 2023-11-24T08:11:07.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f009b640 con 0x7fc3f01569f0 2023-11-24T08:11:07.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.341+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f0156fb0 con 0x7fc3f01569f0 2023-11-24T08:11:07.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.342+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f00b23f0 con 0x7fc3f01569f0 2023-11-24T08:11:07.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.343+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 7) v1 ==== 89333+0+0 (secure 0 0 0) 0x7fc3ec015410 con 0x7fc3f01569f0 2023-11-24T08:11:07.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.345+0000 7fc3e67fc640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3d406af60 0x7fc3d406d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.345+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3ec08a300 con 0x7fc3f01569f0 2023-11-24T08:11:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.345+0000 7fc3f51df640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3d406af60 0x7fc3d406d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:07.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.346+0000 7fc3f51df640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3d406af60 0x7fc3d406d420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0072630 tx=0x7fc3e0000f70 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:07.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.349+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3ec055180 con 0x7fc3f01569f0 2023-11-24T08:11:07.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.395+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc3ec055550 con 0x7fc3f01569f0 2023-11-24T08:11:07.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.572+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp8086", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3f015f6b0 con 0x7fc3d406af60 2023-11-24T08:11:07.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.579+0000 7fc3e67fc640 1 -- 172.21.15.106:0/365388692 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc3e000f7c0 con 0x7fc3d406af60 2023-11-24T08:11:07.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.581+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3d406af60 msgr2=0x7fc3d406d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.581+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3d406af60 0x7fc3d406d420 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc3f0072630 tx=0x7fc3e0000f70 comp rx=0 tx=0).stop 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.581+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 msgr2=0x7fc3f009a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.581+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 0x7fc3f009a3e0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00f580 tx=0x7fc3ec00fa40 comp rx=0 tx=0).stop 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 shutdown_connections 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3d406af60 0x7fc3d406d420 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f00cdea0 0x7fc3f006d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f009a920 0x7fc3f006cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 --2- 172.21.15.106:0/365388692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f01569f0 0x7fc3f009a3e0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 >> 172.21.15.106:0/365388692 conn(0x7fc3f00b0c50 msgr2=0x7fc3f00ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 shutdown_connections 2023-11-24T08:11:07.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:07.582+0000 7fc3f7c44640 1 -- 172.21.15.106:0/365388692 wait complete. 2023-11-24T08:11:07.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:07.603 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T08:11:07.665 INFO:teuthology.orchestra.run.smithi106.stdout:2 2023-11-24T08:11:07.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:07.667 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rm -f ./volumes/subvol_grp8086/subvol5699/.meta.stale_snap_section) 2023-11-24T08:11:07.754 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:11:07.755 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:11:07.863 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:11:08.006 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:11:08.006 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:08.007 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:08.066 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:08.066 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:08.119 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.322+0000 7f4d54ab4640 1 Processor -- start 2023-11-24T08:11:08.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.322+0000 7f4d54ab4640 1 -- start start 2023-11-24T08:11:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.323+0000 7f4d54ab4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d500b2250 0x7f4d5015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.323+0000 7f4d54ab4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4d500b2630 0x7f4d500ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.323+0000 7f4d54ab4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d5015ab80 con 0x7f4d500b2aa0 2023-11-24T08:11:08.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.323+0000 7f4d54ab4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5015a990 con 0x7f4d500b2250 2023-11-24T08:11:08.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.323+0000 7f4d54ab4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d5015e680 con 0x7f4d500b2630 2023-11-24T08:11:08.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4d500b2250 0x7f4d5015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4ed9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4d500b2aa0 0x7f4d5015b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59220/0 (socket says 172.21.15.106:59220) 2023-11-24T08:11:08.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4ed9d640 1 -- 172.21.15.106:0/1516500006 learned_addr learned my addr 172.21.15.106:0/1516500006 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4e59c640 1 -- 172.21.15.106:0/1516500006 >> v2:172.21.15.156:3300/0 conn(0x7f4d500b2630 msgr2=0x7f4d500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4e59c640 1 --2- 172.21.15.106:0/1516500006 >> v2:172.21.15.156:3300/0 conn(0x7f4d500b2630 0x7f4d500ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4e59c640 1 -- 172.21.15.106:0/1516500006 >> v1:172.21.15.106:6789/0 conn(0x7f4d500b2aa0 legacy=0x7f4d5015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.324+0000 7f4d4e59c640 1 -- 172.21.15.106:0/1516500006 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5015e960 con 0x7f4d500b2250 2023-11-24T08:11:08.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.325+0000 7f4d4e59c640 1 --2- 172.21.15.106:0/1516500006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2250 0x7f4d5015df80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4d500ae5c0 tx=0x7f4d4002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3244edc5cd99c419 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.325+0000 7f4d4d59a640 1 -- 172.21.15.106:0/1516500006 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4002ee50 con 0x7f4d500b2250 2023-11-24T08:11:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.325+0000 7f4d4d59a640 1 -- 172.21.15.106:0/1516500006 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d40036440 con 0x7f4d500b2250 2023-11-24T08:11:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.325+0000 7f4d4d59a640 1 -- 172.21.15.106:0/1516500006 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d400367d0 con 0x7f4d500b2250 2023-11-24T08:11:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1516500006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2250 msgr2=0x7f4d5015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1516500006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2250 0x7f4d5015df80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4d500ae5c0 tx=0x7f4d4002e0a0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1516500006 shutdown_connections 2023-11-24T08:11:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1516500006 >> v2:172.21.15.156:3300/0 conn(0x7f4d500b2630 0x7f4d500ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1516500006 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2250 0x7f4d5015df80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1516500006 >> 172.21.15.106:0/1516500006 conn(0x7f4d5006a920 msgr2=0x7f4d5006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1516500006 shutdown_connections 2023-11-24T08:11:08.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.326+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1516500006 wait complete. 2023-11-24T08:11:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.328+0000 7f4d54ab4640 1 Processor -- start 2023-11-24T08:11:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.328+0000 7f4d54ab4640 1 -- start start 2023-11-24T08:11:08.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d54ab4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d500b2250 0x7f4d50155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d54ab4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 0x7f4d500b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d54ab4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d500b2aa0 0x7f4d50071880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d54ab4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d5014dc20 con 0x7f4d500b2630 2023-11-24T08:11:08.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d54ab4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d501558f0 con 0x7f4d500b2250 2023-11-24T08:11:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d54ab4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d50155bd0 con 0x7f4d500b2aa0 2023-11-24T08:11:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d4dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 0x7f4d500b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d4dd9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 0x7f4d500b0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41458/0 (socket says 172.21.15.106:41458) 2023-11-24T08:11:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.329+0000 7f4d4dd9b640 1 -- 172.21.15.106:0/1904376690 learned_addr learned my addr 172.21.15.106:0/1904376690 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:08.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d4dd9b640 1 -- 172.21.15.106:0/1904376690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d500b2aa0 msgr2=0x7f4d50071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d4dd9b640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d500b2aa0 0x7f4d50071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d4dd9b640 1 -- 172.21.15.106:0/1904376690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d500b2250 msgr2=0x7f4d50155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d4dd9b640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d500b2250 0x7f4d50155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d4dd9b640 1 -- 172.21.15.106:0/1904376690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d4000b6a0 con 0x7f4d500b2630 2023-11-24T08:11:08.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d4dd9b640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 0x7f4d500b0d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400b4a0 tx=0x7f4d4400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:08.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44010610 con 0x7f4d500b2630 2023-11-24T08:11:08.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d44010770 con 0x7f4d500b2630 2023-11-24T08:11:08.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d44010b20 con 0x7f4d500b2630 2023-11-24T08:11:08.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d50155eb0 con 0x7f4d500b2630 2023-11-24T08:11:08.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.330+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d50156470 con 0x7f4d500b2630 2023-11-24T08:11:08.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.331+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d50156690 con 0x7f4d500b2630 2023-11-24T08:11:08.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.332+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f4d44010c80 con 0x7f4d500b2630 2023-11-24T08:11:08.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.333+0000 7f4d3effd640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4d3006b080 0x7f4d3006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.333+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d44089750 con 0x7f4d500b2630 2023-11-24T08:11:08.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.333+0000 7f4d4e59c640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4d3006b080 0x7f4d3006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.334+0000 7f4d4e59c640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4d3006b080 0x7f4d3006d540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d40038000 tx=0x7f4d40030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:08.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.338+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4d44054400 con 0x7f4d500b2630 2023-11-24T08:11:08.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.535+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4d50067470 con 0x7f4d500b2630 2023-11-24T08:11:08.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.538+0000 7f4d3effd640 1 -- 172.21.15.106:0/1904376690 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+16561 (secure 0 0 0) 0x7f4d44057a20 con 0x7f4d500b2630 2023-11-24T08:11:08.541 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:08.542 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":48,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:10:42.399599+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"48","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:08.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4d3006b080 msgr2=0x7f4d3006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4d3006b080 0x7f4d3006d540 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4d40038000 tx=0x7f4d40030000 comp rx=0 tx=0).stop 2023-11-24T08:11:08.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 msgr2=0x7f4d500b0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 0x7f4d500b0d60 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f4d4400b4a0 tx=0x7f4d4400b960 comp rx=0 tx=0).stop 2023-11-24T08:11:08.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 shutdown_connections 2023-11-24T08:11:08.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4d3006b080 0x7f4d3006d540 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4d500b2aa0 0x7f4d50071880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4d500b2630 0x7f4d500b0d60 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 --2- 172.21.15.106:0/1904376690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d500b2250 0x7f4d50155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 >> 172.21.15.106:0/1904376690 conn(0x7f4d5006a920 msgr2=0x7f4d5015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:08.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 shutdown_connections 2023-11-24T08:11:08.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.541+0000 7f4d54ab4640 1 -- 172.21.15.106:0/1904376690 wait complete. 2023-11-24T08:11:08.561 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:10:38.577975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:10:39.853606+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '48', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:08.561 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.778+0000 7f082ea9b640 1 Processor -- start 2023-11-24T08:11:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.778+0000 7f082ea9b640 1 -- start start 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082ea9b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f082806e160 0x7f08280ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082ea9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08280cd110 0x7f0828070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082ea9b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0828161a10 con 0x7f0828146eb0 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082ea9b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08280cd580 con 0x7f08280cd110 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082ea9b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0828161cf0 con 0x7f082806e160 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082d038640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08280cd110 0x7f0828070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082c837640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0828146eb0 0x7f0828161150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59234/0 (socket says 172.21.15.106:59234) 2023-11-24T08:11:08.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.780+0000 7f082c837640 1 -- 172.21.15.106:0/3370102120 learned_addr learned my addr 172.21.15.106:0/3370102120 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:08.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.781+0000 7f0827fff640 1 --2- 172.21.15.106:0/3370102120 >> v2:172.21.15.156:3300/0 conn(0x7f082806e160 0x7f08280ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.781+0000 7f082d038640 1 -- 172.21.15.106:0/3370102120 >> v2:172.21.15.156:3300/0 conn(0x7f082806e160 msgr2=0x7f08280ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:08.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.781+0000 7f082d038640 1 --2- 172.21.15.106:0/3370102120 >> v2:172.21.15.156:3300/0 conn(0x7f082806e160 0x7f08280ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.781+0000 7f082d038640 1 -- 172.21.15.106:0/3370102120 >> v1:172.21.15.106:6789/0 conn(0x7f0828146eb0 legacy=0x7f0828161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.781+0000 7f082d038640 1 -- 172.21.15.106:0/3370102120 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0828161fd0 con 0x7f08280cd110 2023-11-24T08:11:08.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.781+0000 7f08277fe640 1 -- 172.21.15.106:0/3370102120 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 552157898 0 0) 0x7f0828161a10 con 0x7f0828146eb0 2023-11-24T08:11:08.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.782+0000 7f082d038640 1 --2- 172.21.15.106:0/3370102120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f0828070e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f082806eab0 tx=0x7f082002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd389ee4abd55d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:08.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.782+0000 7f08277fe640 1 -- 172.21.15.106:0/3370102120 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082002cc80 con 0x7f08280cd110 2023-11-24T08:11:08.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.782+0000 7f08277fe640 1 -- 172.21.15.106:0/3370102120 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082002cde0 con 0x7f08280cd110 2023-11-24T08:11:08.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.782+0000 7f08277fe640 1 -- 172.21.15.106:0/3370102120 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0820034660 con 0x7f08280cd110 2023-11-24T08:11:08.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 -- 172.21.15.106:0/3370102120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 msgr2=0x7f0828070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 --2- 172.21.15.106:0/3370102120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f0828070e40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f082806eab0 tx=0x7f082002bed0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 -- 172.21.15.106:0/3370102120 shutdown_connections 2023-11-24T08:11:08.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 --2- 172.21.15.106:0/3370102120 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f0828070e40 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 --2- 172.21.15.106:0/3370102120 >> v2:172.21.15.156:3300/0 conn(0x7f082806e160 0x7f08280ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 -- 172.21.15.106:0/3370102120 >> 172.21.15.106:0/3370102120 conn(0x7f08280b0c50 msgr2=0x7f08280b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:08.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 -- 172.21.15.106:0/3370102120 shutdown_connections 2023-11-24T08:11:08.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.783+0000 7f082ea9b640 1 -- 172.21.15.106:0/3370102120 wait complete. 2023-11-24T08:11:08.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.785+0000 7f082ea9b640 1 Processor -- start 2023-11-24T08:11:08.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.785+0000 7f082ea9b640 1 -- start start 2023-11-24T08:11:08.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f082ea9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082806e160 0x7f08281f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f082ea9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f08280acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f082ea9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0828146eb0 0x7f08280ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f082ea9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0828141ae0 con 0x7f08280cd110 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f082ea9b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082815fea0 con 0x7f082806e160 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f082ea9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08281f71f0 con 0x7f0828146eb0 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.786+0000 7f0827fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f08280acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f08280acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41480/0 (socket says 172.21.15.106:41480) 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 -- 172.21.15.106:0/4130743386 learned_addr learned my addr 172.21.15.106:0/4130743386 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:08.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f082d038640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0828146eb0 0x7f08280ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f082c837640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082806e160 0x7f08281f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 -- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0828146eb0 msgr2=0x7f08280ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0828146eb0 0x7f08280ad4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 -- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082806e160 msgr2=0x7f08281f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082806e160 0x7f08281f2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 -- 172.21.15.106:0/4130743386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0820009a70 con 0x7f08280cd110 2023-11-24T08:11:08.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.787+0000 7f0827fff640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f08280acf80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f081400d800 tx=0x7f081400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:08.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.788+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400b6c0 con 0x7f08280cd110 2023-11-24T08:11:08.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.788+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081400b820 con 0x7f08280cd110 2023-11-24T08:11:08.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.788+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081400bbb0 con 0x7f08280cd110 2023-11-24T08:11:08.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.788+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0828141cf0 con 0x7f08280cd110 2023-11-24T08:11:08.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.788+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08281422b0 con 0x7f08280cd110 2023-11-24T08:11:08.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.789+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828142440 con 0x7f08280cd110 2023-11-24T08:11:08.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.790+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f081400bd10 con 0x7f08280cd110 2023-11-24T08:11:08.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.791+0000 7f08257fa640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f080c06b0b0 0x7f080c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:08.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.791+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0814089ff0 con 0x7f08280cd110 2023-11-24T08:11:08.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.791+0000 7f082c837640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f080c06b0b0 0x7f080c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:08.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.792+0000 7f082c837640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f080c06b0b0 0x7f080c06d570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f081c000f60 tx=0x7f081c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:08.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.796+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0814054ca0 con 0x7f08280cd110 2023-11-24T08:11:08.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.984+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f082804e8b0 con 0x7f08280cd110 2023-11-24T08:11:08.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.986+0000 7f08257fa640 1 -- 172.21.15.106:0/4130743386 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v48) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08140582c0 con 0x7f08280cd110 2023-11-24T08:11:08.988 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f080c06b0b0 msgr2=0x7f080c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f080c06b0b0 0x7f080c06d570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f081c000f60 tx=0x7f081c009040 comp rx=0 tx=0).stop 2023-11-24T08:11:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 msgr2=0x7f08280acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f08280acf80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f081400d800 tx=0x7f081400dcc0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 shutdown_connections 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f080c06b0b0 0x7f080c06d570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0828146eb0 0x7f08280ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08280cd110 0x7f08280acf80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 --2- 172.21.15.106:0/4130743386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082806e160 0x7f08281f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 >> 172.21.15.106:0/4130743386 conn(0x7f08280b0c50 msgr2=0x7f0828145ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.988+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 shutdown_connections 2023-11-24T08:11:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:08.989+0000 7f082ea9b640 1 -- 172.21.15.106:0/4130743386 wait complete. 2023-11-24T08:11:09.008 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:11:09.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.222+0000 7f5122f83640 1 Processor -- start 2023-11-24T08:11:09.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.222+0000 7f5122f83640 1 -- start start 2023-11-24T08:11:09.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.224+0000 7f5122f83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c161060 0x7f511c161440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.224+0000 7f5122f83640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f511c0b3330 0x7f511c15da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.224+0000 7f5122f83640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f511c06dcf0 con 0x7f511c0b3790 2023-11-24T08:11:09.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.224+0000 7f5122f83640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f511c06ec10 con 0x7f511c0b3330 2023-11-24T08:11:09.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.224+0000 7f5122f83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c161980 con 0x7f511c161060 2023-11-24T08:11:09.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.224+0000 7f511bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f511c0b3330 0x7f511c15da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f511c0b3330 0x7f511c15da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41486/0 (socket says 172.21.15.106:41486) 2023-11-24T08:11:09.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511bfff640 1 -- 172.21.15.106:0/1278389332 learned_addr learned my addr 172.21.15.106:0/1278389332 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:09.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f5120d1f640 1 --2- 172.21.15.106:0/1278389332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c161060 0x7f511c161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511bfff640 1 -- 172.21.15.106:0/1278389332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c161060 msgr2=0x7f511c161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:09.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511bfff640 1 --2- 172.21.15.106:0/1278389332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c161060 0x7f511c161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511bfff640 1 -- 172.21.15.106:0/1278389332 >> v1:172.21.15.106:6789/0 conn(0x7f511c0b3790 legacy=0x7f511c15dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511bfff640 1 -- 172.21.15.106:0/1278389332 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c161ec0 con 0x7f511c0b3330 2023-11-24T08:11:09.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.225+0000 7f511b7fe640 1 -- 172.21.15.106:0/1278389332 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718680332 0 0) 0x7f511c06dcf0 con 0x7f511c0b3790 2023-11-24T08:11:09.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.226+0000 7f511bfff640 1 --2- 172.21.15.106:0/1278389332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c0b3330 0x7f511c15da80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f510c009860 tx=0x7f510c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677f195b74b8d95b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:09.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f511b7fe640 1 -- 172.21.15.106:0/1278389332 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c031640 con 0x7f511c0b3330 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f511b7fe640 1 -- 172.21.15.106:0/1278389332 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510c0317a0 con 0x7f511c0b3330 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f511b7fe640 1 -- 172.21.15.106:0/1278389332 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510c031b30 con 0x7f511c0b3330 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 -- 172.21.15.106:0/1278389332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c0b3330 msgr2=0x7f511c15da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 --2- 172.21.15.106:0/1278389332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c0b3330 0x7f511c15da80 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f510c009860 tx=0x7f510c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 -- 172.21.15.106:0/1278389332 shutdown_connections 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 --2- 172.21.15.106:0/1278389332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c0b3330 0x7f511c15da80 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 --2- 172.21.15.106:0/1278389332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c161060 0x7f511c161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 -- 172.21.15.106:0/1278389332 >> 172.21.15.106:0/1278389332 conn(0x7f511c0cf900 msgr2=0x7f511c0cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:09.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 -- 172.21.15.106:0/1278389332 shutdown_connections 2023-11-24T08:11:09.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.227+0000 7f5122f83640 1 -- 172.21.15.106:0/1278389332 wait complete. 2023-11-24T08:11:09.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.229+0000 7f5122f83640 1 Processor -- start 2023-11-24T08:11:09.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.229+0000 7f5122f83640 1 -- start start 2023-11-24T08:11:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5122f83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c0b3330 0x7f511c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5122f83640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f511c0b3790 0x7f511c1f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5122f83640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 0x7f511c15e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5122f83640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c144fe0 con 0x7f511c161060 2023-11-24T08:11:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5122f83640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c15a340 con 0x7f511c0b3790 2023-11-24T08:11:09.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5122f83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1f6bc0 con 0x7f511c0b3330 2023-11-24T08:11:09.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5121520640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 0x7f511c15e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5121520640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 0x7f511c15e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41490/0 (socket says 172.21.15.106:41490) 2023-11-24T08:11:09.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.230+0000 7f5121520640 1 -- 172.21.15.106:0/1208864447 learned_addr learned my addr 172.21.15.106:0/1208864447 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:09.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f511bfff640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f511c0b3790 0x7f511c1f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5120d1f640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c0b3330 0x7f511c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5121520640 1 -- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c0b3330 msgr2=0x7f511c15cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5121520640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c0b3330 0x7f511c15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5121520640 1 -- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f511c0b3790 msgr2=0x7f511c1f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5121520640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f511c0b3790 0x7f511c1f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5121520640 1 -- 172.21.15.106:0/1208864447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c009470 con 0x7f511c161060 2023-11-24T08:11:09.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f511bfff640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f511c0b3790 0x7f511c1f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:09.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f5121520640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 0x7f511c15e280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f511400c710 tx=0x7f511400cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:09.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114003da0 con 0x7f511c161060 2023-11-24T08:11:09.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.231+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511400ce20 con 0x7f511c161060 2023-11-24T08:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.232+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c1f6ea0 con 0x7f511c161060 2023-11-24T08:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.232+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5114005290 con 0x7f511c161060 2023-11-24T08:11:09.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.232+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c1f7460 con 0x7f511c161060 2023-11-24T08:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.233+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c067990 con 0x7f511c161060 2023-11-24T08:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.234+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f51140053f0 con 0x7f511c161060 2023-11-24T08:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.234+0000 7f51197fa640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f510806b080 0x7f510806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.235+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51140892f0 con 0x7f511c161060 2023-11-24T08:11:09.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.235+0000 7f5120d1f640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f510806b080 0x7f510806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.235+0000 7f5120d1f640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f510806b080 0x7f510806d540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5110006fd0 tx=0x7f5110008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:09.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.239+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5114053fa0 con 0x7f511c161060 2023-11-24T08:11:09.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.434+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f511c145120 con 0x7f511c161060 2023-11-24T08:11:09.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.435+0000 7f51197fa640 1 -- 172.21.15.106:0/1208864447 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v48) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51140575c0 con 0x7f511c161060 2023-11-24T08:11:09.438 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:11:09.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.437+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f510806b080 msgr2=0x7f510806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.437+0000 7f5122f83640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f510806b080 0x7f510806d540 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5110006fd0 tx=0x7f5110008040 comp rx=0 tx=0).stop 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.437+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 msgr2=0x7f511c15e280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.437+0000 7f5122f83640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 0x7f511c15e280 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f511400c710 tx=0x7f511400cbd0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 shutdown_connections 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f511c161060 0x7f511c15e280 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f511c0b3790 0x7f511c1f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f510806b080 0x7f510806d540 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 --2- 172.21.15.106:0/1208864447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f511c0b3330 0x7f511c15cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 >> 172.21.15.106:0/1208864447 conn(0x7f511c0cf900 msgr2=0x7f511c1562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 shutdown_connections 2023-11-24T08:11:09.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.438+0000 7f5122f83640 1 -- 172.21.15.106:0/1208864447 wait complete. 2023-11-24T08:11:09.460 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:11:09.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.662+0000 7feae9748640 1 Processor -- start 2023-11-24T08:11:09.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.663+0000 7feae9748640 1 -- start start 2023-11-24T08:11:09.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae9748640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feae406de10 0x7feae415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae9748640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feae40c49f0 0x7feae40705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae9748640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae415ec90 con 0x7feae4072160 2023-11-24T08:11:09.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae9748640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40cd530 con 0x7feae40c49f0 2023-11-24T08:11:09.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae9748640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae40c4ed0 con 0x7feae406de10 2023-11-24T08:11:09.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feae40c49f0 0x7feae40705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7feae40c49f0 0x7feae40705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41502/0 (socket says 172.21.15.106:41502) 2023-11-24T08:11:09.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.664+0000 7feae37fe640 1 -- 172.21.15.106:0/4208919938 learned_addr learned my addr 172.21.15.106:0/4208919938 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:09.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.665+0000 7feae37fe640 1 -- 172.21.15.106:0/4208919938 >> v2:172.21.15.156:3300/0 conn(0x7feae406de10 msgr2=0x7feae415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:09.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.665+0000 7feae37fe640 1 --2- 172.21.15.106:0/4208919938 >> v2:172.21.15.156:3300/0 conn(0x7feae406de10 0x7feae415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.665+0000 7feae37fe640 1 -- 172.21.15.106:0/4208919938 >> v1:172.21.15.156:6789/0 conn(0x7feae4072160 legacy=0x7feae406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:09.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.665+0000 7feae37fe640 1 -- 172.21.15.106:0/4208919938 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae40c51b0 con 0x7feae40c49f0 2023-11-24T08:11:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.666+0000 7feae37fe640 1 --2- 172.21.15.106:0/4208919938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae40c49f0 0x7feae40705c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feae40cef60 tx=0x7feadc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b604321ec2df21db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.666+0000 7feae27fc640 1 -- 172.21.15.106:0/4208919938 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc02ca80 con 0x7feae40c49f0 2023-11-24T08:11:09.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.666+0000 7feae27fc640 1 -- 172.21.15.106:0/4208919938 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feadc02cbe0 con 0x7feae40c49f0 2023-11-24T08:11:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.666+0000 7feae27fc640 1 -- 172.21.15.106:0/4208919938 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feadc0343f0 con 0x7feae40c49f0 2023-11-24T08:11:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 -- 172.21.15.106:0/4208919938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae40c49f0 msgr2=0x7feae40705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 --2- 172.21.15.106:0/4208919938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae40c49f0 0x7feae40705c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7feae40cef60 tx=0x7feadc02bee0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 -- 172.21.15.106:0/4208919938 shutdown_connections 2023-11-24T08:11:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 --2- 172.21.15.106:0/4208919938 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae40c49f0 0x7feae40705c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 --2- 172.21.15.106:0/4208919938 >> v2:172.21.15.156:3300/0 conn(0x7feae406de10 0x7feae415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 -- 172.21.15.106:0/4208919938 >> 172.21.15.106:0/4208919938 conn(0x7feae40b0c50 msgr2=0x7feae40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:09.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 -- 172.21.15.106:0/4208919938 shutdown_connections 2023-11-24T08:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.667+0000 7feae9748640 1 -- 172.21.15.106:0/4208919938 wait complete. 2023-11-24T08:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.668+0000 7feae9748640 1 Processor -- start 2023-11-24T08:11:09.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.669+0000 7feae9748640 1 -- start start 2023-11-24T08:11:09.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.669+0000 7feae9748640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feae406de10 0x7feae40c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feae9748640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 0x7feae40c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feae9748640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feae40c49f0 0x7feae4159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feae9748640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae415b8a0 con 0x7feae4072160 2023-11-24T08:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feae9748640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae415bb80 con 0x7feae40c49f0 2023-11-24T08:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feae9748640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae415be60 con 0x7feae406de10 2023-11-24T08:11:09.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 0x7feae40c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 0x7feae40c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41514/0 (socket says 172.21.15.106:41514) 2023-11-24T08:11:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 -- 172.21.15.106:0/3425495536 learned_addr learned my addr 172.21.15.106:0/3425495536 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 -- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feae406de10 msgr2=0x7feae40c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feae406de10 0x7feae40c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 -- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feae40c49f0 msgr2=0x7feae4159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feae37fe640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feae40c49f0 0x7feae4159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feae40c49f0 0x7feae4159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.670+0000 7feadbfff640 1 -- 172.21.15.106:0/3425495536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc009b30 con 0x7feae4072160 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.671+0000 7feadbfff640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 0x7feae40c7390 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b4a0 tx=0x7feacc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.671+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc010610 con 0x7feae4072160 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.671+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc010770 con 0x7feae4072160 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.671+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae415c140 con 0x7feae4072160 2023-11-24T08:11:09.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.671+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc010b20 con 0x7feae4072160 2023-11-24T08:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.671+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae415c940 con 0x7feae4072160 2023-11-24T08:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.672+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae40c94a0 con 0x7feae4072160 2023-11-24T08:11:09.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.673+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7feacc010c80 con 0x7feae4072160 2023-11-24T08:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.674+0000 7feadb7fe640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac406b080 0x7feac406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.675+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feacc089700 con 0x7feae4072160 2023-11-24T08:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.675+0000 7feae2ffd640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac406b080 0x7feac406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:09.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.675+0000 7feae2ffd640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac406b080 0x7feac406d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fead4001560 tx=0x7fead4007990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:09.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.679+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feacc0543b0 con 0x7feae4072160 2023-11-24T08:11:09.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.867+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feae404e8b0 con 0x7feae4072160 2023-11-24T08:11:09.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.868+0000 7feadb7fe640 1 -- 172.21.15.106:0/3425495536 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v48) v1 ==== 100+0+11 (secure 0 0 0) 0x7feacc0579d0 con 0x7feae4072160 2023-11-24T08:11:09.871 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:11:09.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac406b080 msgr2=0x7feac406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac406b080 0x7feac406d540 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fead4001560 tx=0x7fead4007990 comp rx=0 tx=0).stop 2023-11-24T08:11:09.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 msgr2=0x7feae40c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 0x7feae40c7390 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7feacc00b4a0 tx=0x7feacc00b960 comp rx=0 tx=0).stop 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 shutdown_connections 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feae40c49f0 0x7feae4159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feae4072160 0x7feae40c7390 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac406b080 0x7feac406d540 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 --2- 172.21.15.106:0/3425495536 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feae406de10 0x7feae40c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 >> 172.21.15.106:0/3425495536 conn(0x7feae40b0c50 msgr2=0x7feae40af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.871+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 shutdown_connections 2023-11-24T08:11:09.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:09.872+0000 7feae9748640 1 -- 172.21.15.106:0/3425495536 wait complete. 2023-11-24T08:11:09.892 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:11:10.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.098+0000 7fc8e5261640 1 Processor -- start 2023-11-24T08:11:10.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.098+0000 7fc8e5261640 1 -- start start 2023-11-24T08:11:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8e5261640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc8e00b3330 0x7fc8e006a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8e5261640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e006dcf0 con 0x7fc8e0161060 2023-11-24T08:11:10.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8e5261640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e0161ec0 con 0x7fc8e00b3790 2023-11-24T08:11:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8e5261640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e006ec10 con 0x7fc8e00b3330 2023-11-24T08:11:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8deffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc8e0161060 0x7fc8e0161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59246/0 (socket says 172.21.15.106:59246) 2023-11-24T08:11:10.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8deffd640 1 -- 172.21.15.106:0/3859673338 learned_addr learned my addr 172.21.15.106:0/3859673338 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3826009250 0 0) 0x7fc8e006dcf0 con 0x7fc8e0161060 2023-11-24T08:11:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8cc003520 con 0x7fc8e0161060 2023-11-24T08:11:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.100+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/3859673338 >> v2:172.21.15.156:3300/0 conn(0x7fc8e00b3330 0x7fc8e006a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3018042876 0 0) 0x7fc8cc003520 con 0x7fc8e0161060 2023-11-24T08:11:10.105 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 >> v2:172.21.15.156:3300/0 conn(0x7fc8e00b3330 msgr2=0x7fc8e006a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 --2- 172.21.15.106:0/3859673338 >> v2:172.21.15.156:3300/0 conn(0x7fc8e00b3330 0x7fc8e006a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 >> v1:172.21.15.156:6789/0 conn(0x7fc8e00b3790 legacy=0x7fc8e006abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e006df90 con 0x7fc8e0161060 2023-11-24T08:11:10.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc8d00019f0 con 0x7fc8e0161060 2023-11-24T08:11:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 >> v1:172.21.15.106:6789/0 conn(0x7fc8e0161060 legacy=0x7fc8e0161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8cc0039c0 0x7fc8cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8cc005250 0x7fc8cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e006ec10 con 0x7fc8cc0047c0 2023-11-24T08:11:10.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00c1a0 con 0x7fc8cc0039c0 2023-11-24T08:11:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8cc00c360 con 0x7fc8cc005250 2023-11-24T08:11:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.101+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.108 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.102+0000 7fc8d67fc640 1 -- 172.21.15.106:0/3859673338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8cc005250 msgr2=0x7fc8cc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.102+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8cc005250 0x7fc8cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.102+0000 7fc8d67fc640 1 -- 172.21.15.106:0/3859673338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8cc0039c0 msgr2=0x7fc8cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.102+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8cc0039c0 0x7fc8cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.102+0000 7fc8d67fc640 1 -- 172.21.15.106:0/3859673338 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8cc003520 con 0x7fc8cc0047c0 2023-11-24T08:11:10.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.102+0000 7fc8df7fe640 1 -- 172.21.15.106:0/3859673338 reap_dead start 2023-11-24T08:11:10.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.103+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8cc004c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800ba10 tx=0x7fc8cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ef4ffc0f1cc092 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:10.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.103+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8005b90 con 0x7fc8cc0047c0 2023-11-24T08:11:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.103+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c8005cf0 con 0x7fc8cc0047c0 2023-11-24T08:11:10.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.103+0000 7fc8de7fc640 1 -- 172.21.15.106:0/3859673338 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c8006270 con 0x7fc8cc0047c0 2023-11-24T08:11:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.103+0000 7fc8e5261640 1 -- 172.21.15.106:0/3859673338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 msgr2=0x7fc8cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.103+0000 7fc8e5261640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8cc004c40 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fc8c800ba10 tx=0x7fc8cc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.104+0000 7fc8e5261640 1 -- 172.21.15.106:0/3859673338 shutdown_connections 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.104+0000 7fc8e5261640 1 --2- 172.21.15.106:0/3859673338 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8cc004c40 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.104+0000 7fc8e5261640 1 -- 172.21.15.106:0/3859673338 >> 172.21.15.106:0/3859673338 conn(0x7fc8e00cf900 msgr2=0x7fc8e00cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.104+0000 7fc8e5261640 1 -- 172.21.15.106:0/3859673338 shutdown_connections 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.104+0000 7fc8e5261640 1 -- 172.21.15.106:0/3859673338 wait complete. 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.105+0000 7fc8e5261640 1 Processor -- start 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.105+0000 7fc8e5261640 1 -- start start 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.106+0000 7fc8e5261640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8e0145140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.106+0000 7fc8e5261640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8e01f6d70 0x7fc8e01f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.106+0000 7fc8e5261640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8e006a950 0x7fc8e006add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.106+0000 7fc8e5261640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e014fdf0 con 0x7fc8cc0047c0 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.106+0000 7fc8e5261640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e006b310 con 0x7fc8e006a950 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.106+0000 7fc8e5261640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00cd760 con 0x7fc8e01f6d70 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8e0145140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8e0145140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41536/0 (socket says 172.21.15.106:41536) 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 -- 172.21.15.106:0/2227765668 learned_addr learned my addr 172.21.15.106:0/2227765668 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 -- 172.21.15.106:0/2227765668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8e01f6d70 msgr2=0x7fc8e01f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8e01f6d70 0x7fc8e01f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 -- 172.21.15.106:0/2227765668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8e006a950 msgr2=0x7fc8e006add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8e006a950 0x7fc8e006add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 -- 172.21.15.106:0/2227765668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0145680 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.107+0000 7fc8deffd640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8e0145140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000cb10 tx=0x7fc8d000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.108+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d000a9a0 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.108+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d000ab00 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.108+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0004e30 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.108+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e01f7830 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.108+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e00cd3f0 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.109+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0067990 con 0x7fc8cc0047c0 2023-11-24T08:11:10.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.110+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc8d00070d0 con 0x7fc8cc0047c0 2023-11-24T08:11:10.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.111+0000 7fc8d7fff640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc8c006b080 0x7fc8c006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.111+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8d0089320 con 0x7fc8cc0047c0 2023-11-24T08:11:10.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.111+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc8c006b080 0x7fc8c006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.112+0000 7fc8d67fc640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc8c006b080 0x7fc8c006d540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8036000 tx=0x7fc8c8031000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:10.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.115+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc8d0054060 con 0x7fc8cc0047c0 2023-11-24T08:11:10.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.320+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8e004e8b0 con 0x7fc8cc0047c0 2023-11-24T08:11:10.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.322+0000 7fc8d7fff640 1 -- 172.21.15.106:0/2227765668 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v48) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc8d0057680 con 0x7fc8cc0047c0 2023-11-24T08:11:10.324 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:11:10.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc8c006b080 msgr2=0x7fc8c006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc8c006b080 0x7fc8c006d540 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8036000 tx=0x7fc8c8031000 comp rx=0 tx=0).stop 2023-11-24T08:11:10.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 msgr2=0x7fc8e0145140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8e0145140 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fc8d000cb10 tx=0x7fc8d000a060 comp rx=0 tx=0).stop 2023-11-24T08:11:10.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 shutdown_connections 2023-11-24T08:11:10.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8e006a950 0x7fc8e006add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc8c006b080 0x7fc8c006d540 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc8e01f6d70 0x7fc8e01f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 --2- 172.21.15.106:0/2227765668 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc8cc0047c0 0x7fc8e0145140 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 >> 172.21.15.106:0/2227765668 conn(0x7fc8e00cf900 msgr2=0x7fc8e00af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:10.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 shutdown_connections 2023-11-24T08:11:10.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.324+0000 7fc8e5261640 1 -- 172.21.15.106:0/2227765668 wait complete. 2023-11-24T08:11:10.347 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:11:10.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:10.348 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T08:11:10.461 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:11:10.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:10.461 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:11:10.508 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:10.509 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:10.566 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:10.566 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:10.618 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:10.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.828+0000 7feb4e2c9640 1 Processor -- start 2023-11-24T08:11:10.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.828+0000 7feb4e2c9640 1 -- start start 2023-11-24T08:11:10.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.830+0000 7feb4e2c9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7feb480ce6c0 0x7feb480cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.830+0000 7feb4e2c9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb480cece0 con 0x7feb4815c7f0 2023-11-24T08:11:10.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.830+0000 7feb4e2c9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb4806d040 con 0x7feb480cf450 2023-11-24T08:11:10.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.830+0000 7feb4e2c9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb480cfe40 con 0x7feb480ce6c0 2023-11-24T08:11:10.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.830+0000 7feb47fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7feb4815c7f0 0x7feb4806db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59256/0 (socket says 172.21.15.106:59256) 2023-11-24T08:11:10.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.830+0000 7feb47fff640 1 -- 172.21.15.106:0/2977397892 learned_addr learned my addr 172.21.15.106:0/2977397892 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469711364 0 0) 0x7feb480cece0 con 0x7feb4815c7f0 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb34003520 con 0x7feb4815c7f0 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb4c866640 1 --2- 172.21.15.106:0/2977397892 >> v2:172.21.15.156:3300/0 conn(0x7feb480ce6c0 0x7feb480cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3469103067 0 0) 0x7feb34003520 con 0x7feb4815c7f0 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 >> v2:172.21.15.156:3300/0 conn(0x7feb480ce6c0 msgr2=0x7feb480cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 --2- 172.21.15.106:0/2977397892 >> v2:172.21.15.156:3300/0 conn(0x7feb480ce6c0 0x7feb480cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 >> v1:172.21.15.156:6789/0 conn(0x7feb480cf450 legacy=0x7feb480cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4806d320 con 0x7feb4815c7f0 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7feb380019f0 con 0x7feb4815c7f0 2023-11-24T08:11:10.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 >> v1:172.21.15.106:6789/0 conn(0x7feb4815c7f0 legacy=0x7feb4806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb34003970 0x7feb34003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb340047c0 0x7feb34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb34005180 0x7feb3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb480cfe40 con 0x7feb340047c0 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3400c0d0 con 0x7feb34005180 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.831+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3400c290 con 0x7feb34003970 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb3f7fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb340047c0 0x7feb34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb4c866640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb34005180 0x7feb3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb47fff640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb34003970 0x7feb34003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb3f7fe640 1 -- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb34003970 msgr2=0x7feb34003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb3f7fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb34003970 0x7feb34003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb3f7fe640 1 -- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb34005180 msgr2=0x7feb3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb3f7fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb34005180 0x7feb3400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb3f7fe640 1 -- 172.21.15.106:0/2977397892 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34003520 con 0x7feb340047c0 2023-11-24T08:11:10.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.832+0000 7feb4c866640 1 -- 172.21.15.106:0/2977397892 reap_dead start 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.833+0000 7feb3f7fe640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb340047c0 0x7feb34004c40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feb30001750 tx=0x7feb340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2119c7f142c4c3ea server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.833+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300052d0 con 0x7feb340047c0 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.833+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3002e050 con 0x7feb340047c0 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.833+0000 7feb477fe640 1 -- 172.21.15.106:0/2977397892 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30003830 con 0x7feb340047c0 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2977397892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb340047c0 msgr2=0x7feb34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb340047c0 0x7feb34004c40 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7feb30001750 tx=0x7feb340057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2977397892 shutdown_connections 2023-11-24T08:11:10.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2977397892 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb340047c0 0x7feb34004c40 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2977397892 >> 172.21.15.106:0/2977397892 conn(0x7feb4806a920 msgr2=0x7feb4806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:10.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2977397892 shutdown_connections 2023-11-24T08:11:10.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.834+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2977397892 wait complete. 2023-11-24T08:11:10.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.836+0000 7feb4e2c9640 1 Processor -- start 2023-11-24T08:11:10.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.836+0000 7feb4e2c9640 1 -- start start 2023-11-24T08:11:10.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4e2c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb340047c0 0x7feb480b2c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4e2c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb480cb9e0 0x7feb48154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4e2c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 0x7feb481411e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4e2c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb48154c20 con 0x7feb481547a0 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4e2c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb480b31a0 con 0x7feb480cb9e0 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4e2c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb481525e0 con 0x7feb340047c0 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4c866640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 0x7feb481411e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4c866640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 0x7feb481411e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41556/0 (socket says 172.21.15.106:41556) 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.837+0000 7feb4c866640 1 -- 172.21.15.106:0/2573191126 learned_addr learned my addr 172.21.15.106:0/2573191126 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb3f7fe640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb480cb9e0 0x7feb48154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4c866640 1 -- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb340047c0 msgr2=0x7feb480b2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4c866640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb340047c0 0x7feb480b2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4c866640 1 -- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb480cb9e0 msgr2=0x7feb48154260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4c866640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb480cb9e0 0x7feb48154260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4c866640 1 -- 172.21.15.106:0/2573191126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb48141dd0 con 0x7feb481547a0 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb3f7fe640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb480cb9e0 0x7feb48154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4c866640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 0x7feb481411e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feb480cca70 tx=0x7feb4000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40004440 con 0x7feb481547a0 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb4000fe70 con 0x7feb481547a0 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb40003850 con 0x7feb481547a0 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.838+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb48143e30 con 0x7feb481547a0 2023-11-24T08:11:10.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.839+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb480b2180 con 0x7feb481547a0 2023-11-24T08:11:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.840+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb48067470 con 0x7feb481547a0 2023-11-24T08:11:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.841+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7feb400039b0 con 0x7feb481547a0 2023-11-24T08:11:10.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.842+0000 7feb457fa640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb2806b080 0x7feb2806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:10.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.842+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7feb400898d0 con 0x7feb481547a0 2023-11-24T08:11:10.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.842+0000 7feb47fff640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb2806b080 0x7feb2806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:10.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.843+0000 7feb47fff640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb2806b080 0x7feb2806d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb38007c40 tx=0x7feb38009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:10.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:10.846+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feb40054550 con 0x7feb481547a0 2023-11-24T08:11:11.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.040+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb480cd730 con 0x7feb481547a0 2023-11-24T08:11:11.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.043+0000 7feb457fa640 1 -- 172.21.15.106:0/2573191126 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16669 (secure 0 0 0) 0x7feb40057b70 con 0x7feb481547a0 2023-11-24T08:11:11.046 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:11.046 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":49,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:10.401248+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb2806b080 msgr2=0x7feb2806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb2806b080 0x7feb2806d540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7feb38007c40 tx=0x7feb38009040 comp rx=0 tx=0).stop 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 msgr2=0x7feb481411e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 0x7feb481411e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7feb480cca70 tx=0x7feb4000f670 comp rx=0 tx=0).stop 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 shutdown_connections 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7feb481547a0 0x7feb481411e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb480cb9e0 0x7feb48154260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feb2806b080 0x7feb2806d540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 --2- 172.21.15.106:0/2573191126 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7feb340047c0 0x7feb480b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 >> 172.21.15.106:0/2573191126 conn(0x7feb4806a920 msgr2=0x7feb4806ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:11.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 shutdown_connections 2023-11-24T08:11:11.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.046+0000 7feb4e2c9640 1 -- 172.21.15.106:0/2573191126 wait complete. 2023-11-24T08:11:11.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:10:38.577975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:10:39.853606+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:11.068 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.282+0000 7f7c01ce1640 1 Processor -- start 2023-11-24T08:11:11.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.282+0000 7f7c01ce1640 1 -- start start 2023-11-24T08:11:11.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c01ce1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bfc0c27a0 0x7f7bfc0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c01ce1640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7bfc0ca940 0x7f7bfc0b13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c01ce1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bfc0cb960 con 0x7f7bfc0cb580 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c01ce1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc161e40 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c01ce1640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bfc162120 con 0x7f7bfc0ca940 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7bfbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bfc0c27a0 0x7f7bfc0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7bfbfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7bfc0c27a0 0x7f7bfc0b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41570/0 (socket says 172.21.15.106:41570) 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c00cdf640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7bfc0cb580 0x7f7bfc0c2090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59270/0 (socket says 172.21.15.106:59270) 2023-11-24T08:11:11.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7bfbfff640 1 -- 172.21.15.106:0/493807201 learned_addr learned my addr 172.21.15.106:0/493807201 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.284+0000 7f7c014e0640 1 --2- 172.21.15.106:0/493807201 >> v2:172.21.15.156:3300/0 conn(0x7f7bfc0ca940 0x7f7bfc0b13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.285+0000 7f7bfbfff640 1 -- 172.21.15.106:0/493807201 >> v2:172.21.15.156:3300/0 conn(0x7f7bfc0ca940 msgr2=0x7f7bfc0b13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.285+0000 7f7bfbfff640 1 --2- 172.21.15.106:0/493807201 >> v2:172.21.15.156:3300/0 conn(0x7f7bfc0ca940 0x7f7bfc0b13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.285+0000 7f7bfbfff640 1 -- 172.21.15.106:0/493807201 >> v1:172.21.15.106:6789/0 conn(0x7f7bfc0cb580 legacy=0x7f7bfc0c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.285+0000 7f7bfbfff640 1 -- 172.21.15.106:0/493807201 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bfc162400 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.285+0000 7f7bfb7fe640 1 -- 172.21.15.106:0/493807201 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1744636417 0 0) 0x7f7bfc0cb960 con 0x7f7bfc0cb580 2023-11-24T08:11:11.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.286+0000 7f7bfbfff640 1 --2- 172.21.15.106:0/493807201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc0b0e90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc161d30 tx=0x7f7be802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a343f0ccf9624b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:11.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.286+0000 7f7bfb7fe640 1 -- 172.21.15.106:0/493807201 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8030860 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.286+0000 7f7bfb7fe640 1 -- 172.21.15.106:0/493807201 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be80309c0 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.286+0000 7f7bfb7fe640 1 -- 172.21.15.106:0/493807201 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be8030d50 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 -- 172.21.15.106:0/493807201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 msgr2=0x7f7bfc0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/493807201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc0b0e90 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc161d30 tx=0x7f7be802b9a0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 -- 172.21.15.106:0/493807201 shutdown_connections 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/493807201 >> v2:172.21.15.156:3300/0 conn(0x7f7bfc0ca940 0x7f7bfc0b13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/493807201 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc0b0e90 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 -- 172.21.15.106:0/493807201 >> 172.21.15.106:0/493807201 conn(0x7f7bfc0cbb40 msgr2=0x7f7bfc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 -- 172.21.15.106:0/493807201 shutdown_connections 2023-11-24T08:11:11.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.287+0000 7f7c01ce1640 1 -- 172.21.15.106:0/493807201 wait complete. 2023-11-24T08:11:11.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.289+0000 7f7c01ce1640 1 Processor -- start 2023-11-24T08:11:11.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.289+0000 7f7c01ce1640 1 -- start start 2023-11-24T08:11:11.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c01ce1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc158660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c01ce1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bfc0ca940 0x7f7bfc0b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c01ce1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bfc0cb580 0x7f7bfc1fb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c01ce1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc158ba0 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c01ce1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc099f40 con 0x7f7bfc0cb580 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c01ce1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bfc09a220 con 0x7f7bfc0ca940 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc158660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc158660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41574/0 (socket says 172.21.15.106:41574) 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 -- 172.21.15.106:0/935886102 learned_addr learned my addr 172.21.15.106:0/935886102 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c014e0640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bfc0cb580 0x7f7bfc1fb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7bfbfff640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bfc0ca940 0x7f7bfc0b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 -- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bfc0ca940 msgr2=0x7f7bfc0b1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bfc0ca940 0x7f7bfc0b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 -- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bfc0cb580 msgr2=0x7f7bfc1fb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bfc0cb580 0x7f7bfc1fb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c00cdf640 1 -- 172.21.15.106:0/935886102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be80094d0 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.290+0000 7f7c014e0640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bfc0cb580 0x7f7bfc1fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.291+0000 7f7c00cdf640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc158660 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c430 tx=0x7f7bf000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.291+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00074e0 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.291+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf0007660 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.291+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0004a20 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.291+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bfc09a500 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.291+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bfc1fac40 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.292+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bfc04e840 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.293+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7bf0004b80 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.295+0000 7f7bf97fa640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7be006b080 0x7f7be006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.295+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7bf0089880 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.295+0000 7f7bfbfff640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7be006b080 0x7f7be006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.298 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.296+0000 7f7bfbfff640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7be006b080 0x7f7be006d540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7be8036000 tx=0x7f7be803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:11.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.299+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7bf0054620 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.490+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7bfc06c710 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.491+0000 7f7bf97fa640 1 -- 172.21.15.106:0/935886102 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7bf005e020 con 0x7f7bfc0c27a0 2023-11-24T08:11:11.494 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7be006b080 msgr2=0x7f7be006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7be006b080 0x7f7be006d540 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7be8036000 tx=0x7f7be803b040 comp rx=0 tx=0).stop 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 msgr2=0x7f7bfc158660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc158660 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c430 tx=0x7f7bf000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 shutdown_connections 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bfc0cb580 0x7f7bfc1fb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7be006b080 0x7f7be006d540 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7bfc0ca940 0x7f7bfc0b1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 --2- 172.21.15.106:0/935886102 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7bfc0c27a0 0x7f7bfc158660 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 >> 172.21.15.106:0/935886102 conn(0x7f7bfc0cbb40 msgr2=0x7f7bfc164480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:11.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 shutdown_connections 2023-11-24T08:11:11.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.494+0000 7f7c01ce1640 1 -- 172.21.15.106:0/935886102 wait complete. 2023-11-24T08:11:11.514 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:11:11.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.719+0000 7f9fe359e640 1 Processor -- start 2023-11-24T08:11:11.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.719+0000 7f9fe359e640 1 -- start start 2023-11-24T08:11:11.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.721+0000 7f9fe359e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9fe415b260 0x7f9fe406e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.721+0000 7f9fe359e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe415ed20 con 0x7f9fe40b1250 2023-11-24T08:11:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.721+0000 7f9fe359e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe415f8a0 con 0x7f9fe406e580 2023-11-24T08:11:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.721+0000 7f9fe359e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe415b640 con 0x7f9fe415b260 2023-11-24T08:11:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.721+0000 7f9fe2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9fe40b1250 0x7f9fe406c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59278/0 (socket says 172.21.15.106:59278) 2023-11-24T08:11:11.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.721+0000 7f9fe2d9d640 1 -- 172.21.15.106:0/80290016 learned_addr learned my addr 172.21.15.106:0/80290016 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182522268 0 0) 0x7f9fe415ed20 con 0x7f9fe40b1250 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe259c640 1 --2- 172.21.15.106:0/80290016 >> v2:172.21.15.156:3300/0 conn(0x7f9fe415b260 0x7f9fe406e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd0003520 con 0x7f9fe40b1250 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 283444789 0 0) 0x7f9fd0003520 con 0x7f9fe40b1250 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 >> v2:172.21.15.156:3300/0 conn(0x7f9fe415b260 msgr2=0x7f9fe406e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 --2- 172.21.15.106:0/80290016 >> v2:172.21.15.156:3300/0 conn(0x7f9fe415b260 0x7f9fe406e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 >> v1:172.21.15.156:6789/0 conn(0x7f9fe406e580 legacy=0x7f9fe40b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe40abe50 con 0x7f9fe40b1250 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f9fdc0019f0 con 0x7f9fe40b1250 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 >> v1:172.21.15.106:6789/0 conn(0x7f9fe40b1250 legacy=0x7f9fe406c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fd0003970 0x7f9fd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.722+0000 7f9fe1d9b640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00047c0 0x7f9fd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe1d9b640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fd00052a0 0x7f9fd000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe415b640 con 0x7f9fd00052a0 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd000c170 con 0x7f9fd00047c0 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd000c330 con 0x7f9fd0003970 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe2d9d640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fd00052a0 0x7f9fd000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe259c640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fd0003970 0x7f9fd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fdbfff640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00047c0 0x7f9fd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe2d9d640 1 -- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fd0003970 msgr2=0x7f9fd0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:11.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe2d9d640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fd0003970 0x7f9fd0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe2d9d640 1 -- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00047c0 msgr2=0x7f9fd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:11.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe2d9d640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00047c0 0x7f9fd0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe2d9d640 1 -- 172.21.15.106:0/80290016 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0003520 con 0x7f9fd00052a0 2023-11-24T08:11:11.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.723+0000 7f9fe259c640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fd0003970 0x7f9fd0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.724+0000 7f9fe2d9d640 1 -- 172.21.15.106:0/80290016 reap_dead start 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.724+0000 7f9fe2d9d640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fd00052a0 0x7f9fd000b980 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9fe406dff0 tx=0x7f9fd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c765eb9ccba6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0072d0 con 0x7f9fd00052a0 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc02f070 con 0x7f9fd00052a0 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe1d9b640 1 -- 172.21.15.106:0/80290016 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc007690 con 0x7f9fd00052a0 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 -- 172.21.15.106:0/80290016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fd00052a0 msgr2=0x7f9fd000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fd00052a0 0x7f9fd000b980 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9fe406dff0 tx=0x7f9fd00057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 -- 172.21.15.106:0/80290016 shutdown_connections 2023-11-24T08:11:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 --2- 172.21.15.106:0/80290016 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fd00052a0 0x7f9fd000b980 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 -- 172.21.15.106:0/80290016 >> 172.21.15.106:0/80290016 conn(0x7f9fe40cbb40 msgr2=0x7f9fe40cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 -- 172.21.15.106:0/80290016 shutdown_connections 2023-11-24T08:11:11.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.725+0000 7f9fe359e640 1 -- 172.21.15.106:0/80290016 wait complete. 2023-11-24T08:11:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.727+0000 7f9fe359e640 1 Processor -- start 2023-11-24T08:11:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.727+0000 7f9fe359e640 1 -- start start 2023-11-24T08:11:11.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fe359e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00052a0 0x7f9fe415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fe359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 0x7f9fe415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fe359e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fe406cdc0 0x7f9fe4157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fe359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe415bfb0 con 0x7f9fe40b47f0 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fe359e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe415c290 con 0x7f9fd00052a0 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fe359e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe415c570 con 0x7f9fe406cdc0 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 0x7f9fe415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 0x7f9fe415b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41578/0 (socket says 172.21.15.106:41578) 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.728+0000 7f9fdbfff640 1 -- 172.21.15.106:0/3930431665 learned_addr learned my addr 172.21.15.106:0/3930431665 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fe259c640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00052a0 0x7f9fe415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdbfff640 1 -- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fe406cdc0 msgr2=0x7f9fe4157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdbfff640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fe406cdc0 0x7f9fe4157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdbfff640 1 -- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00052a0 msgr2=0x7f9fe415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdbfff640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00052a0 0x7f9fe415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdbfff640 1 -- 172.21.15.106:0/3930431665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe415c850 con 0x7f9fe40b47f0 2023-11-24T08:11:11.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fe259c640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00052a0 0x7f9fe415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:11.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdbfff640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 0x7f9fe415b780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e480 tx=0x7f9fcc00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:11.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0052e0 con 0x7f9fe40b47f0 2023-11-24T08:11:11.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc005440 con 0x7f9fe40b47f0 2023-11-24T08:11:11.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe415cb30 con 0x7f9fe40b47f0 2023-11-24T08:11:11.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.729+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc003810 con 0x7f9fe40b47f0 2023-11-24T08:11:11.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.730+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe406d240 con 0x7f9fe40b47f0 2023-11-24T08:11:11.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.731+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe404e840 con 0x7f9fe40b47f0 2023-11-24T08:11:11.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.732+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9fcc003970 con 0x7f9fe40b47f0 2023-11-24T08:11:11.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.733+0000 7f9fdaffd640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9fc406b080 0x7f9fc406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:11.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.733+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9fcc089820 con 0x7f9fe40b47f0 2023-11-24T08:11:11.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.733+0000 7f9fe259c640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9fc406b080 0x7f9fc406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:11.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.734+0000 7f9fe259c640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9fc406b080 0x7f9fc406d540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009aa0 tx=0x7f9fd4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:11.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.739+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9fcc054520 con 0x7f9fe40b47f0 2023-11-24T08:11:11.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.926+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fe40ce640 con 0x7f9fe40b47f0 2023-11-24T08:11:11.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.927+0000 7f9fdaffd640 1 -- 172.21.15.106:0/3930431665 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9fcc057b40 con 0x7f9fe40b47f0 2023-11-24T08:11:11.930 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:11:11.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9fc406b080 msgr2=0x7f9fc406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9fc406b080 0x7f9fc406d540 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009aa0 tx=0x7f9fd4008040 comp rx=0 tx=0).stop 2023-11-24T08:11:11.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 msgr2=0x7f9fe415b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:11.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 0x7f9fe415b780 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00e480 tx=0x7f9fcc00e940 comp rx=0 tx=0).stop 2023-11-24T08:11:11.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 shutdown_connections 2023-11-24T08:11:11.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9fc406b080 0x7f9fc406d540 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9fe406cdc0 0x7f9fe4157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9fe40b47f0 0x7f9fe415b780 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 --2- 172.21.15.106:0/3930431665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fd00052a0 0x7f9fe415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:11.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.930+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 >> 172.21.15.106:0/3930431665 conn(0x7f9fe40cbb40 msgr2=0x7f9fe4071ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:11.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.931+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 shutdown_connections 2023-11-24T08:11:11.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:11.931+0000 7f9fe359e640 1 -- 172.21.15.106:0/3930431665 wait complete. 2023-11-24T08:11:11.952 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:11:12.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.164+0000 7eff999ff640 1 Processor -- start 2023-11-24T08:11:12.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.164+0000 7eff999ff640 1 -- start start 2023-11-24T08:11:12.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff999ff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 0x7eff940cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff999ff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940d15e0 con 0x7eff940d1f40 2023-11-24T08:11:12.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff999ff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940d0280 con 0x7eff9415d3f0 2023-11-24T08:11:12.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff999ff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b0eb0 con 0x7eff9415d7d0 2023-11-24T08:11:12.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff991fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7eff940d1f40 0x7eff940ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59284/0 (socket says 172.21.15.106:59284) 2023-11-24T08:11:12.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff991fe640 1 -- 172.21.15.106:0/1221008852 learned_addr learned my addr 172.21.15.106:0/1221008852 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:12.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff93fff640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 0x7eff940cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.166+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3096829507 0 0) 0x7eff940d15e0 con 0x7eff940d1f40 2023-11-24T08:11:12.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff7c003520 con 0x7eff940d1f40 2023-11-24T08:11:12.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3011445772 0 0) 0x7eff7c003520 con 0x7eff940d1f40 2023-11-24T08:11:12.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 msgr2=0x7eff940cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:12.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 0x7eff940cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 >> v1:172.21.15.156:6789/0 conn(0x7eff9415d3f0 legacy=0x7eff940d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940d1300 con 0x7eff940d1f40 2023-11-24T08:11:12.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7eff8c0019f0 con 0x7eff940d1f40 2023-11-24T08:11:12.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 >> v1:172.21.15.106:6789/0 conn(0x7eff940d1f40 legacy=0x7eff940ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff7c0039a0 0x7eff7c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 0x7eff7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff7c004d00 0x7eff7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b0eb0 con 0x7eff7c0039a0 2023-11-24T08:11:12.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c00ba30 con 0x7eff7c004d00 2023-11-24T08:11:12.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7c00bbf0 con 0x7eff9415d7d0 2023-11-24T08:11:12.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.167+0000 7eff989fd640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff7c0039a0 0x7eff7c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.168+0000 7eff989fd640 1 -- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 msgr2=0x7eff7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:12.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.168+0000 7eff989fd640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff9415d7d0 0x7eff7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.168+0000 7eff989fd640 1 -- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff7c004d00 msgr2=0x7eff7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:12.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.168+0000 7eff989fd640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff7c004d00 0x7eff7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.168+0000 7eff989fd640 1 -- 172.21.15.106:0/1221008852 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7c003520 con 0x7eff7c0039a0 2023-11-24T08:11:12.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.169+0000 7eff989fd640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff7c0039a0 0x7eff7c003dc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7eff88001720 tx=0x7eff7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7974b00bd320f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:12.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.169+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880052d0 con 0x7eff7c0039a0 2023-11-24T08:11:12.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.169+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8802e050 con 0x7eff7c0039a0 2023-11-24T08:11:12.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.169+0000 7eff937fe640 1 -- 172.21.15.106:0/1221008852 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff880035f0 con 0x7eff7c0039a0 2023-11-24T08:11:12.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff999ff640 1 -- 172.21.15.106:0/1221008852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff7c0039a0 msgr2=0x7eff7c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff999ff640 1 --2- 172.21.15.106:0/1221008852 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff7c0039a0 0x7eff7c003dc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7eff88001720 tx=0x7eff7c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff991fe640 1 -- 172.21.15.106:0/1221008852 reap_dead start 2023-11-24T08:11:12.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff999ff640 1 -- 172.21.15.106:0/1221008852 shutdown_connections 2023-11-24T08:11:12.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff999ff640 1 -- 172.21.15.106:0/1221008852 >> 172.21.15.106:0/1221008852 conn(0x7eff9409c5b0 msgr2=0x7eff940d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:12.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff999ff640 1 -- 172.21.15.106:0/1221008852 shutdown_connections 2023-11-24T08:11:12.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.170+0000 7eff999ff640 1 -- 172.21.15.106:0/1221008852 wait complete. 2023-11-24T08:11:12.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.172+0000 7eff999ff640 1 Processor -- start 2023-11-24T08:11:12.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.172+0000 7eff999ff640 1 -- start start 2023-11-24T08:11:12.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff999ff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7c0039a0 0x7eff94158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff999ff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff940cb860 0x7eff940b2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff999ff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 0x7eff940ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff999ff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940cace0 con 0x7eff940ca340 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff999ff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b3270 con 0x7eff940cb860 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff999ff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff940b3550 con 0x7eff7c0039a0 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff991fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 0x7eff940ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff991fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 0x7eff940ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41610/0 (socket says 172.21.15.106:41610) 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff991fe640 1 -- 172.21.15.106:0/3140468111 learned_addr learned my addr 172.21.15.106:0/3140468111 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff989fd640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7c0039a0 0x7eff94158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.173+0000 7eff93fff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff940cb860 0x7eff940b2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff991fe640 1 -- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7c0039a0 msgr2=0x7eff94158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff991fe640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7c0039a0 0x7eff94158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff991fe640 1 -- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff940cb860 msgr2=0x7eff940b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff991fe640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff940cb860 0x7eff940b2970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff991fe640 1 -- 172.21.15.106:0/3140468111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff940b3830 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff989fd640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7c0039a0 0x7eff94158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff991fe640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 0x7eff940ca7a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7eff940cf950 tx=0x7eff8c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00a610 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c00a770 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00ab00 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff940b3b10 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.174+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff940b4310 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.175+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94067990 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.176+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7eff8c00ac60 con 0x7eff940ca340 2023-11-24T08:11:12.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.178+0000 7eff917fa640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff8006b080 0x7eff8006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.178+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7eff8c0892d0 con 0x7eff940ca340 2023-11-24T08:11:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.178+0000 7eff989fd640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff8006b080 0x7eff8006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.178+0000 7eff989fd640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff8006b080 0x7eff8006d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7eff88033000 tx=0x7eff88009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:12.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.182+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff8c053fc0 con 0x7eff940ca340 2023-11-24T08:11:12.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.373+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff940b4030 con 0x7eff940ca340 2023-11-24T08:11:12.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.373+0000 7eff917fa640 1 -- 172.21.15.106:0/3140468111 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff8c0575e0 con 0x7eff940ca340 2023-11-24T08:11:12.376 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff8006b080 msgr2=0x7eff8006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff8006b080 0x7eff8006d540 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7eff88033000 tx=0x7eff88009030 comp rx=0 tx=0).stop 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 msgr2=0x7eff940ca7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 0x7eff940ca7a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7eff940cf950 tx=0x7eff8c00cf00 comp rx=0 tx=0).stop 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 shutdown_connections 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff940ca340 0x7eff940ca7a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff940cb860 0x7eff940b2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff8006b080 0x7eff8006d540 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 --2- 172.21.15.106:0/3140468111 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff7c0039a0 0x7eff94158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 >> 172.21.15.106:0/3140468111 conn(0x7eff9409c5b0 msgr2=0x7eff9406c480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:12.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 shutdown_connections 2023-11-24T08:11:12.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.376+0000 7eff999ff640 1 -- 172.21.15.106:0/3140468111 wait complete. 2023-11-24T08:11:12.401 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:11:12.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.602+0000 7ff4fdadf640 1 Processor -- start 2023-11-24T08:11:12.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.603+0000 7ff4fdadf640 1 -- start start 2023-11-24T08:11:12.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4fdadf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff4f815d490 0x7ff4f80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4fdadf640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80cea30 con 0x7ff4f80b63d0 2023-11-24T08:11:12.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4fdadf640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80ce750 con 0x7ff4f80d3940 2023-11-24T08:11:12.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4fdadf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80d0450 con 0x7ff4f815d490 2023-11-24T08:11:12.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4fcadd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff4f815d490 0x7ff4f80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4f7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff4f80b63d0 0x7ff4f80d3230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59288/0 (socket says 172.21.15.106:59288) 2023-11-24T08:11:12.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.604+0000 7ff4f7fff640 1 -- 172.21.15.106:0/4090461541 learned_addr learned my addr 172.21.15.106:0/4090461541 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:12.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.605+0000 7ff4fcadd640 1 -- 172.21.15.106:0/4090461541 >> v1:172.21.15.156:6789/0 conn(0x7ff4f80d3940 legacy=0x7ff4f80d1250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:12.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.605+0000 7ff4fcadd640 1 -- 172.21.15.106:0/4090461541 >> v1:172.21.15.106:6789/0 conn(0x7ff4f80b63d0 legacy=0x7ff4f80d3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.605+0000 7ff4fcadd640 1 -- 172.21.15.106:0/4090461541 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80d0630 con 0x7ff4f815d490 2023-11-24T08:11:12.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.605+0000 7ff4f77fe640 1 -- 172.21.15.106:0/4090461541 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947026286 0 0) 0x7ff4f80cea30 con 0x7ff4f80b63d0 2023-11-24T08:11:12.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.606+0000 7ff4fcadd640 1 --2- 172.21.15.106:0/4090461541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cd8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0097a0 tx=0x7ff4ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a8fcbc0c2d60c48 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:12.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.606+0000 7ff4f77fe640 1 -- 172.21.15.106:0/4090461541 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec031830 con 0x7ff4f815d490 2023-11-24T08:11:12.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.606+0000 7ff4f77fe640 1 -- 172.21.15.106:0/4090461541 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ec031990 con 0x7ff4f815d490 2023-11-24T08:11:12.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.606+0000 7ff4f77fe640 1 -- 172.21.15.106:0/4090461541 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ec031d20 con 0x7ff4f815d490 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.606+0000 7ff4fdadf640 1 -- 172.21.15.106:0/4090461541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 msgr2=0x7ff4f80cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.606+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/4090461541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cd8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec0097a0 tx=0x7ff4ec02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.607+0000 7ff4fdadf640 1 -- 172.21.15.106:0/4090461541 shutdown_connections 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.607+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/4090461541 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cd8f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.607+0000 7ff4fdadf640 1 -- 172.21.15.106:0/4090461541 >> 172.21.15.106:0/4090461541 conn(0x7ff4f8061340 msgr2=0x7ff4f8061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.607+0000 7ff4fdadf640 1 -- 172.21.15.106:0/4090461541 shutdown_connections 2023-11-24T08:11:12.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.607+0000 7ff4fdadf640 1 -- 172.21.15.106:0/4090461541 wait complete. 2023-11-24T08:11:12.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.608+0000 7ff4fdadf640 1 Processor -- start 2023-11-24T08:11:12.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.609+0000 7ff4fdadf640 1 -- start start 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fdadf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff4f80d3940 0x7ff4f80c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fdadf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fdadf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4f80ccf20 0x7ff4f81580d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fdadf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80d0bf0 con 0x7ff4f815d490 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fdadf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80c7bb0 con 0x7ff4f80ccf20 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fdadf640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80c6f30 con 0x7ff4f80d3940 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cc9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41626/0 (socket says 172.21.15.106:41626) 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 -- 172.21.15.106:0/696772262 learned_addr learned my addr 172.21.15.106:0/696772262 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fcadd640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff4f80d3940 0x7ff4f80c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 -- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff4f80d3940 msgr2=0x7ff4f80c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff4f80d3940 0x7ff4f80c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 -- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4f80ccf20 msgr2=0x7ff4f81580d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fd2de640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4f80ccf20 0x7ff4f81580d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4f80ccf20 0x7ff4f81580d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4f7fff640 1 -- 172.21.15.106:0/696772262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec009400 con 0x7ff4f815d490 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.610+0000 7ff4fd2de640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4f80ccf20 0x7ff4f81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:12.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.611+0000 7ff4f7fff640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cc9e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c4d0 tx=0x7ff4e400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.611+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e40074e0 con 0x7ff4f815d490 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.611+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e4007660 con 0x7ff4f815d490 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.611+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4004a20 con 0x7ff4f815d490 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.611+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f80c7210 con 0x7ff4f815d490 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.611+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80c77d0 con 0x7ff4f815d490 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.612+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f80c7cf0 con 0x7ff4f815d490 2023-11-24T08:11:12.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.613+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff4e4004b80 con 0x7ff4f815d490 2023-11-24T08:11:12.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.614+0000 7ff4f57fa640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff4dc06b0b0 0x7ff4dc06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:12.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.614+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff4e4088cd0 con 0x7ff4f815d490 2023-11-24T08:11:12.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.614+0000 7ff4fcadd640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff4dc06b0b0 0x7ff4dc06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:12.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.615+0000 7ff4fcadd640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff4dc06b0b0 0x7ff4dc06d570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec036000 tx=0x7ff4ec008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:12.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.619+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff4e4053940 con 0x7ff4f815d490 2023-11-24T08:11:12.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.820+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff4f804e840 con 0x7ff4f815d490 2023-11-24T08:11:12.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.821+0000 7ff4f57fa640 1 -- 172.21.15.106:0/696772262 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff4e4018020 con 0x7ff4f815d490 2023-11-24T08:11:12.823 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:11:12.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff4dc06b0b0 msgr2=0x7ff4dc06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff4dc06b0b0 0x7ff4dc06d570 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff4ec036000 tx=0x7ff4ec008030 comp rx=0 tx=0).stop 2023-11-24T08:11:12.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 msgr2=0x7ff4f80cc9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cc9e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c4d0 tx=0x7ff4e400c990 comp rx=0 tx=0).stop 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 shutdown_connections 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4f80ccf20 0x7ff4f81580d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff4f815d490 0x7ff4f80cc9e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff4dc06b0b0 0x7ff4dc06d570 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 --2- 172.21.15.106:0/696772262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff4f80d3940 0x7ff4f80c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.823+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 >> 172.21.15.106:0/696772262 conn(0x7ff4f8061340 msgr2=0x7ff4f80b4450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.824+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 shutdown_connections 2023-11-24T08:11:12.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:12.824+0000 7ff4fdadf640 1 -- 172.21.15.106:0/696772262 wait complete. 2023-11-24T08:11:12.845 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:11:12.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:12.845 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T08:11:12.962 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:11:12.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:12.962 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:11:13.013 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:13.013 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:13.073 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:11:13.073 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:13.073 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:13.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:11:13.135 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:13.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.355+0000 7fcc1e683640 1 Processor -- start 2023-11-24T08:11:13.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.355+0000 7fcc1e683640 1 -- start start 2023-11-24T08:11:13.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1e683640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 0x7fcc1815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1e683640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcc180b2630 0x7fcc180ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1e683640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc1815ab80 con 0x7fcc180cd470 2023-11-24T08:11:13.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1e683640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc1815a990 con 0x7fcc180b2630 2023-11-24T08:11:13.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1e683640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1815e680 con 0x7fcc180b2250 2023-11-24T08:11:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1cc20640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcc180cd470 0x7fcc1815b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:59292/0 (socket says 172.21.15.106:59292) 2023-11-24T08:11:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc1cc20640 1 -- 172.21.15.106:0/1731315025 learned_addr learned my addr 172.21.15.106:0/1731315025 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1731315025 >> v2:172.21.15.156:3300/0 conn(0x7fcc180b2630 0x7fcc180ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc17fff640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 0x7fcc1815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887597159 0 0) 0x7fcc1815ab80 con 0x7fcc180cd470 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.357+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc04005730 con 0x7fcc180cd470 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2160860923 0 0) 0x7fcc04005730 con 0x7fcc180cd470 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 msgr2=0x7fcc1815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 0x7fcc1815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 >> v2:172.21.15.156:3300/0 conn(0x7fcc180b2630 msgr2=0x7fcc180ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 --2- 172.21.15.106:0/1731315025 >> v2:172.21.15.156:3300/0 conn(0x7fcc180b2630 0x7fcc180ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc1815e960 con 0x7fcc180cd470 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fcc100019f0 con 0x7fcc180cd470 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 >> v1:172.21.15.106:6789/0 conn(0x7fcc180cd470 legacy=0x7fcc1815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 0x7fcc04005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc040069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc04006f10 0x7fcc04007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1815e680 con 0x7fcc04006150 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0400dbc0 con 0x7fcc04006f10 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0400dd80 con 0x7fcc180b2250 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.358+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc040069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc177fe640 1 -- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 msgr2=0x7fcc04005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc17fff640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 0x7fcc04005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc1cc20640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc04006f10 0x7fcc04007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180b2250 0x7fcc04005b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc177fe640 1 -- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc04006f10 msgr2=0x7fcc04007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc04006f10 0x7fcc04007390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.359+0000 7fcc177fe640 1 -- 172.21.15.106:0/1731315025 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc04005730 con 0x7fcc04006150 2023-11-24T08:11:13.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.360+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc040069d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009ac0 tx=0x7fcc04007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7397ab9c06e256f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.360+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0002c710 con 0x7fcc04006150 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.360+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0002c870 con 0x7fcc04006150 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.360+0000 7fcc16ffd640 1 -- 172.21.15.106:0/1731315025 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0002cc00 con 0x7fcc04006150 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1e683640 1 -- 172.21.15.106:0/1731315025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 msgr2=0x7fcc040069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1731315025 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc040069d0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009ac0 tx=0x7fcc04007a00 comp rx=0 tx=0).stop 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1cc20640 1 -- 172.21.15.106:0/1731315025 reap_dead start 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1e683640 1 -- 172.21.15.106:0/1731315025 shutdown_connections 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1e683640 1 -- 172.21.15.106:0/1731315025 >> 172.21.15.106:0/1731315025 conn(0x7fcc1806a920 msgr2=0x7fcc1806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1e683640 1 -- 172.21.15.106:0/1731315025 shutdown_connections 2023-11-24T08:11:13.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.361+0000 7fcc1e683640 1 -- 172.21.15.106:0/1731315025 wait complete. 2023-11-24T08:11:13.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.363+0000 7fcc1e683640 1 Processor -- start 2023-11-24T08:11:13.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.363+0000 7fcc1e683640 1 -- start start 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1e683640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc1814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1e683640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180c8ec0 0x7fcc180ca660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1e683640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc180caba0 0x7fcc18154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1e683640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc181535a0 con 0x7fcc04006150 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1e683640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc1814faa0 con 0x7fcc180caba0 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1e683640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc18153780 con 0x7fcc180c8ec0 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc1814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc1814f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41648/0 (socket says 172.21.15.106:41648) 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 -- 172.21.15.106:0/1139501404 learned_addr learned my addr 172.21.15.106:0/1139501404 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180c8ec0 0x7fcc180ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 -- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180c8ec0 msgr2=0x7fcc180ca660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180c8ec0 0x7fcc180ca660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc1cc20640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc180caba0 0x7fcc18154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 -- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc180caba0 msgr2=0x7fcc18154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.364+0000 7fcc17fff640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc180caba0 0x7fcc18154540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc17fff640 1 -- 172.21.15.106:0/1139501404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc18153a60 con 0x7fcc04006150 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180c8ec0 0x7fcc180ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc17fff640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc1814f560 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e7e0 tx=0x7fcc0800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0800c6d0 con 0x7fcc04006150 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0800c830 con 0x7fcc04006150 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0800cbe0 con 0x7fcc04006150 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc18153d40 con 0x7fcc04006150 2023-11-24T08:11:13.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.365+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc18154a80 con 0x7fcc04006150 2023-11-24T08:11:13.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.366+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc181538c0 con 0x7fcc04006150 2023-11-24T08:11:13.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.367+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcc0800cd40 con 0x7fcc04006150 2023-11-24T08:11:13.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.368+0000 7fcc14ff9640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcbf806b080 0x7fcbf806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.368+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcc080898d0 con 0x7fcc04006150 2023-11-24T08:11:13.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.368+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcbf806b080 0x7fcbf806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.369+0000 7fcc177fe640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcbf806b080 0x7fcbf806d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc00036000 tx=0x7fcc00030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:13.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.372+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc181538c0 con 0x7fcc04006150 2023-11-24T08:11:13.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.608+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc181538c0 con 0x7fcc04006150 2023-11-24T08:11:13.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.610+0000 7fcc14ff9640 1 -- 172.21.15.106:0/1139501404 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4925 (secure 0 0 0) 0x7fcc181538c0 con 0x7fcc04006150 2023-11-24T08:11:13.612 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:13.613 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:10:40.829930+0000","modified":"2023-11-24T08:10:43.906826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1634060253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1634060253},{"type":"v1","addr":"172.21.15.106:6837","nonce":1634060253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:11:13.613 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2023-11-24T08:11:13.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcbf806b080 msgr2=0x7fcbf806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcbf806b080 0x7fcbf806d540 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fcc00036000 tx=0x7fcc00030000 comp rx=0 tx=0).stop 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 msgr2=0x7fcc1814f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc1814f560 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fcc0800e7e0 tx=0x7fcc0800eca0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 shutdown_connections 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc180caba0 0x7fcc18154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcbf806b080 0x7fcbf806d540 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcc180c8ec0 0x7fcc180ca660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 --2- 172.21.15.106:0/1139501404 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc04006150 0x7fcc1814f560 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 >> 172.21.15.106:0/1139501404 conn(0x7fcc1806a920 msgr2=0x7fcc181559f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 shutdown_connections 2023-11-24T08:11:13.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.613+0000 7fcc1e683640 1 -- 172.21.15.106:0/1139501404 wait complete. 2023-11-24T08:11:13.633 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:13.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.842+0000 7f3ed6767640 1 Processor -- start 2023-11-24T08:11:13.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.842+0000 7f3ed6767640 1 -- start start 2023-11-24T08:11:13.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.843+0000 7f3ed6767640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3ed0143870 0x7f3ed0146970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.843+0000 7f3ed6767640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0161080 0x7f3ed00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.843+0000 7f3ed6767640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ed0161510 con 0x7f3ed0146eb0 2023-11-24T08:11:13.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.843+0000 7f3ed6767640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed0070460 con 0x7f3ed0143870 2023-11-24T08:11:13.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.843+0000 7f3ed6767640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed01617f0 con 0x7f3ed0161080 2023-11-24T08:11:13.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3ed0143870 0x7f3ed0146970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3ed0143870 0x7f3ed0146970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41664/0 (socket says 172.21.15.106:41664) 2023-11-24T08:11:13.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 -- 172.21.15.106:0/395066040 learned_addr learned my addr 172.21.15.106:0/395066040 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:13.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 -- 172.21.15.106:0/395066040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0161080 msgr2=0x7f3ed00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:13.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed4f64640 1 --2- 172.21.15.106:0/395066040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0161080 0x7f3ed00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 --2- 172.21.15.106:0/395066040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0161080 0x7f3ed00b0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 -- 172.21.15.106:0/395066040 >> v1:172.21.15.156:6789/0 conn(0x7f3ed0146eb0 legacy=0x7f3ed006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:13.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.844+0000 7f3ed5765640 1 -- 172.21.15.106:0/395066040 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed0161ad0 con 0x7f3ed0143870 2023-11-24T08:11:13.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.845+0000 7f3ed5765640 1 --2- 172.21.15.106:0/395066040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed0146970 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009870 tx=0x7f3ecc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0ecb081a2bffc31 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:13.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ec7fff640 1 -- 172.21.15.106:0/395066040 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc031640 con 0x7f3ed0143870 2023-11-24T08:11:13.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ec7fff640 1 -- 172.21.15.106:0/395066040 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc0317a0 con 0x7f3ed0143870 2023-11-24T08:11:13.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ec7fff640 1 -- 172.21.15.106:0/395066040 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc031b30 con 0x7f3ed0143870 2023-11-24T08:11:13.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 -- 172.21.15.106:0/395066040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 msgr2=0x7f3ed0146970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 --2- 172.21.15.106:0/395066040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed0146970 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009870 tx=0x7f3ecc02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 -- 172.21.15.106:0/395066040 shutdown_connections 2023-11-24T08:11:13.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 --2- 172.21.15.106:0/395066040 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0161080 0x7f3ed00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 --2- 172.21.15.106:0/395066040 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed0146970 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 -- 172.21.15.106:0/395066040 >> 172.21.15.106:0/395066040 conn(0x7f3ed00cb830 msgr2=0x7f3ed00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:13.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.846+0000 7f3ed6767640 1 -- 172.21.15.106:0/395066040 shutdown_connections 2023-11-24T08:11:13.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.847+0000 7f3ed6767640 1 -- 172.21.15.106:0/395066040 wait complete. 2023-11-24T08:11:13.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.848+0000 7f3ed6767640 1 Processor -- start 2023-11-24T08:11:13.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.848+0000 7f3ed6767640 1 -- start start 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed6767640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed013ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed6767640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0146eb0 0x7f3ed00b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed6767640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ed00b2470 0x7f3ed0134270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed6767640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00b2c50 con 0x7f3ed0143870 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed6767640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed00b2970 con 0x7f3ed00b2470 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed6767640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed015e890 con 0x7f3ed0146eb0 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.849+0000 7f3ed5765640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed013ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed013ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:41674/0 (socket says 172.21.15.106:41674) 2023-11-24T08:11:13.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 -- 172.21.15.106:0/4147446424 learned_addr learned my addr 172.21.15.106:0/4147446424 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:13.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 -- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0146eb0 msgr2=0x7f3ed00b4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0146eb0 0x7f3ed00b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 -- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ed00b2470 msgr2=0x7f3ed0134270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5f66640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ed00b2470 0x7f3ed0134270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ed00b2470 0x7f3ed0134270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 -- 172.21.15.106:0/4147446424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc0094d0 con 0x7f3ed0143870 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5f66640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ed00b2470 0x7f3ed0134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.850+0000 7f3ed5765640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed013ee30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009870 tx=0x7f3ecc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.851+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc03ac60 con 0x7f3ed0143870 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.851+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc03adc0 con 0x7f3ed0143870 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.851+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc044630 con 0x7f3ed0143870 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.851+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed015eb70 con 0x7f3ed0143870 2023-11-24T08:11:13.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.851+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed013ea70 con 0x7f3ed0143870 2023-11-24T08:11:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.852+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed015e6f0 con 0x7f3ed0143870 2023-11-24T08:11:13.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.853+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3ecc02f050 con 0x7f3ed0143870 2023-11-24T08:11:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.854+0000 7f3ec5ffb640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3eb406b0b0 0x7f3eb406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.854+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3ecc0abd50 con 0x7f3ed0143870 2023-11-24T08:11:13.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.854+0000 7f3ed4f64640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3eb406b0b0 0x7f3eb406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:13.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.855+0000 7f3ed4f64640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3eb406b0b0 0x7f3eb406d570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0134f20 tx=0x7f3ec0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:13.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:13.858+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ed015e6f0 con 0x7f3ed0143870 2023-11-24T08:11:14.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.048+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ed015e6f0 con 0x7f3ed0143870 2023-11-24T08:11:14.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.051+0000 7f3ec5ffb640 1 -- 172.21.15.106:0/4147446424 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f3ed015e6f0 con 0x7f3ed0143870 2023-11-24T08:11:14.054 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:14.054 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":49,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:10.401248+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3eb406b0b0 msgr2=0x7f3eb406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3eb406b0b0 0x7f3eb406d570 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0134f20 tx=0x7f3ec0008040 comp rx=0 tx=0).stop 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 msgr2=0x7f3ed013ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed013ee30 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc009870 tx=0x7f3ecc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 shutdown_connections 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ed00b2470 0x7f3ed0134270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3eb406b0b0 0x7f3eb406d570 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ed0146eb0 0x7f3ed00b4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 --2- 172.21.15.106:0/4147446424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ed0143870 0x7f3ed013ee30 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 >> 172.21.15.106:0/4147446424 conn(0x7f3ed00cb830 msgr2=0x7f3ed00717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:14.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 shutdown_connections 2023-11-24T08:11:14.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.054+0000 7f3ed6767640 1 -- 172.21.15.106:0/4147446424 wait complete. 2023-11-24T08:11:14.075 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:10:38.577975+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:10:39.853606+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '49', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 49, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:14.076 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.285+0000 7fc5e89a9640 1 Processor -- start 2023-11-24T08:11:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.285+0000 7fc5e89a9640 1 -- start start 2023-11-24T08:11:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.287+0000 7fc5e89a9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc5e415b530 0x7fc5e406a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.287+0000 7fc5e89a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 0x7fc5e406aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.287+0000 7fc5e89a9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5e4072160 con 0x7fc5e40999d0 2023-11-24T08:11:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.287+0000 7fc5e89a9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5e415b910 con 0x7fc5e415b530 2023-11-24T08:11:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.287+0000 7fc5e89a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40b3c80 con 0x7fc5e40afa50 2023-11-24T08:11:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 0x7fc5e406aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 0x7fc5e406aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:46380/0 (socket says 172.21.15.106:46380) 2023-11-24T08:11:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 -- 172.21.15.106:0/2425250527 learned_addr learned my addr 172.21.15.106:0/2425250527 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:14.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5e259c640 1 --2- 172.21.15.106:0/2425250527 >> v2:172.21.15.156:3300/0 conn(0x7fc5e415b530 0x7fc5e406a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 -- 172.21.15.106:0/2425250527 >> v2:172.21.15.156:3300/0 conn(0x7fc5e415b530 msgr2=0x7fc5e406a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 --2- 172.21.15.106:0/2425250527 >> v2:172.21.15.156:3300/0 conn(0x7fc5e415b530 0x7fc5e406a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 -- 172.21.15.106:0/2425250527 >> v1:172.21.15.156:6789/0 conn(0x7fc5e40999d0 legacy=0x7fc5e40ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.288+0000 7fc5dbfff640 1 -- 172.21.15.106:0/2425250527 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5e40b27b0 con 0x7fc5e40afa50 2023-11-24T08:11:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.290+0000 7fc5dbfff640 1 --2- 172.21.15.106:0/2425250527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 0x7fc5e406aa40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009870 tx=0x7fc5cc02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=548fcdf487f70383 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.291+0000 7fc5e1d9b640 1 -- 172.21.15.106:0/2425250527 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc030840 con 0x7fc5e40afa50 2023-11-24T08:11:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.291+0000 7fc5e1d9b640 1 -- 172.21.15.106:0/2425250527 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5cc0309a0 con 0x7fc5e40afa50 2023-11-24T08:11:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.291+0000 7fc5e1d9b640 1 -- 172.21.15.106:0/2425250527 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc030d30 con 0x7fc5e40afa50 2023-11-24T08:11:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 -- 172.21.15.106:0/2425250527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 msgr2=0x7fc5e406aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/2425250527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 0x7fc5e406aa40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009870 tx=0x7fc5cc02b980 comp rx=0 tx=0).stop 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 -- 172.21.15.106:0/2425250527 shutdown_connections 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/2425250527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e40afa50 0x7fc5e406aa40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/2425250527 >> v2:172.21.15.156:3300/0 conn(0x7fc5e415b530 0x7fc5e406a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 -- 172.21.15.106:0/2425250527 >> 172.21.15.106:0/2425250527 conn(0x7fc5e406c090 msgr2=0x7fc5e40b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 -- 172.21.15.106:0/2425250527 shutdown_connections 2023-11-24T08:11:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.292+0000 7fc5e89a9640 1 -- 172.21.15.106:0/2425250527 wait complete. 2023-11-24T08:11:14.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.294+0000 7fc5e89a9640 1 Processor -- start 2023-11-24T08:11:14.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.294+0000 7fc5e89a9640 1 -- start start 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e89a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 0x7fc5e40b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e89a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e40afa50 0x7fc5e415d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e89a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e415b530 0x7fc5e41426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e89a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40cf6d0 con 0x7fc5e40999d0 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e89a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40cd1d0 con 0x7fc5e40afa50 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e89a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e40b1650 con 0x7fc5e415b530 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 0x7fc5e40b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 0x7fc5e40b0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34350/0 (socket says 172.21.15.106:34350) 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.295+0000 7fc5e259c640 1 -- 172.21.15.106:0/1220354571 learned_addr learned my addr 172.21.15.106:0/1220354571 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5dbfff640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e40afa50 0x7fc5e415d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e2d9d640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e415b530 0x7fc5e41426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e259c640 1 -- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e415b530 msgr2=0x7fc5e41426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e259c640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e415b530 0x7fc5e41426e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e259c640 1 -- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e40afa50 msgr2=0x7fc5e415d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e259c640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e40afa50 0x7fc5e415d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e259c640 1 -- 172.21.15.106:0/1220354571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc0094d0 con 0x7fc5e40999d0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e259c640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 0x7fc5e40b0d20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400e870 tx=0x7fc5d400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40094f0 con 0x7fc5e40999d0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5d400c940 con 0x7fc5e40999d0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e40b1930 con 0x7fc5e40999d0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400ccf0 con 0x7fc5e40999d0 2023-11-24T08:11:14.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.296+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e40b1ef0 con 0x7fc5e40999d0 2023-11-24T08:11:14.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.298+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e40b4bc0 con 0x7fc5e40999d0 2023-11-24T08:11:14.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.298+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc5d4013470 con 0x7fc5e40999d0 2023-11-24T08:11:14.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.299+0000 7fc5daffd640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc5c406b0b0 0x7fc5c406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.299+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc5d4089c40 con 0x7fc5e40999d0 2023-11-24T08:11:14.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.300+0000 7fc5dbfff640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc5c406b0b0 0x7fc5c406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.300+0000 7fc5dbfff640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc5c406b0b0 0x7fc5c406d570 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc036000 tx=0x7fc5cc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:14.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.304+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc5d40548b0 con 0x7fc5e40999d0 2023-11-24T08:11:14.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.492+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc5e40cc580 con 0x7fc5e40999d0 2023-11-24T08:11:14.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.493+0000 7fc5daffd640 1 -- 172.21.15.106:0/1220354571 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc5d40210a0 con 0x7fc5e40999d0 2023-11-24T08:11:14.496 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc5c406b0b0 msgr2=0x7fc5c406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc5c406b0b0 0x7fc5c406d570 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc036000 tx=0x7fc5cc008030 comp rx=0 tx=0).stop 2023-11-24T08:11:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 msgr2=0x7fc5e40b0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 0x7fc5e40b0d20 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400e870 tx=0x7fc5d400ed30 comp rx=0 tx=0).stop 2023-11-24T08:11:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 shutdown_connections 2023-11-24T08:11:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc5c406b0b0 0x7fc5c406d570 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5e415b530 0x7fc5e41426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5e40afa50 0x7fc5e415d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 --2- 172.21.15.106:0/1220354571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc5e40999d0 0x7fc5e40b0d20 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 >> 172.21.15.106:0/1220354571 conn(0x7fc5e406c090 msgr2=0x7fc5e40aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:14.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 shutdown_connections 2023-11-24T08:11:14.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.495+0000 7fc5e89a9640 1 -- 172.21.15.106:0/1220354571 wait complete. 2023-11-24T08:11:14.515 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:11:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.734+0000 7f8803fff640 1 Processor -- start 2023-11-24T08:11:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.734+0000 7f8803fff640 1 -- start start 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.735+0000 7f8803fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f880406ac10 0x7f88040ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.735+0000 7f8803fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8804161d20 0x7f880406eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.735+0000 7f8803fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88040ce170 con 0x7f8804161940 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.735+0000 7f8803fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88040ce450 con 0x7f880406ac10 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.735+0000 7f8803fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040ce730 con 0x7f8804161d20 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f8802ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8804161940 0x7f880406a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52158/0 (socket says 172.21.15.106:52158) 2023-11-24T08:11:14.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f8802ffd640 1 -- 172.21.15.106:0/3269786737 learned_addr learned my addr 172.21.15.106:0/3269786737 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f88027fc640 1 --2- 172.21.15.106:0/3269786737 >> v2:172.21.15.106:3300/0 conn(0x7f880406ac10 0x7f88040ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f88037fe640 1 --2- 172.21.15.106:0/3269786737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8804161d20 0x7f880406eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f88027fc640 1 -- 172.21.15.106:0/3269786737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8804161d20 msgr2=0x7f880406eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f88027fc640 1 --2- 172.21.15.106:0/3269786737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8804161d20 0x7f880406eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f88027fc640 1 -- 172.21.15.106:0/3269786737 >> v1:172.21.15.106:6789/0 conn(0x7f8804161940 legacy=0x7f880406a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.736+0000 7f88027fc640 1 -- 172.21.15.106:0/3269786737 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88040cea10 con 0x7f880406ac10 2023-11-24T08:11:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.737+0000 7f88027fc640 1 --2- 172.21.15.106:0/3269786737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f880406ac10 0x7f88040ca820 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009d90 tx=0x7f87f402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7cc79f543bb7ed server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.737+0000 7f8801ffb640 1 -- 172.21.15.106:0/3269786737 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f402cc60 con 0x7f880406ac10 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8801ffb640 1 -- 172.21.15.106:0/3269786737 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f402cdc0 con 0x7f880406ac10 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8801ffb640 1 -- 172.21.15.106:0/3269786737 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4034660 con 0x7f880406ac10 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 -- 172.21.15.106:0/3269786737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f880406ac10 msgr2=0x7f88040ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 --2- 172.21.15.106:0/3269786737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f880406ac10 0x7f88040ca820 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f87f4009d90 tx=0x7f87f402beb0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 -- 172.21.15.106:0/3269786737 shutdown_connections 2023-11-24T08:11:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 --2- 172.21.15.106:0/3269786737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8804161d20 0x7f880406eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 --2- 172.21.15.106:0/3269786737 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f880406ac10 0x7f88040ca820 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 -- 172.21.15.106:0/3269786737 >> 172.21.15.106:0/3269786737 conn(0x7f88040d01d0 msgr2=0x7f88040d05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 -- 172.21.15.106:0/3269786737 shutdown_connections 2023-11-24T08:11:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.738+0000 7f8803fff640 1 -- 172.21.15.106:0/3269786737 wait complete. 2023-11-24T08:11:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.740+0000 7f8803fff640 1 Processor -- start 2023-11-24T08:11:14.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.740+0000 7f8803fff640 1 -- start start 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.741+0000 7f8803fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f880406ac10 0x7f88040b0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.741+0000 7f8803fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 0x7f88040b5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.741+0000 7f8803fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8804161d20 0x7f8804147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.741+0000 7f8803fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040b2050 con 0x7f8804161940 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.741+0000 7f8803fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040b5e40 con 0x7f8804161d20 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.741+0000 7f8803fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040b1440 con 0x7f880406ac10 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 0x7f88040b5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 0x7f88040b5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34366/0 (socket says 172.21.15.106:34366) 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 -- 172.21.15.106:0/2295811337 learned_addr learned my addr 172.21.15.106:0/2295811337 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:14.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f8802ffd640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f880406ac10 0x7f88040b0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88037fe640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8804161d20 0x7f8804147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 -- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f880406ac10 msgr2=0x7f88040b0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f880406ac10 0x7f88040b0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 -- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8804161d20 msgr2=0x7f8804147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8804161d20 0x7f8804147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.742+0000 7f88027fc640 1 -- 172.21.15.106:0/2295811337 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009a70 con 0x7f8804161940 2023-11-24T08:11:14.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.743+0000 7f88027fc640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 0x7f88040b5900 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f87f4037040 tx=0x7f87f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:14.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.743+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4040650 con 0x7f8804161940 2023-11-24T08:11:14.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.743+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f40407b0 con 0x7f8804161940 2023-11-24T08:11:14.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.743+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88040b1720 con 0x7f8804161940 2023-11-24T08:11:14.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.743+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4040b60 con 0x7f8804161940 2023-11-24T08:11:14.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.743+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88040b1ce0 con 0x7f8804161940 2023-11-24T08:11:14.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.744+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88040b2200 con 0x7f8804161940 2023-11-24T08:11:14.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.745+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f87f4040cc0 con 0x7f8804161940 2023-11-24T08:11:14.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.747+0000 7f87fb7fe640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f87e406b080 0x7f87e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:14.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.747+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f87f40ac8b0 con 0x7f8804161940 2023-11-24T08:11:14.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.747+0000 7f8802ffd640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f87e406b080 0x7f87e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:14.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.748+0000 7f8802ffd640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f87e406b080 0x7f87e406d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005880 tx=0x7f87ec009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:14.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.751+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f88040b2200 con 0x7f8804161940 2023-11-24T08:11:14.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.942+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88040b2200 con 0x7f8804161940 2023-11-24T08:11:14.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.943+0000 7f87fb7fe640 1 -- 172.21.15.106:0/2295811337 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v49) v1 ==== 104+0+11 (secure 0 0 0) 0x7f88040b2200 con 0x7f8804161940 2023-11-24T08:11:14.961 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:11:14.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f87e406b080 msgr2=0x7f87e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f87e406b080 0x7f87e406d540 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87ec005880 tx=0x7f87ec009040 comp rx=0 tx=0).stop 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 msgr2=0x7f88040b5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 0x7f88040b5900 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f87f4037040 tx=0x7f87f4004a40 comp rx=0 tx=0).stop 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 shutdown_connections 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8804161d20 0x7f8804147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8804161940 0x7f88040b5900 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f87e406b080 0x7f87e406d540 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 --2- 172.21.15.106:0/2295811337 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f880406ac10 0x7f88040b0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 >> 172.21.15.106:0/2295811337 conn(0x7f88040d01d0 msgr2=0x7f88040d05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 shutdown_connections 2023-11-24T08:11:14.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:14.947+0000 7f8803fff640 1 -- 172.21.15.106:0/2295811337 wait complete. 2023-11-24T08:11:14.968 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:11:15.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.182+0000 7f55f835a640 1 Processor -- start 2023-11-24T08:11:15.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.182+0000 7f55f835a640 1 -- start start 2023-11-24T08:11:15.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f835a640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f55f0072160 0x7f55f015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f835a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55f00c49f0 0x7f55f006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f835a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f006dc40 0x7f55f00cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f835a640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f006e0d0 con 0x7f55f00c49f0 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f835a640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f015ec90 con 0x7f55f0072160 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f835a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00cd530 con 0x7f55f006dc40 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f58f5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55f00c49f0 0x7f55f006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f58f5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f55f00c49f0 0x7f55f006d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34376/0 (socket says 172.21.15.106:34376) 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.183+0000 7f55f58f5640 1 -- 172.21.15.106:0/3883851698 learned_addr learned my addr 172.21.15.106:0/3883851698 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f60f6640 1 --2- 172.21.15.106:0/3883851698 >> v2:172.21.15.156:3300/0 conn(0x7f55f0072160 0x7f55f015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f68f7640 1 --2- 172.21.15.106:0/3883851698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f006dc40 0x7f55f00cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f58f5640 1 -- 172.21.15.106:0/3883851698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f006dc40 msgr2=0x7f55f00cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f58f5640 1 --2- 172.21.15.106:0/3883851698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f006dc40 0x7f55f00cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f58f5640 1 -- 172.21.15.106:0/3883851698 >> v2:172.21.15.156:3300/0 conn(0x7f55f0072160 msgr2=0x7f55f015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f58f5640 1 --2- 172.21.15.106:0/3883851698 >> v2:172.21.15.156:3300/0 conn(0x7f55f0072160 0x7f55f015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.184+0000 7f55f58f5640 1 -- 172.21.15.106:0/3883851698 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00c4ed0 con 0x7f55f00c49f0 2023-11-24T08:11:15.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.185+0000 7f55f58f5640 1 --2- 172.21.15.106:0/3883851698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00c49f0 0x7f55f006d700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55f00ccae0 tx=0x7f55e002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d23c8bb3c100e62c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:15.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.185+0000 7f55f50f4640 1 -- 172.21.15.106:0/3883851698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e002ee90 con 0x7f55f00c49f0 2023-11-24T08:11:15.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.185+0000 7f55f50f4640 1 -- 172.21.15.106:0/3883851698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e0036440 con 0x7f55f00c49f0 2023-11-24T08:11:15.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.185+0000 7f55f50f4640 1 -- 172.21.15.106:0/3883851698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e00367d0 con 0x7f55f00c49f0 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 -- 172.21.15.106:0/3883851698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00c49f0 msgr2=0x7f55f006d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 --2- 172.21.15.106:0/3883851698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00c49f0 0x7f55f006d700 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f55f00ccae0 tx=0x7f55e002e0e0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 -- 172.21.15.106:0/3883851698 shutdown_connections 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 --2- 172.21.15.106:0/3883851698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f006dc40 0x7f55f00cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 --2- 172.21.15.106:0/3883851698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f00c49f0 0x7f55f006d700 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 --2- 172.21.15.106:0/3883851698 >> v2:172.21.15.156:3300/0 conn(0x7f55f0072160 0x7f55f015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 -- 172.21.15.106:0/3883851698 >> 172.21.15.106:0/3883851698 conn(0x7f55f00b0c50 msgr2=0x7f55f00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:15.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 -- 172.21.15.106:0/3883851698 shutdown_connections 2023-11-24T08:11:15.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.186+0000 7f55f835a640 1 -- 172.21.15.106:0/3883851698 wait complete. 2023-11-24T08:11:15.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.188+0000 7f55f835a640 1 Processor -- start 2023-11-24T08:11:15.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.188+0000 7f55f835a640 1 -- start start 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f835a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 0x7f55f014c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f835a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f0072160 0x7f55f015c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f835a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f00c49f0 0x7f55f00b3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f835a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015a210 con 0x7f55f006dc40 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f835a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015b4c0 con 0x7f55f0072160 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f835a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f015c320 con 0x7f55f00c49f0 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.189+0000 7f55f60f6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 0x7f55f014c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 0x7f55f014c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34382/0 (socket says 172.21.15.106:34382) 2023-11-24T08:11:15.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 -- 172.21.15.106:0/672931926 learned_addr learned my addr 172.21.15.106:0/672931926 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f68f7640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f00c49f0 0x7f55f00b3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f58f5640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f0072160 0x7f55f015c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 -- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f00c49f0 msgr2=0x7f55f00b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f00c49f0 0x7f55f00b3250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 -- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f0072160 msgr2=0x7f55f015c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f0072160 0x7f55f015c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 -- 172.21.15.106:0/672931926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55e000b770 con 0x7f55f006dc40 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f58f5640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f0072160 0x7f55f015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.190+0000 7f55f60f6640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 0x7f55f014c260 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d830 tx=0x7f55ec00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.191+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00b6c0 con 0x7f55f006dc40 2023-11-24T08:11:15.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.191+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec00b820 con 0x7f55f006dc40 2023-11-24T08:11:15.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.191+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec00bbb0 con 0x7f55f006dc40 2023-11-24T08:11:15.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.191+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f015b710 con 0x7f55f006dc40 2023-11-24T08:11:15.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.191+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f015bcd0 con 0x7f55f006dc40 2023-11-24T08:11:15.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.192+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f004e8b0 con 0x7f55f006dc40 2023-11-24T08:11:15.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.193+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f55ec01e460 con 0x7f55f006dc40 2023-11-24T08:11:15.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.194+0000 7f55e6ffd640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f55d406b080 0x7f55d406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.194+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f55ec08a2c0 con 0x7f55f006dc40 2023-11-24T08:11:15.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.194+0000 7f55f58f5640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f55d406b080 0x7f55d406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.195+0000 7f55f58f5640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f55d406b080 0x7f55d406d540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55e0038000 tx=0x7f55e0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:15.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.198+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f55ec054f30 con 0x7f55f006dc40 2023-11-24T08:11:15.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.389+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f55f00c9390 con 0x7f55f006dc40 2023-11-24T08:11:15.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.390+0000 7f55e6ffd640 1 -- 172.21.15.106:0/672931926 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v49) v1 ==== 100+0+11 (secure 0 0 0) 0x7f55ec058550 con 0x7f55f006dc40 2023-11-24T08:11:15.393 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f55d406b080 msgr2=0x7f55d406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f55d406b080 0x7f55d406d540 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f55e0038000 tx=0x7f55e0008040 comp rx=0 tx=0).stop 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 msgr2=0x7f55f014c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 0x7f55f014c260 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00d830 tx=0x7f55ec00dcf0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 shutdown_connections 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f55d406b080 0x7f55d406d540 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f55f00c49f0 0x7f55f00b3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f0072160 0x7f55f015c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 --2- 172.21.15.106:0/672931926 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f55f006dc40 0x7f55f014c260 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 >> 172.21.15.106:0/672931926 conn(0x7f55f00b0c50 msgr2=0x7f55f00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:15.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.392+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 shutdown_connections 2023-11-24T08:11:15.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.393+0000 7f55f835a640 1 -- 172.21.15.106:0/672931926 wait complete. 2023-11-24T08:11:15.413 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:15.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 Processor -- start 2023-11-24T08:11:15.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 -- start start 2023-11-24T08:11:15.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b7c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b7c0b2a70 0x7f7b7c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b7c15ab80 con 0x7f7b7c0b2630 2023-11-24T08:11:15.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b7c15a990 con 0x7f7b7c0b2a70 2023-11-24T08:11:15.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.625+0000 7f7b82c32640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c15e680 con 0x7f7b7c0b2250 2023-11-24T08:11:15.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.626+0000 7f7b73fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7b7c0b2630 0x7f7b7c0ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52168/0 (socket says 172.21.15.106:52168) 2023-11-24T08:11:15.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.626+0000 7f7b73fff640 1 -- 172.21.15.106:0/3606472222 learned_addr learned my addr 172.21.15.106:0/3606472222 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:15.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.626+0000 7f7b811cf640 1 --2- 172.21.15.106:0/3606472222 >> v2:172.21.15.156:3300/0 conn(0x7f7b7c0b2a70 0x7f7b7c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.626+0000 7f7b809ce640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b7c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.626+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 127168561 0 0) 0x7f7b7c15ab80 con 0x7f7b7c0b2630 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.626+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b64003520 con 0x7f7b7c0b2630 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3058542362 0 0) 0x7f7b64003520 con 0x7f7b7c0b2630 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 msgr2=0x7f7b7c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b7c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 >> v2:172.21.15.156:3300/0 conn(0x7f7b7c0b2a70 msgr2=0x7f7b7c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 --2- 172.21.15.106:0/3606472222 >> v2:172.21.15.156:3300/0 conn(0x7f7b7c0b2a70 0x7f7b7c15b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c15e960 con 0x7f7b7c0b2630 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f7b6c001670 con 0x7f7b7c0b2630 2023-11-24T08:11:15.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 >> v1:172.21.15.106:6789/0 conn(0x7f7b7c0b2630 legacy=0x7f7b7c0ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b64003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b64003ef0 0x7f7b640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b64004d00 0x7f7b64005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c15e680 con 0x7f7b64004d00 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400b980 con 0x7f7b64003ef0 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400bac0 con 0x7f7b7c0b2250 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b811cf640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b64004d00 0x7f7b64005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.627+0000 7f7b809ce640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b64003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.628+0000 7f7b811cf640 1 -- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 msgr2=0x7f7b64003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.628+0000 7f7b811cf640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b64003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.628+0000 7f7b811cf640 1 -- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b64003ef0 msgr2=0x7f7b640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.628+0000 7f7b811cf640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b64003ef0 0x7f7b640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.628+0000 7f7b811cf640 1 -- 172.21.15.106:0/3606472222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64003520 con 0x7f7b64004d00 2023-11-24T08:11:15.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.628+0000 7f7b809ce640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c0b2250 0x7f7b64003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b811cf640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b64004d00 0x7f7b64005180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009fc0 tx=0x7f7b640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bcf7274f18abc10 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7402cc40 con 0x7f7b64004d00 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7400f040 con 0x7f7b64004d00 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b737fe640 1 -- 172.21.15.106:0/3606472222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74034440 con 0x7f7b64004d00 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b82c32640 1 -- 172.21.15.106:0/3606472222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b64004d00 msgr2=0x7f7b64005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b82c32640 1 --2- 172.21.15.106:0/3606472222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b64004d00 0x7f7b64005180 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009fc0 tx=0x7f7b640057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.629+0000 7f7b811cf640 1 -- 172.21.15.106:0/3606472222 reap_dead start 2023-11-24T08:11:15.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.630+0000 7f7b82c32640 1 -- 172.21.15.106:0/3606472222 shutdown_connections 2023-11-24T08:11:15.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.630+0000 7f7b82c32640 1 -- 172.21.15.106:0/3606472222 >> 172.21.15.106:0/3606472222 conn(0x7f7b7c06a920 msgr2=0x7f7b7c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:15.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.630+0000 7f7b82c32640 1 -- 172.21.15.106:0/3606472222 shutdown_connections 2023-11-24T08:11:15.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.630+0000 7f7b82c32640 1 -- 172.21.15.106:0/3606472222 wait complete. 2023-11-24T08:11:15.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.631+0000 7f7b82c32640 1 Processor -- start 2023-11-24T08:11:15.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 -- start start 2023-11-24T08:11:15.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c14ecd0 0x7f7b7c14c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 0x7f7b7c14d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b7c0cc2e0 0x7f7b7c14d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c14c1d0 con 0x7f7b7c151a30 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c151f80 con 0x7f7b7c0cc2e0 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.632+0000 7f7b82c32640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7c152260 con 0x7f7b7c14ecd0 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 0x7f7b7c14d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 0x7f7b7c14d330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34408/0 (socket says 172.21.15.106:34408) 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 -- 172.21.15.106:0/1676189930 learned_addr learned my addr 172.21.15.106:0/1676189930 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b809ce640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c14ecd0 0x7f7b7c14c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 -- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c14ecd0 msgr2=0x7f7b7c14c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c14ecd0 0x7f7b7c14c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 -- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b7c0cc2e0 msgr2=0x7f7b7c14d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b7c0cc2e0 0x7f7b7c14d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 -- 172.21.15.106:0/1676189930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7c0cd7c0 con 0x7f7b7c151a30 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.633+0000 7f7b73fff640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 0x7f7b7c14d330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00caa0 tx=0x7f7b6c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.634+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c004cb0 con 0x7f7b7c151a30 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.634+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c004e30 con 0x7f7b7c151a30 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.634+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c0072f0 con 0x7f7b7c151a30 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.634+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7c152540 con 0x7f7b7c151a30 2023-11-24T08:11:15.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.634+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7c14ddb0 con 0x7f7b7c151a30 2023-11-24T08:11:15.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.635+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b7c14c360 con 0x7f7b7c151a30 2023-11-24T08:11:15.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.636+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7b6c007450 con 0x7f7b7c151a30 2023-11-24T08:11:15.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.637+0000 7f7b717fa640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b6806b080 0x7f7b6806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:15.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.637+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f7b6c089050 con 0x7f7b7c151a30 2023-11-24T08:11:15.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.637+0000 7f7b809ce640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b6806b080 0x7f7b6806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:15.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.638+0000 7f7b809ce640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b6806b080 0x7f7b6806d540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b780099c0 tx=0x7f7b78008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:15.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.641+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b6c053d60 con 0x7f7b7c151a30 2023-11-24T08:11:15.859 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:11:15.868 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:11:15.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.869+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b7c067470 con 0x7f7b7c151a30 2023-11-24T08:11:15.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.870+0000 7f7b717fa640 1 -- 172.21.15.106:0/1676189930 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f7b6c018020 con 0x7f7b7c151a30 2023-11-24T08:11:15.872 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:15.872 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:10:40.829930+0000","modified":"2023-11-24T08:10:43.906826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1634060253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1634060253},{"type":"v1","addr":"172.21.15.106:6837","nonce":1634060253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:11:15.872 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b6806b080 msgr2=0x7f7b6806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b6806b080 0x7f7b6806d540 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7b780099c0 tx=0x7f7b78008040 comp rx=0 tx=0).stop 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 msgr2=0x7f7b7c14d330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 0x7f7b7c14d330 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c00caa0 tx=0x7f7b6c00cf60 comp rx=0 tx=0).stop 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 shutdown_connections 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b7c0cc2e0 0x7f7b7c14d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b7c151a30 0x7f7b7c14d330 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b6806b080 0x7f7b6806d540 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 --2- 172.21.15.106:0/1676189930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b7c14ecd0 0x7f7b7c14c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:15.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 >> 172.21.15.106:0/1676189930 conn(0x7f7b7c06a920 msgr2=0x7f7b7c15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:15.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 shutdown_connections 2023-11-24T08:11:15.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:15.873+0000 7f7b82c32640 1 -- 172.21.15.106:0/1676189930 wait complete. 2023-11-24T08:11:15.893 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:16.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.128+0000 7f5254d83640 1 Processor -- start 2023-11-24T08:11:16.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.128+0000 7f5254d83640 1 -- start start 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f5254d83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f5254d83640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f52500b21d0 0x7f52500cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f5254d83640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52500b3a40 con 0x7f52500cbf80 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f5254d83640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52500cc410 con 0x7f52500b21d0 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f5254d83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52500cdfa0 con 0x7f5250159ce0 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f524dd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f52500b21d0 0x7f52500cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f524dd9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f52500b21d0 0x7f52500cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34412/0 (socket says 172.21.15.106:34412) 2023-11-24T08:11:16.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.130+0000 7f524dd9b640 1 -- 172.21.15.106:0/863362320 learned_addr learned my addr 172.21.15.106:0/863362320 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:16.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.131+0000 7f524e59c640 1 --2- 172.21.15.106:0/863362320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.131+0000 7f524dd9b640 1 -- 172.21.15.106:0/863362320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 msgr2=0x7f525006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:16.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.131+0000 7f524dd9b640 1 --2- 172.21.15.106:0/863362320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525006de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.131+0000 7f524dd9b640 1 -- 172.21.15.106:0/863362320 >> v1:172.21.15.106:6789/0 conn(0x7f52500cbf80 legacy=0x7f525015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.131+0000 7f524dd9b640 1 -- 172.21.15.106:0/863362320 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52500cd9e0 con 0x7f52500b21d0 2023-11-24T08:11:16.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.132+0000 7f524dd9b640 1 --2- 172.21.15.106:0/863362320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500cba40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5238009870 tx=0x7f523802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db1ac4da0882912 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:16.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.133+0000 7f524d59a640 1 -- 172.21.15.106:0/863362320 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238030840 con 0x7f52500b21d0 2023-11-24T08:11:16.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.133+0000 7f524d59a640 1 -- 172.21.15.106:0/863362320 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52380309a0 con 0x7f52500b21d0 2023-11-24T08:11:16.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.133+0000 7f524d59a640 1 -- 172.21.15.106:0/863362320 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238030d30 con 0x7f52500b21d0 2023-11-24T08:11:16.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.133+0000 7f5254d83640 1 -- 172.21.15.106:0/863362320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 msgr2=0x7f52500cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.133+0000 7f5254d83640 1 --2- 172.21.15.106:0/863362320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500cba40 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f5238009870 tx=0x7f523802b980 comp rx=0 tx=0).stop 2023-11-24T08:11:16.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.133+0000 7f5254d83640 1 -- 172.21.15.106:0/863362320 shutdown_connections 2023-11-24T08:11:16.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.134+0000 7f5254d83640 1 --2- 172.21.15.106:0/863362320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500cba40 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.134+0000 7f5254d83640 1 --2- 172.21.15.106:0/863362320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.134+0000 7f5254d83640 1 -- 172.21.15.106:0/863362320 >> 172.21.15.106:0/863362320 conn(0x7f525006a920 msgr2=0x7f525006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:16.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.134+0000 7f5254d83640 1 -- 172.21.15.106:0/863362320 shutdown_connections 2023-11-24T08:11:16.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.134+0000 7f5254d83640 1 -- 172.21.15.106:0/863362320 wait complete. 2023-11-24T08:11:16.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.135+0000 7f5254d83640 1 Processor -- start 2023-11-24T08:11:16.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 -- start start 2023-11-24T08:11:16.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52500cbf80 0x7f525014ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525014d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525014bfb0 con 0x7f52500b21d0 2023-11-24T08:11:16.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f525014f6f0 con 0x7f52500cbf80 2023-11-24T08:11:16.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.136+0000 7f5254d83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f525014f940 con 0x7f5250159ce0 2023-11-24T08:11:16.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500afb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34428/0 (socket says 172.21.15.106:34428) 2023-11-24T08:11:16.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 -- 172.21.15.106:0/995231403 learned_addr learned my addr 172.21.15.106:0/995231403 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 -- 172.21.15.106:0/995231403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 msgr2=0x7f525014d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525014d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 -- 172.21.15.106:0/995231403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52500cbf80 msgr2=0x7f525014ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52500cbf80 0x7f525014ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 -- 172.21.15.106:0/995231403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380094d0 con 0x7f52500b21d0 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.137+0000 7f524e59c640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500afb40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5240007e40 tx=0x7f524000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.138+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5240014620 con 0x7f52500b21d0 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.138+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524000ee90 con 0x7f52500b21d0 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.138+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52400149c0 con 0x7f52500b21d0 2023-11-24T08:11:16.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.138+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f525014fc20 con 0x7f52500b21d0 2023-11-24T08:11:16.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.138+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52501501e0 con 0x7f52500b21d0 2023-11-24T08:11:16.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.139+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5250150370 con 0x7f52500b21d0 2023-11-24T08:11:16.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.140+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f5240014b20 con 0x7f52500b21d0 2023-11-24T08:11:16.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.141+0000 7f5246ffd640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f523006b080 0x7f523006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.141+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f524008a300 con 0x7f52500b21d0 2023-11-24T08:11:16.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.141+0000 7f524dd9b640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f523006b080 0x7f523006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.142+0000 7f524dd9b640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f523006b080 0x7f523006d540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5238036000 tx=0x7f523803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:16.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.145+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52400550d0 con 0x7f52500b21d0 2023-11-24T08:11:16.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.340+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5250067470 con 0x7f52500b21d0 2023-11-24T08:11:16.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.341+0000 7f5246ffd640 1 -- 172.21.15.106:0/995231403 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f52400586f0 con 0x7f52500b21d0 2023-11-24T08:11:16.343 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:16.343 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":49,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:10.401248+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:16.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f523006b080 msgr2=0x7f523006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f523006b080 0x7f523006d540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5238036000 tx=0x7f523803b040 comp rx=0 tx=0).stop 2023-11-24T08:11:16.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 msgr2=0x7f52500afb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500afb40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5240007e40 tx=0x7f524000e4b0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 shutdown_connections 2023-11-24T08:11:16.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f523006b080 0x7f523006d540 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5250159ce0 0x7f525014d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52500cbf80 0x7f525014ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 --2- 172.21.15.106:0/995231403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f52500b21d0 0x7f52500afb40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 >> 172.21.15.106:0/995231403 conn(0x7f525006a920 msgr2=0x7f52501531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:16.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 shutdown_connections 2023-11-24T08:11:16.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.344+0000 7f5254d83640 1 -- 172.21.15.106:0/995231403 wait complete. 2023-11-24T08:11:16.365 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:11:16.365 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.573+0000 7f01dd25c640 1 Processor -- start 2023-11-24T08:11:16.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.573+0000 7f01dd25c640 1 -- start start 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01dd25c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01d815c7f0 0x7f01d806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01dd25c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01d80cf450 0x7f01d80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01dd25c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80cece0 con 0x7f01d80cdfb0 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01dd25c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80cfe40 con 0x7f01d815c7f0 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01dd25c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80ceaf0 con 0x7f01d80cf450 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01d6d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01d815c7f0 0x7f01d806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01d6d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f01d815c7f0 0x7f01d806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34442/0 (socket says 172.21.15.106:34442) 2023-11-24T08:11:16.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.574+0000 7f01d6d9d640 1 -- 172.21.15.106:0/3139556996 learned_addr learned my addr 172.21.15.106:0/3139556996 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.575+0000 7f01d6d9d640 1 -- 172.21.15.106:0/3139556996 >> v2:172.21.15.156:3300/0 conn(0x7f01d80cf450 msgr2=0x7f01d80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.575+0000 7f01d6d9d640 1 --2- 172.21.15.106:0/3139556996 >> v2:172.21.15.156:3300/0 conn(0x7f01d80cf450 0x7f01d80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.575+0000 7f01d6d9d640 1 -- 172.21.15.106:0/3139556996 >> v1:172.21.15.156:6789/0 conn(0x7f01d80cdfb0 legacy=0x7f01d80ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:16.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.575+0000 7f01d6d9d640 1 -- 172.21.15.106:0/3139556996 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d806d040 con 0x7f01d815c7f0 2023-11-24T08:11:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.576+0000 7f01d6d9d640 1 --2- 172.21.15.106:0/3139556996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d815c7f0 0x7f01d806db20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f01d8070410 tx=0x7f01c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7980cef809268090 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.576+0000 7f01d659c640 1 -- 172.21.15.106:0/3139556996 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8031870 con 0x7f01d815c7f0 2023-11-24T08:11:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.576+0000 7f01d659c640 1 -- 172.21.15.106:0/3139556996 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c80319d0 con 0x7f01d815c7f0 2023-11-24T08:11:16.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.576+0000 7f01d659c640 1 -- 172.21.15.106:0/3139556996 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8031d60 con 0x7f01d815c7f0 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 -- 172.21.15.106:0/3139556996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d815c7f0 msgr2=0x7f01d806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 --2- 172.21.15.106:0/3139556996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d815c7f0 0x7f01d806db20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f01d8070410 tx=0x7f01c802b910 comp rx=0 tx=0).stop 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 -- 172.21.15.106:0/3139556996 shutdown_connections 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 --2- 172.21.15.106:0/3139556996 >> v2:172.21.15.156:3300/0 conn(0x7f01d80cf450 0x7f01d80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 --2- 172.21.15.106:0/3139556996 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d815c7f0 0x7f01d806db20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 -- 172.21.15.106:0/3139556996 >> 172.21.15.106:0/3139556996 conn(0x7f01d806a920 msgr2=0x7f01d806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 -- 172.21.15.106:0/3139556996 shutdown_connections 2023-11-24T08:11:16.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.577+0000 7f01dd25c640 1 -- 172.21.15.106:0/3139556996 wait complete. 2023-11-24T08:11:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.579+0000 7f01dd25c640 1 Processor -- start 2023-11-24T08:11:16.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.579+0000 7f01dd25c640 1 -- start start 2023-11-24T08:11:16.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01dd25c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cdfb0 0x7f01d81f7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01dd25c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 0x7f01d8151c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01dd25c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d815c7f0 0x7f01d815a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01dd25c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8099ca0 con 0x7f01d80cf450 2023-11-24T08:11:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01dd25c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80996e0 con 0x7f01d815c7f0 2023-11-24T08:11:16.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01dd25c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80999c0 con 0x7f01d80cdfb0 2023-11-24T08:11:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 0x7f01d8151c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 0x7f01d8151c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34454/0 (socket says 172.21.15.106:34454) 2023-11-24T08:11:16.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.580+0000 7f01cffff640 1 -- 172.21.15.106:0/527066964 learned_addr learned my addr 172.21.15.106:0/527066964 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:16.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cffff640 1 -- 172.21.15.106:0/527066964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cdfb0 msgr2=0x7f01d81f7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cffff640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cdfb0 0x7f01d81f7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cffff640 1 -- 172.21.15.106:0/527066964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d815c7f0 msgr2=0x7f01d815a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cffff640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d815c7f0 0x7f01d815a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cffff640 1 -- 172.21.15.106:0/527066964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80094d0 con 0x7f01d80cf450 2023-11-24T08:11:16.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cffff640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 0x7f01d8151c10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01c000b4a0 tx=0x7f01c000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:16.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0010610 con 0x7f01d80cf450 2023-11-24T08:11:16.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0010770 con 0x7f01d80cf450 2023-11-24T08:11:16.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0010b20 con 0x7f01d80cf450 2023-11-24T08:11:16.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.581+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d81f7320 con 0x7f01d80cf450 2023-11-24T08:11:16.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.582+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81f8060 con 0x7f01d80cf450 2023-11-24T08:11:16.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.583+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d81f7180 con 0x7f01d80cf450 2023-11-24T08:11:16.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.583+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f01c0010c80 con 0x7f01d80cf450 2023-11-24T08:11:16.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.585+0000 7f01cf7fe640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01b806b080 0x7f01b806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:16.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.585+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f01c0089740 con 0x7f01d80cf450 2023-11-24T08:11:16.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.585+0000 7f01d6d9d640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01b806b080 0x7f01b806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:16.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.586+0000 7f01d6d9d640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01b806b080 0x7f01b806d540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01c8036000 tx=0x7f01c8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:16.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.589+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01d81f7180 con 0x7f01d80cf450 2023-11-24T08:11:16.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.822+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f01d81f7180 con 0x7f01d80cf450 2023-11-24T08:11:16.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.823+0000 7f01cf7fe640 1 -- 172.21.15.106:0/527066964 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f01d81f7180 con 0x7f01d80cf450 2023-11-24T08:11:16.825 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:16.825 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:10:40.829930+0000","modified":"2023-11-24T08:10:43.906826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1634060253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1634060253},{"type":"v1","addr":"172.21.15.106:6837","nonce":1634060253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:11:16.825 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2023-11-24T08:11:16.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01b806b080 msgr2=0x7f01b806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01b806b080 0x7f01b806d540 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f01c8036000 tx=0x7f01c8030000 comp rx=0 tx=0).stop 2023-11-24T08:11:16.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 msgr2=0x7f01d8151c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:16.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 0x7f01d8151c10 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f01c000b4a0 tx=0x7f01c000b960 comp rx=0 tx=0).stop 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 shutdown_connections 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d815c7f0 0x7f01d815a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01d80cf450 0x7f01d8151c10 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01b806b080 0x7f01b806d540 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 --2- 172.21.15.106:0/527066964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01d80cdfb0 0x7f01d81f7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 >> 172.21.15.106:0/527066964 conn(0x7f01d806a920 msgr2=0x7f01d815eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:16.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 shutdown_connections 2023-11-24T08:11:16.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:16.826+0000 7f01dd25c640 1 -- 172.21.15.106:0/527066964 wait complete. 2023-11-24T08:11:16.847 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:17.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.057+0000 7f9a56444640 1 Processor -- start 2023-11-24T08:11:17.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.057+0000 7f9a56444640 1 -- start start 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.058+0000 7f9a56444640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a500b2250 0x7f9a5015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.058+0000 7f9a56444640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f9a500b2630 0x7f9a500ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.058+0000 7f9a56444640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5015ab80 con 0x7f9a500b2aa0 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.058+0000 7f9a56444640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5015a990 con 0x7f9a500b2250 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.058+0000 7f9a56444640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5015e680 con 0x7f9a500b2630 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a500b2250 0x7f9a5015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9a500b2250 0x7f9a5015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34456/0 (socket says 172.21.15.106:34456) 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 -- 172.21.15.106:0/1084522684 learned_addr learned my addr 172.21.15.106:0/1084522684 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:17.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 -- 172.21.15.106:0/1084522684 >> v2:172.21.15.156:3300/0 conn(0x7f9a500b2630 msgr2=0x7f9a500ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:17.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 --2- 172.21.15.106:0/1084522684 >> v2:172.21.15.156:3300/0 conn(0x7f9a500b2630 0x7f9a500ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 -- 172.21.15.106:0/1084522684 >> v1:172.21.15.156:6789/0 conn(0x7f9a500b2aa0 legacy=0x7f9a5015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:17.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.059+0000 7f9a4ffff640 1 -- 172.21.15.106:0/1084522684 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5015e960 con 0x7f9a500b2250 2023-11-24T08:11:17.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.060+0000 7f9a4ffff640 1 --2- 172.21.15.106:0/1084522684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2250 0x7f9a5015df80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9a500ae5c0 tx=0x7f9a4002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959fb8637b889c48 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a4f7fe640 1 -- 172.21.15.106:0/1084522684 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40031870 con 0x7f9a500b2250 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a4f7fe640 1 -- 172.21.15.106:0/1084522684 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a400319d0 con 0x7f9a500b2250 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a4f7fe640 1 -- 172.21.15.106:0/1084522684 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40031d60 con 0x7f9a500b2250 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 -- 172.21.15.106:0/1084522684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2250 msgr2=0x7f9a5015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 --2- 172.21.15.106:0/1084522684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2250 0x7f9a5015df80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9a500ae5c0 tx=0x7f9a4002b910 comp rx=0 tx=0).stop 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 -- 172.21.15.106:0/1084522684 shutdown_connections 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 --2- 172.21.15.106:0/1084522684 >> v2:172.21.15.156:3300/0 conn(0x7f9a500b2630 0x7f9a500ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 --2- 172.21.15.106:0/1084522684 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2250 0x7f9a5015df80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 -- 172.21.15.106:0/1084522684 >> 172.21.15.106:0/1084522684 conn(0x7f9a5006a920 msgr2=0x7f9a5006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:17.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 -- 172.21.15.106:0/1084522684 shutdown_connections 2023-11-24T08:11:17.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.061+0000 7f9a56444640 1 -- 172.21.15.106:0/1084522684 wait complete. 2023-11-24T08:11:17.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.063+0000 7f9a56444640 1 Processor -- start 2023-11-24T08:11:17.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.063+0000 7f9a56444640 1 -- start start 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.064+0000 7f9a56444640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a500b2250 0x7f9a501544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.064+0000 7f9a56444640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 0x7f9a501568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.064+0000 7f9a56444640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a500b2aa0 0x7f9a501594d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.064+0000 7f9a56444640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a500717d0 con 0x7f9a500b2630 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.064+0000 7f9a56444640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50071ab0 con 0x7f9a500b2aa0 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.064+0000 7f9a56444640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a50155b00 con 0x7f9a500b2250 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 0x7f9a501568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 0x7f9a501568c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34460/0 (socket says 172.21.15.106:34460) 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 -- 172.21.15.106:0/2826167691 learned_addr learned my addr 172.21.15.106:0/2826167691 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a4ffff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a500b2250 0x7f9a501544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 -- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a500b2250 msgr2=0x7f9a501544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a500b2250 0x7f9a501544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 -- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a500b2aa0 msgr2=0x7f9a501594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a549e1640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a500b2aa0 0x7f9a501594d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a500b2aa0 0x7f9a501594d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 -- 172.21.15.106:0/2826167691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a400094d0 con 0x7f9a500b2630 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a4ffff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a500b2250 0x7f9a501544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:17.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.065+0000 7f9a47fff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 0x7f9a501568c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800b4a0 tx=0x7f9a3800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.066+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38010610 con 0x7f9a500b2630 2023-11-24T08:11:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.066+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a38010770 con 0x7f9a500b2630 2023-11-24T08:11:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.066+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a38010b20 con 0x7f9a500b2630 2023-11-24T08:11:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.066+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a50155de0 con 0x7f9a500b2630 2023-11-24T08:11:17.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.066+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a501563a0 con 0x7f9a500b2630 2023-11-24T08:11:17.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.067+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a501565c0 con 0x7f9a500b2630 2023-11-24T08:11:17.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.068+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9a38010c80 con 0x7f9a500b2630 2023-11-24T08:11:17.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.069+0000 7f9a4d7fa640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a3006b080 0x7f9a3006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.069+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9a380896c0 con 0x7f9a500b2630 2023-11-24T08:11:17.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.069+0000 7f9a4ffff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a3006b080 0x7f9a3006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.070+0000 7f9a4ffff640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a3006b080 0x7f9a3006d540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9a40036000 tx=0x7f9a4003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.073+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9a380543b0 con 0x7f9a500b2630 2023-11-24T08:11:17.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.300+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a50067470 con 0x7f9a500b2630 2023-11-24T08:11:17.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.301+0000 7f9a4d7fa640 1 -- 172.21.15.106:0/2826167691 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4925 (secure 0 0 0) 0x7f9a380579d0 con 0x7f9a500b2630 2023-11-24T08:11:17.302 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:17.302 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":26,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:10:40.829930+0000","modified":"2023-11-24T08:10:43.906826+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5562":{"gid":5562,"name":"a","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1634060253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1634060253},{"type":"v1","addr":"172.21.15.106:6837","nonce":1634060253}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-24T08:11:17.303 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 26 2023-11-24T08:11:17.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.303+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a3006b080 msgr2=0x7f9a3006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.303+0000 7f9a56444640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a3006b080 0x7f9a3006d540 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f9a40036000 tx=0x7f9a4003c040 comp rx=0 tx=0).stop 2023-11-24T08:11:17.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.303+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 msgr2=0x7f9a501568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.303+0000 7f9a56444640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 0x7f9a501568c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800b4a0 tx=0x7f9a3800b960 comp rx=0 tx=0).stop 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 shutdown_connections 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9a500b2aa0 0x7f9a501594d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9a500b2630 0x7f9a501568c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9a3006b080 0x7f9a3006d540 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 --2- 172.21.15.106:0/2826167691 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9a500b2250 0x7f9a501544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 >> 172.21.15.106:0/2826167691 conn(0x7f9a5006a920 msgr2=0x7f9a5015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 shutdown_connections 2023-11-24T08:11:17.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.304+0000 7f9a56444640 1 -- 172.21.15.106:0/2826167691 wait complete. 2023-11-24T08:11:17.324 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.535+0000 7f19c8c56640 1 Processor -- start 2023-11-24T08:11:17.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.535+0000 7f19c8c56640 1 -- start start 2023-11-24T08:11:17.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19c8c56640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19c40d3400 0x7f19c40d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19c8c56640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f19c40d1040 0x7f19c40d1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19c8c56640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19c40d3da0 con 0x7f19c415d780 2023-11-24T08:11:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19c8c56640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40ce4f0 con 0x7f19c40d3400 2023-11-24T08:11:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19c8c56640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c40d1d30 con 0x7f19c40d1040 2023-11-24T08:11:17.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19b9d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19c40d3400 0x7f19c40d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19b9d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f19c40d3400 0x7f19c40d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34466/0 (socket says 172.21.15.106:34466) 2023-11-24T08:11:17.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.537+0000 7f19b9d9b640 1 -- 172.21.15.106:0/411517917 learned_addr learned my addr 172.21.15.106:0/411517917 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:17.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.538+0000 7f19c2d9d640 1 --2- 172.21.15.106:0/411517917 >> v2:172.21.15.156:3300/0 conn(0x7f19c40d1040 0x7f19c40d1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.538+0000 7f19b9d9b640 1 -- 172.21.15.106:0/411517917 >> v2:172.21.15.156:3300/0 conn(0x7f19c40d1040 msgr2=0x7f19c40d1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.538+0000 7f19b9d9b640 1 --2- 172.21.15.106:0/411517917 >> v2:172.21.15.156:3300/0 conn(0x7f19c40d1040 0x7f19c40d1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.538+0000 7f19b9d9b640 1 -- 172.21.15.106:0/411517917 >> v1:172.21.15.106:6789/0 conn(0x7f19c415d780 legacy=0x7f19c40b3180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.538+0000 7f19b9d9b640 1 -- 172.21.15.106:0/411517917 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40d2010 con 0x7f19c40d3400 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.538+0000 7f19c2d9d640 1 --2- 172.21.15.106:0/411517917 >> v2:172.21.15.156:3300/0 conn(0x7f19c40d1040 0x7f19c40d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.539+0000 7f19b9d9b640 1 --2- 172.21.15.106:0/411517917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40d3860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b4de0 tx=0x7f19ac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f99f366ba0e2d168 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.539+0000 7f19c1d9b640 1 -- 172.21.15.106:0/411517917 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0317e0 con 0x7f19c40d3400 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.539+0000 7f19c1d9b640 1 -- 172.21.15.106:0/411517917 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac031940 con 0x7f19c40d3400 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.539+0000 7f19c1d9b640 1 -- 172.21.15.106:0/411517917 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac031cd0 con 0x7f19c40d3400 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 -- 172.21.15.106:0/411517917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 msgr2=0x7f19c40d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 --2- 172.21.15.106:0/411517917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40d3860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f19c40b4de0 tx=0x7f19ac02b880 comp rx=0 tx=0).stop 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 -- 172.21.15.106:0/411517917 shutdown_connections 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 --2- 172.21.15.106:0/411517917 >> v2:172.21.15.156:3300/0 conn(0x7f19c40d1040 0x7f19c40d1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 --2- 172.21.15.106:0/411517917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40d3860 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 -- 172.21.15.106:0/411517917 >> 172.21.15.106:0/411517917 conn(0x7f19c408cec0 msgr2=0x7f19c409c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 -- 172.21.15.106:0/411517917 shutdown_connections 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.540+0000 7f19c8c56640 1 -- 172.21.15.106:0/411517917 wait complete. 2023-11-24T08:11:17.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.542+0000 7f19c8c56640 1 Processor -- start 2023-11-24T08:11:17.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.542+0000 7f19c8c56640 1 -- start start 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c8c56640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c40d1040 0x7f19c40c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c8c56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40b6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c8c56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19c415d780 0x7f19c4158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c8c56640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40b65d0 con 0x7f19c40d3400 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c8c56640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40b6850 con 0x7f19c40d1040 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c8c56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c40b6b30 con 0x7f19c415d780 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19b9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40b6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19b9d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40b6090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34468/0 (socket says 172.21.15.106:34468) 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19b9d9b640 1 -- 172.21.15.106:0/946921647 learned_addr learned my addr 172.21.15.106:0/946921647 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:17.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c2d9d640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19c415d780 0x7f19c4158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.543+0000 7f19c259c640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c40d1040 0x7f19c40c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19b9d9b640 1 -- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19c415d780 msgr2=0x7f19c4158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19b9d9b640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19c415d780 0x7f19c4158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19b9d9b640 1 -- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c40d1040 msgr2=0x7f19c40c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19b9d9b640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c40d1040 0x7f19c40c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19b9d9b640 1 -- 172.21.15.106:0/946921647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ac0094d0 con 0x7f19c40d3400 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19b9d9b640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40b6090 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac032a00 con 0x7f19c40d3400 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ac032b60 con 0x7f19c40d3400 2023-11-24T08:11:17.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ac0453f0 con 0x7f19c40d3400 2023-11-24T08:11:17.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.544+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c40b6e10 con 0x7f19c40d3400 2023-11-24T08:11:17.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.545+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c40b73d0 con 0x7f19c40d3400 2023-11-24T08:11:17.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.546+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c40b6c70 con 0x7f19c40d3400 2023-11-24T08:11:17.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.547+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f19ac04e450 con 0x7f19c40d3400 2023-11-24T08:11:17.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.548+0000 7f19bb7fe640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19a406b0b0 0x7f19a406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.548+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f19ac0abfa0 con 0x7f19c40d3400 2023-11-24T08:11:17.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.548+0000 7f19c259c640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19a406b0b0 0x7f19a406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.549+0000 7f19c259c640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19a406b0b0 0x7f19a406d570 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f19b40012e0 tx=0x7f19b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.552+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f19c40b6c70 con 0x7f19c40d3400 2023-11-24T08:11:17.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.745+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19c40b6c70 con 0x7f19c40d3400 2023-11-24T08:11:17.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.747+0000 7f19bb7fe640 1 -- 172.21.15.106:0/946921647 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16669 (secure 0 0 0) 0x7f19c40b6c70 con 0x7f19c40d3400 2023-11-24T08:11:17.749 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:17.749 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":49,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:10.401248+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:10:38.577975+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-24T08:10:39.853606+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":49,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:17.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19a406b0b0 msgr2=0x7f19a406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19a406b0b0 0x7f19a406d570 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f19b40012e0 tx=0x7f19b4009040 comp rx=0 tx=0).stop 2023-11-24T08:11:17.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 msgr2=0x7f19c40b6090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40b6090 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f19ac000c00 tx=0x7f19ac0046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 shutdown_connections 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f19a406b0b0 0x7f19a406d570 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f19c415d780 0x7f19c4158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f19c40d3400 0x7f19c40b6090 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 --2- 172.21.15.106:0/946921647 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19c40d1040 0x7f19c40c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.751+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 >> 172.21.15.106:0/946921647 conn(0x7f19c408cec0 msgr2=0x7f19c40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.752+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 shutdown_connections 2023-11-24T08:11:17.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.752+0000 7f19c8c56640 1 -- 172.21.15.106:0/946921647 wait complete. 2023-11-24T08:11:17.773 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:11:17.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.984+0000 7f8369197640 1 Processor -- start 2023-11-24T08:11:17.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.984+0000 7f8369197640 1 -- start start 2023-11-24T08:11:17.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.986+0000 7f8369197640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f83640b3790 0x7f836406adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.986+0000 7f8369197640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f836406dcf0 con 0x7f8364161060 2023-11-24T08:11:17.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.986+0000 7f8369197640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8364161ec0 con 0x7f83640b3330 2023-11-24T08:11:17.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.986+0000 7f8369197640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f836406ec10 con 0x7f83640b3790 2023-11-24T08:11:17.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.986+0000 7f8362d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8364161060 0x7f8364161440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52204/0 (socket says 172.21.15.106:52204) 2023-11-24T08:11:17.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.986+0000 7f8362d9d640 1 -- 172.21.15.106:0/1807631242 learned_addr learned my addr 172.21.15.106:0/1807631242 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f836359e640 1 --2- 172.21.15.106:0/1807631242 >> v2:172.21.15.156:3300/0 conn(0x7f83640b3790 0x7f836406adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3200899279 0 0) 0x7f836406dcf0 con 0x7f8364161060 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f834c003520 con 0x7f8364161060 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3899030701 0 0) 0x7f834c003520 con 0x7f8364161060 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 >> v2:172.21.15.156:3300/0 conn(0x7f83640b3790 msgr2=0x7f836406adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 --2- 172.21.15.106:0/1807631242 >> v2:172.21.15.156:3300/0 conn(0x7f83640b3790 0x7f836406adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 >> v1:172.21.15.156:6789/0 conn(0x7f83640b3330 legacy=0x7f836406a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f836406df90 con 0x7f8364161060 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f8354001670 con 0x7f8364161060 2023-11-24T08:11:17.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 >> v1:172.21.15.106:6789/0 conn(0x7f8364161060 legacy=0x7f8364161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c003970 0x7f834c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f834c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f834c005180 0x7f834c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f836406ec10 con 0x7f834c0047c0 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f834c00c120 con 0x7f834c003970 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.987+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f834c00c2e0 con 0x7f834c005180 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836259c640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f834c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836259c640 1 -- 172.21.15.106:0/1807631242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f834c005180 msgr2=0x7f834c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836259c640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f834c005180 0x7f834c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836259c640 1 -- 172.21.15.106:0/1807631242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c003970 msgr2=0x7f834c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836259c640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f834c003970 0x7f834c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836259c640 1 -- 172.21.15.106:0/1807631242 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c003520 con 0x7f834c0047c0 2023-11-24T08:11:17.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.988+0000 7f836359e640 1 -- 172.21.15.106:0/1807631242 reap_dead start 2023-11-24T08:11:17.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.989+0000 7f836259c640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f834c004c40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8358001770 tx=0x7f834c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afec0a1d236a8506 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.989+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83580052d0 con 0x7f834c0047c0 2023-11-24T08:11:17.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.989+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835802e050 con 0x7f834c0047c0 2023-11-24T08:11:17.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.989+0000 7f8361d9b640 1 -- 172.21.15.106:0/1807631242 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8358003640 con 0x7f834c0047c0 2023-11-24T08:11:17.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 -- 172.21.15.106:0/1807631242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 msgr2=0x7f834c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f834c004c40 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8358001770 tx=0x7f834c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 -- 172.21.15.106:0/1807631242 shutdown_connections 2023-11-24T08:11:17.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 --2- 172.21.15.106:0/1807631242 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f834c004c40 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 -- 172.21.15.106:0/1807631242 >> 172.21.15.106:0/1807631242 conn(0x7f83640cf900 msgr2=0x7f83640cfd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:17.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 -- 172.21.15.106:0/1807631242 shutdown_connections 2023-11-24T08:11:17.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.990+0000 7f8369197640 1 -- 172.21.15.106:0/1807631242 wait complete. 2023-11-24T08:11:17.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.992+0000 7f8369197640 1 Processor -- start 2023-11-24T08:11:17.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.992+0000 7f8369197640 1 -- start start 2023-11-24T08:11:17.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8369197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f83641e82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8369197640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f836413f290 0x7f83640ccd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8369197640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83640cd2c0 0x7f83641e8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8369197640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641e7e80 con 0x7f834c0047c0 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8369197640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641e87e0 con 0x7f83640cd2c0 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8369197640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83641e7200 con 0x7f836413f290 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8362d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f83641e82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8362d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f83641e82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34486/0 (socket says 172.21.15.106:34486) 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.993+0000 7f8362d9d640 1 -- 172.21.15.106:0/3719400511 learned_addr learned my addr 172.21.15.106:0/3719400511 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8362d9d640 1 -- 172.21.15.106:0/3719400511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f836413f290 msgr2=0x7f83640ccd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8362d9d640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f836413f290 0x7f83640ccd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8362d9d640 1 -- 172.21.15.106:0/3719400511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83640cd2c0 msgr2=0x7f83641e8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8362d9d640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83640cd2c0 0x7f83641e8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8362d9d640 1 -- 172.21.15.106:0/3719400511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83641e74e0 con 0x7f834c0047c0 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8362d9d640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f83641e82a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f835400c730 tx=0x7f835400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:17.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540072d0 con 0x7f834c0047c0 2023-11-24T08:11:17.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8354007450 con 0x7f834c0047c0 2023-11-24T08:11:17.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83540047e0 con 0x7f834c0047c0 2023-11-24T08:11:17.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.994+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83641e77c0 con 0x7f834c0047c0 2023-11-24T08:11:17.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.995+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83641e6e60 con 0x7f834c0047c0 2023-11-24T08:11:17.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.996+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8364067990 con 0x7f834c0047c0 2023-11-24T08:11:17.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.997+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8354004b70 con 0x7f834c0047c0 2023-11-24T08:11:17.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.997+0000 7f834b7fe640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f835006b0b0 0x7f835006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:17.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.998+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8354089080 con 0x7f834c0047c0 2023-11-24T08:11:17.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.998+0000 7f836259c640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f835006b0b0 0x7f835006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:18.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:17.999+0000 7f836259c640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f835006b0b0 0x7f835006d570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8358033000 tx=0x7f835802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:18.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.002+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f83540bbe90 con 0x7f834c0047c0 2023-11-24T08:11:18.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.230+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f83641e7fc0 con 0x7f834c0047c0 2023-11-24T08:11:18.232 INFO:tasks.ceph.mon.a.smithi106.stderr:2023-11-24T08:11:18.231+0000 7fb1127dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:11:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.235+0000 7f834b7fe640 1 -- 172.21.15.106:0/3719400511 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f83641e7fc0 con 0x7f834c0047c0 2023-11-24T08:11:18.236 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:11:18.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.238+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f835006b0b0 msgr2=0x7f835006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:18.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.238+0000 7f8369197640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f835006b0b0 0x7f835006d570 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8358033000 tx=0x7f835802f000 comp rx=0 tx=0).stop 2023-11-24T08:11:18.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.238+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 msgr2=0x7f83641e82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:18.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.238+0000 7f8369197640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f83641e82a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f835400c730 tx=0x7f835400cbf0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 shutdown_connections 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83640cd2c0 0x7f83641e8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f835006b0b0 0x7f835006d570 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f836413f290 0x7f83640ccd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 --2- 172.21.15.106:0/3719400511 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f834c0047c0 0x7f83641e82a0 secure :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f835400c730 tx=0x7f835400cbf0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 >> 172.21.15.106:0/3719400511 conn(0x7f83640cf900 msgr2=0x7f83640ac8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 shutdown_connections 2023-11-24T08:11:18.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.239+0000 7f8369197640 1 -- 172.21.15.106:0/3719400511 wait complete. 2023-11-24T08:11:18.259 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:11:18.320 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:11:18.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.449+0000 7f083ee0e640 1 Processor -- start 2023-11-24T08:11:18.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.449+0000 7f083ee0e640 1 -- start start 2023-11-24T08:11:18.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f083ee0e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f083806de10 0x7f083815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:18.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f083ee0e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f083815ec90 con 0x7f0838072160 2023-11-24T08:11:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f083ee0e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08380cd530 con 0x7f08380c49f0 2023-11-24T08:11:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f083ee0e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08380c4ed0 con 0x7f083806de10 2023-11-24T08:11:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f0837fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f083806de10 0x7f083815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f083cbaa640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0838072160 0x7f083806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52232/0 (socket says 172.21.15.106:52232) 2023-11-24T08:11:18.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.451+0000 7f083cbaa640 1 -- 172.21.15.106:0/2956648443 learned_addr learned my addr 172.21.15.106:0/2956648443 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:18.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.452+0000 7f0837fff640 1 -- 172.21.15.106:0/2956648443 >> v1:172.21.15.156:6789/0 conn(0x7f08380c49f0 legacy=0x7f08380705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:18.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.452+0000 7f0837fff640 1 -- 172.21.15.106:0/2956648443 >> v1:172.21.15.106:6789/0 conn(0x7f0838072160 legacy=0x7f083806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:18.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.452+0000 7f0837fff640 1 -- 172.21.15.106:0/2956648443 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08380c51b0 con 0x7f083806de10 2023-11-24T08:11:18.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.453+0000 7f0837fff640 1 --2- 172.21.15.106:0/2956648443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083815e750 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f08240097a0 tx=0x7f082402dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1877747e335ba3a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.455+0000 7f08377fe640 1 -- 172.21.15.106:0/2956648443 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082402f8a0 con 0x7f083806de10 2023-11-24T08:11:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.455+0000 7f08377fe640 1 -- 172.21.15.106:0/2956648443 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082402fa00 con 0x7f083806de10 2023-11-24T08:11:18.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.455+0000 7f08377fe640 1 -- 172.21.15.106:0/2956648443 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082402fd90 con 0x7f083806de10 2023-11-24T08:11:18.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 -- 172.21.15.106:0/2956648443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 msgr2=0x7f083815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:18.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 --2- 172.21.15.106:0/2956648443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083815e750 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f08240097a0 tx=0x7f082402dae0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 -- 172.21.15.106:0/2956648443 shutdown_connections 2023-11-24T08:11:18.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 --2- 172.21.15.106:0/2956648443 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083815e750 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.459 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 -- 172.21.15.106:0/2956648443 >> 172.21.15.106:0/2956648443 conn(0x7f08380b0c50 msgr2=0x7f08380b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:18.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 -- 172.21.15.106:0/2956648443 shutdown_connections 2023-11-24T08:11:18.460 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.456+0000 7f083ee0e640 1 -- 172.21.15.106:0/2956648443 wait complete. 2023-11-24T08:11:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.458+0000 7f083ee0e640 1 Processor -- start 2023-11-24T08:11:18.462 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.458+0000 7f083ee0e640 1 -- start start 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083ee0e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083ee0e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0838072160 0x7f08380cb290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083ee0e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08380c49f0 0x7f08380cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083ee0e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083815d0c0 con 0x7f083806de10 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083ee0e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083815c440 con 0x7f0838072160 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083ee0e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083815c720 con 0x7f08380c49f0 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083cbaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083cbaa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083814ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34516/0 (socket says 172.21.15.106:34516) 2023-11-24T08:11:18.463 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.459+0000 7f083cbaa640 1 -- 172.21.15.106:0/1791871347 learned_addr learned my addr 172.21.15.106:0/1791871347 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083cbaa640 1 -- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08380c49f0 msgr2=0x7f08380cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083d3ab640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08380c49f0 0x7f08380cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083cbaa640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08380c49f0 0x7f08380cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083cbaa640 1 -- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0838072160 msgr2=0x7f08380cb290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083cbaa640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0838072160 0x7f08380cb290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083cbaa640 1 -- 172.21.15.106:0/1791871347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824009400 con 0x7f083806de10 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083d3ab640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08380c49f0 0x7f08380cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083cbaa640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083814ccc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c4a0 tx=0x7f082c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c005ce0 con 0x7f083806de10 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f082c005e40 con 0x7f083806de10 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f082c007270 con 0x7f083806de10 2023-11-24T08:11:18.464 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.460+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f083815ca00 con 0x7f083806de10 2023-11-24T08:11:18.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.461+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f083814d200 con 0x7f083806de10 2023-11-24T08:11:18.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.462+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f083815d3e0 con 0x7f083806de10 2023-11-24T08:11:18.465 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.463+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f082c007490 con 0x7f083806de10 2023-11-24T08:11:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.464+0000 7f08357fa640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f081c06b080 0x7f081c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.464+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f082c088e70 con 0x7f083806de10 2023-11-24T08:11:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.464+0000 7f0837fff640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f081c06b080 0x7f081c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:18.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.465+0000 7f0837fff640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f081c06b080 0x7f081c06d540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0824038000 tx=0x7f082403b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:18.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.469+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f082c053b60 con 0x7f083806de10 2023-11-24T08:11:18.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:18.700+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f083815ec90 con 0x7f083806de10 2023-11-24T08:11:19.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.322+0000 7f08357fa640 1 -- 172.21.15.106:0/1791871347 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f083815ec90 con 0x7f083806de10 2023-11-24T08:11:19.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f081c06b080 msgr2=0x7f081c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:19.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f081c06b080 0x7f081c06d540 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0824038000 tx=0x7f082403b040 comp rx=0 tx=0).stop 2023-11-24T08:11:19.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 msgr2=0x7f083814ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:19.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083814ccc0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f082c00c4a0 tx=0x7f082c00c960 comp rx=0 tx=0).stop 2023-11-24T08:11:19.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 shutdown_connections 2023-11-24T08:11:19.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f081c06b080 0x7f081c06d540 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08380c49f0 0x7f08380cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0838072160 0x7f08380cb290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 --2- 172.21.15.106:0/1791871347 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f083806de10 0x7f083814ccc0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 >> 172.21.15.106:0/1791871347 conn(0x7f08380b0c50 msgr2=0x7f083815e440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:19.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.326+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 shutdown_connections 2023-11-24T08:11:19.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.327+0000 7f083ee0e640 1 -- 172.21.15.106:0/1791871347 wait complete. 2023-11-24T08:11:19.351 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:11:19.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.557+0000 7ff98782d640 1 Processor -- start 2023-11-24T08:11:19.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.557+0000 7ff98782d640 1 -- start start 2023-11-24T08:11:19.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff98782d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9800b35f0 0x7ff980140950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:19.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff98782d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff98013e350 0x7ff98013f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:19.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff98782d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98013e8c0 con 0x7ff980140e90 2023-11-24T08:11:19.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff98782d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9801413d0 con 0x7ff9800b35f0 2023-11-24T08:11:19.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff98782d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9801416b0 con 0x7ff98013e350 2023-11-24T08:11:19.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff9855c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9800b35f0 0x7ff980140950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:19.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff984dc8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff980140e90 0x7ff98013dc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52246/0 (socket says 172.21.15.106:52246) 2023-11-24T08:11:19.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.559+0000 7ff984dc8640 1 -- 172.21.15.106:0/4273577628 learned_addr learned my addr 172.21.15.106:0/4273577628 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:19.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff985dca640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff98013e350 0x7ff98013f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:19.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff9855c9640 1 -- 172.21.15.106:0/4273577628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff98013e350 msgr2=0x7ff98013f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:19.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff9855c9640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff98013e350 0x7ff98013f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff9855c9640 1 -- 172.21.15.106:0/4273577628 >> v1:172.21.15.106:6789/0 conn(0x7ff980140e90 legacy=0x7ff98013dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:19.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff9855c9640 1 -- 172.21.15.106:0/4273577628 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98013eb10 con 0x7ff9800b35f0 2023-11-24T08:11:19.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff985dca640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff98013e350 0x7ff98013f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:19.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.560+0000 7ff97ffff640 1 -- 172.21.15.106:0/4273577628 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4254069822 0 0) 0x7ff98013e8c0 con 0x7ff980140e90 2023-11-24T08:11:19.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.561+0000 7ff9855c9640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff980140950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff970009d90 tx=0x7ff97002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112384b8212ed86b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:19.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.561+0000 7ff97ffff640 1 -- 172.21.15.106:0/4273577628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97002cbd0 con 0x7ff9800b35f0 2023-11-24T08:11:19.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.561+0000 7ff97ffff640 1 -- 172.21.15.106:0/4273577628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff97002cd30 con 0x7ff9800b35f0 2023-11-24T08:11:19.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.561+0000 7ff97ffff640 1 -- 172.21.15.106:0/4273577628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97002cea0 con 0x7ff9800b35f0 2023-11-24T08:11:19.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 -- 172.21.15.106:0/4273577628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 msgr2=0x7ff980140950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:19.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff980140950 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff970009d90 tx=0x7ff97002be20 comp rx=0 tx=0).stop 2023-11-24T08:11:19.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 -- 172.21.15.106:0/4273577628 shutdown_connections 2023-11-24T08:11:19.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff98013e350 0x7ff98013f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 --2- 172.21.15.106:0/4273577628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff980140950 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 -- 172.21.15.106:0/4273577628 >> 172.21.15.106:0/4273577628 conn(0x7ff980061340 msgr2=0x7ff980061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:19.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 -- 172.21.15.106:0/4273577628 shutdown_connections 2023-11-24T08:11:19.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.562+0000 7ff98782d640 1 -- 172.21.15.106:0/4273577628 wait complete. 2023-11-24T08:11:19.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.564+0000 7ff98782d640 1 Processor -- start 2023-11-24T08:11:19.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.564+0000 7ff98782d640 1 -- start start 2023-11-24T08:11:19.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff98782d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff9800b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:19.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff98782d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98013e350 0x7ff980151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:19.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff98782d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff980140e90 0x7ff9800d0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:19.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff98782d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800b4d30 con 0x7ff9800b35f0 2023-11-24T08:11:19.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff98782d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff980151940 con 0x7ff98013e350 2023-11-24T08:11:19.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff98782d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9800b4120 con 0x7ff980140e90 2023-11-24T08:11:19.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff9800b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:19.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff9800b51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34530/0 (socket says 172.21.15.106:34530) 2023-11-24T08:11:19.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 -- 172.21.15.106:0/2617282879 learned_addr learned my addr 172.21.15.106:0/2617282879 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:19.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 -- 172.21.15.106:0/2617282879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff980140e90 msgr2=0x7ff9800d0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:19.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff980140e90 0x7ff9800d0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 -- 172.21.15.106:0/2617282879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98013e350 msgr2=0x7ff980151400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.565+0000 7ff9855c9640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98013e350 0x7ff980151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff9855c9640 1 -- 172.21.15.106:0/2617282879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff970009a70 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff9855c9640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff9800b51c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff970037040 tx=0x7ff970004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97002cea0 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff970040650 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9800b4400 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff970040a00 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.566+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9800b49c0 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.567+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff980067470 con 0x7ff9800b35f0 2023-11-24T08:11:19.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.568+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff970040b60 con 0x7ff9800b35f0 2023-11-24T08:11:19.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.570+0000 7ff97dffb640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff96c06b080 0x7ff96c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:19.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.570+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5823+0+0 (secure 0 0 0) 0x7ff9700ac940 con 0x7ff9800b35f0 2023-11-24T08:11:19.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.570+0000 7ff984dc8640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff96c06b080 0x7ff96c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:19.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.570+0000 7ff984dc8640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff96c06b080 0x7ff96c06d540 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff974007fd0 tx=0x7ff974007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:19.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.574+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff970077580 con 0x7ff9800b35f0 2023-11-24T08:11:19.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:19.750+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff98015eca0 con 0x7ff9800b35f0 2023-11-24T08:11:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.467+0000 7ff97dffb640 1 -- 172.21.15.106:0/2617282879 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v52) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff97007aba0 con 0x7ff9800b35f0 2023-11-24T08:11:20.469 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:11:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff96c06b080 msgr2=0x7ff96c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:20.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff96c06b080 0x7ff96c06d540 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7ff974007fd0 tx=0x7ff974007960 comp rx=0 tx=0).stop 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 msgr2=0x7ff9800b51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff9800b51c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff970037040 tx=0x7ff970004a40 comp rx=0 tx=0).stop 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 shutdown_connections 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff96c06b080 0x7ff96c06d540 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff980140e90 0x7ff9800d0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff98013e350 0x7ff980151400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 --2- 172.21.15.106:0/2617282879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9800b35f0 0x7ff9800b51c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 >> 172.21.15.106:0/2617282879 conn(0x7ff980061340 msgr2=0x7ff98013ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 shutdown_connections 2023-11-24T08:11:20.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.471+0000 7ff98782d640 1 -- 172.21.15.106:0/2617282879 wait complete. 2023-11-24T08:11:20.495 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:11:20.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.703+0000 7f834237c640 1 Processor -- start 2023-11-24T08:11:20.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.703+0000 7f834237c640 1 -- start start 2023-11-24T08:11:20.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.704+0000 7f834237c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f833c15c7f0 0x7f833c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:20.706 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f834237c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0cece0 con 0x7f833c0ce6c0 2023-11-24T08:11:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f834237c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c06d040 con 0x7f833c0cf450 2023-11-24T08:11:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f834237c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0ceaf0 con 0x7f833c15c7f0 2023-11-24T08:11:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f833bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f833c15c7f0 0x7f833c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f8340919640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f833c0ce6c0 0x7f833c0714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52256/0 (socket says 172.21.15.106:52256) 2023-11-24T08:11:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f8340919640 1 -- 172.21.15.106:0/2859225335 learned_addr learned my addr 172.21.15.106:0/2859225335 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:20.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f833bfff640 1 -- 172.21.15.106:0/2859225335 >> v1:172.21.15.156:6789/0 conn(0x7f833c0cf450 legacy=0x7f833c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f833bfff640 1 -- 172.21.15.106:0/2859225335 >> v1:172.21.15.106:6789/0 conn(0x7f833c0ce6c0 legacy=0x7f833c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.705+0000 7f833bfff640 1 -- 172.21.15.106:0/2859225335 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c06d320 con 0x7f833c15c7f0 2023-11-24T08:11:20.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.706+0000 7f833bfff640 1 --2- 172.21.15.106:0/2859225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c15c7f0 0x7f833c06db20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097a0 tx=0x7f832c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba4a6503ebeeb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f833affd640 1 -- 172.21.15.106:0/2859225335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c031830 con 0x7f833c15c7f0 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f833affd640 1 -- 172.21.15.106:0/2859225335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c031990 con 0x7f833c15c7f0 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f833affd640 1 -- 172.21.15.106:0/2859225335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c031d20 con 0x7f833c15c7f0 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 -- 172.21.15.106:0/2859225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c15c7f0 msgr2=0x7f833c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 --2- 172.21.15.106:0/2859225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c15c7f0 0x7f833c06db20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097a0 tx=0x7f832c02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 -- 172.21.15.106:0/2859225335 shutdown_connections 2023-11-24T08:11:20.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 --2- 172.21.15.106:0/2859225335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c15c7f0 0x7f833c06db20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 -- 172.21.15.106:0/2859225335 >> 172.21.15.106:0/2859225335 conn(0x7f833c06a920 msgr2=0x7f833c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:20.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 -- 172.21.15.106:0/2859225335 shutdown_connections 2023-11-24T08:11:20.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.707+0000 7f834237c640 1 -- 172.21.15.106:0/2859225335 wait complete. 2023-11-24T08:11:20.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.709+0000 7f834237c640 1 Processor -- start 2023-11-24T08:11:20.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.709+0000 7f834237c640 1 -- start start 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f834237c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f833c0ce6c0 0x7f833c148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f834237c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 0x7f833c1f4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f834237c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833c15c7f0 0x7f833c154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f834237c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1f52e0 con 0x7f833c0cf450 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f834237c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1f55c0 con 0x7f833c0ce6c0 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f834237c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c155f60 con 0x7f833c15c7f0 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 0x7f833c1f4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f833b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 0x7f833c1f4890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34540/0 (socket says 172.21.15.106:34540) 2023-11-24T08:11:20.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.710+0000 7f833b7fe640 1 -- 172.21.15.106:0/2724152503 learned_addr learned my addr 172.21.15.106:0/2724152503 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f833b7fe640 1 -- 172.21.15.106:0/2724152503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833c15c7f0 msgr2=0x7f833c154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:20.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f833b7fe640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833c15c7f0 0x7f833c154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f833b7fe640 1 -- 172.21.15.106:0/2724152503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f833c0ce6c0 msgr2=0x7f833c148330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f833b7fe640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f833c0ce6c0 0x7f833c148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f833b7fe640 1 -- 172.21.15.106:0/2724152503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009400 con 0x7f833c0cf450 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f833b7fe640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 0x7f833c1f4890 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f833000c430 tx=0x7f833000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330011790 con 0x7f833c0cf450 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83300118f0 con 0x7f833c0cf450 2023-11-24T08:11:20.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8330011ce0 con 0x7f833c0cf450 2023-11-24T08:11:20.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.711+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c156240 con 0x7f833c0cf450 2023-11-24T08:11:20.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.712+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c1527c0 con 0x7f833c0cf450 2023-11-24T08:11:20.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.713+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c156480 con 0x7f833c0cf450 2023-11-24T08:11:20.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.714+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f833001c460 con 0x7f833c0cf450 2023-11-24T08:11:20.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.714+0000 7f8338ff9640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f832806b0b0 0x7f832806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:20.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.715+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f8330089680 con 0x7f833c0cf450 2023-11-24T08:11:20.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.715+0000 7f833bfff640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f832806b0b0 0x7f832806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:20.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.716+0000 7f833bfff640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f832806b0b0 0x7f832806d570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f832c036000 tx=0x7f832c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:20.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.719+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f833c156480 con 0x7f833c0cf450 2023-11-24T08:11:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:20.912+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f833c156480 con 0x7f833c0cf450 2023-11-24T08:11:21.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.471+0000 7f8338ff9640 1 -- 172.21.15.106:0/2724152503 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v53) v1 ==== 169+0+0 (secure 0 0 0) 0x7f833c156480 con 0x7f833c0cf450 2023-11-24T08:11:21.473 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2023-11-24T08:11:21.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.474+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f832806b0b0 msgr2=0x7f832806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.474+0000 7f834237c640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f832806b0b0 0x7f832806d570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f832c036000 tx=0x7f832c030000 comp rx=0 tx=0).stop 2023-11-24T08:11:21.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.474+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 msgr2=0x7f833c1f4890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.474+0000 7f834237c640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 0x7f833c1f4890 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f833000c430 tx=0x7f833000c8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 shutdown_connections 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f832806b0b0 0x7f832806d570 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f833c15c7f0 0x7f833c154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f833c0cf450 0x7f833c1f4890 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 --2- 172.21.15.106:0/2724152503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f833c0ce6c0 0x7f833c148330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 >> 172.21.15.106:0/2724152503 conn(0x7f833c06a920 msgr2=0x7f833c15eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 shutdown_connections 2023-11-24T08:11:21.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.477+0000 7f834237c640 1 -- 172.21.15.106:0/2724152503 wait complete. 2023-11-24T08:11:21.502 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:11:21.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.708+0000 7fa8d1533640 1 Processor -- start 2023-11-24T08:11:21.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.708+0000 7fa8d1533640 1 -- start start 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.710+0000 7fa8d1533640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8cc15b440 0x7fa8cc0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.710+0000 7fa8d1533640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc0cbf20 0x7fa8cc0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.710+0000 7fa8d1533640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8cc15eea0 con 0x7fa8cc0c57e0 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.710+0000 7fa8d1533640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8cc0cc3d0 con 0x7fa8cc15b440 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.710+0000 7fa8d1533640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15b820 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8cc15b440 0x7fa8cc0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8cc15b440 0x7fa8cc0cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34548/0 (socket says 172.21.15.106:34548) 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8cb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa8cc0c57e0 0x7fa8cc0c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52270/0 (socket says 172.21.15.106:52270) 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 -- 172.21.15.106:0/3909415742 learned_addr learned my addr 172.21.15.106:0/3909415742 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:21.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8c27fc640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc0cbf20 0x7fa8cc0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:21.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 -- 172.21.15.106:0/3909415742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc0cbf20 msgr2=0x7fa8cc0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:21.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc0cbf20 0x7fa8cc0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 -- 172.21.15.106:0/3909415742 >> v1:172.21.15.106:6789/0 conn(0x7fa8cc0c57e0 legacy=0x7fa8cc0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8caffd640 1 -- 172.21.15.106:0/3909415742 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8cc06d600 con 0x7fa8cc15b440 2023-11-24T08:11:21.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.711+0000 7fa8c27fc640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc0cbf20 0x7fa8cc0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:21.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.713+0000 7fa8caffd640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc15b440 0x7fa8cc0cb9e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009870 tx=0x7fa8bc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3aec47edf3f628 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:21.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.713+0000 7fa8ca7fc640 1 -- 172.21.15.106:0/3909415742 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc031640 con 0x7fa8cc15b440 2023-11-24T08:11:21.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.713+0000 7fa8ca7fc640 1 -- 172.21.15.106:0/3909415742 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc0317a0 con 0x7fa8cc15b440 2023-11-24T08:11:21.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.713+0000 7fa8ca7fc640 1 -- 172.21.15.106:0/3909415742 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc031b30 con 0x7fa8cc15b440 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 -- 172.21.15.106:0/3909415742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc15b440 msgr2=0x7fa8cc0cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc15b440 0x7fa8cc0cb9e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc009870 tx=0x7fa8bc02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 -- 172.21.15.106:0/3909415742 shutdown_connections 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc0cbf20 0x7fa8cc0c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 --2- 172.21.15.106:0/3909415742 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc15b440 0x7fa8cc0cb9e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 -- 172.21.15.106:0/3909415742 >> 172.21.15.106:0/3909415742 conn(0x7fa8cc0b0c50 msgr2=0x7fa8cc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 -- 172.21.15.106:0/3909415742 shutdown_connections 2023-11-24T08:11:21.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.714+0000 7fa8d1533640 1 -- 172.21.15.106:0/3909415742 wait complete. 2023-11-24T08:11:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.716+0000 7fa8d1533640 1 Processor -- start 2023-11-24T08:11:21.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.716+0000 7fa8d1533640 1 -- start start 2023-11-24T08:11:21.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8d1533640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8cc0c57e0 0x7fa8cc156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8d1533640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 0x7fa8cc156c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8d1533640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc15b440 0x7fa8cc0ce500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8d1533640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc15b960 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8d1533640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc156580 con 0x7fa8cc0c57e0 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8d1533640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8cc0cd0b0 con 0x7fa8cc15b440 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 0x7fa8cc156c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 0x7fa8cc156c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34562/0 (socket says 172.21.15.106:34562) 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 -- 172.21.15.106:0/2655814290 learned_addr learned my addr 172.21.15.106:0/2655814290 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 -- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc15b440 msgr2=0x7fa8cc0ce500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc15b440 0x7fa8cc0ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 -- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8cc0c57e0 msgr2=0x7fa8cc156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8caffd640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8cc0c57e0 0x7fa8cc156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8cc0c57e0 0x7fa8cc156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.717+0000 7fa8c27fc640 1 -- 172.21.15.106:0/2655814290 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8bc0094d0 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.718+0000 7fa8c27fc640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 0x7fa8cc156c60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c820 tx=0x7fa8b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.718+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b4007a30 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.718+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b4007b90 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.718+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b400ce70 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.718+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8cc0cd390 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.718+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8cc0cd950 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.719+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8cc15baa0 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.720+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa8b40113f0 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.721+0000 7fa8c3fff640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8ac06b080 0x7fa8ac06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.721+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa8b4089960 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.721+0000 7fa8caffd640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8ac06b080 0x7fa8ac06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:21.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.722+0000 7fa8caffd640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8ac06b080 0x7fa8ac06d540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc036000 tx=0x7fa8bc03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:21.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.726+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8cc15baa0 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.918+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa8cc15baa0 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.923+0000 7fa8c3fff640 1 -- 172.21.15.106:0/2655814290 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa8cc15baa0 con 0x7fa8cc0cbf20 2023-11-24T08:11:21.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8ac06b080 msgr2=0x7fa8ac06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8ac06b080 0x7fa8ac06d540 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc036000 tx=0x7fa8bc03c040 comp rx=0 tx=0).stop 2023-11-24T08:11:21.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 msgr2=0x7fa8cc156c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:21.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 0x7fa8cc156c60 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fa8b400c820 tx=0x7fa8b400cc20 comp rx=0 tx=0).stop 2023-11-24T08:11:21.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 shutdown_connections 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8ac06b080 0x7fa8ac06d540 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8cc15b440 0x7fa8cc0ce500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8cc0cbf20 0x7fa8cc156c60 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 --2- 172.21.15.106:0/2655814290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8cc0c57e0 0x7fa8cc156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.926+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 >> 172.21.15.106:0/2655814290 conn(0x7fa8cc0b0c50 msgr2=0x7fa8cc06cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.927+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 shutdown_connections 2023-11-24T08:11:21.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:21.927+0000 7fa8d1533640 1 -- 172.21.15.106:0/2655814290 wait complete. 2023-11-24T08:11:21.947 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata' 2023-11-24T08:11:22.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.156+0000 7fc30889b640 1 Processor -- start 2023-11-24T08:11:22.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.156+0000 7fc30889b640 1 -- start start 2023-11-24T08:11:22.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30889b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 0x7fc304070b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30889b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc3040cfe50 0x7fc3040d1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30889b640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3040d1b10 con 0x7fc3040d16c0 2023-11-24T08:11:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30889b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3040b4870 con 0x7fc3040cfe50 2023-11-24T08:11:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30889b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc304071070 con 0x7fc304144e00 2023-11-24T08:11:22.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30259c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 0x7fc304070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30259c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 0x7fc304070b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:46606/0 (socket says 172.21.15.106:46606) 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc30259c640 1 -- 172.21.15.106:0/2717127885 learned_addr learned my addr 172.21.15.106:0/2717127885 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.158+0000 7fc301d9b640 1 --2- 172.21.15.106:0/2717127885 >> v2:172.21.15.156:3300/0 conn(0x7fc3040cfe50 0x7fc3040d1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.159+0000 7fc30259c640 1 -- 172.21.15.106:0/2717127885 >> v2:172.21.15.156:3300/0 conn(0x7fc3040cfe50 msgr2=0x7fc3040d1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.159+0000 7fc30259c640 1 --2- 172.21.15.106:0/2717127885 >> v2:172.21.15.156:3300/0 conn(0x7fc3040cfe50 0x7fc3040d1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.159+0000 7fc30259c640 1 -- 172.21.15.106:0/2717127885 >> v1:172.21.15.156:6789/0 conn(0x7fc3040d16c0 legacy=0x7fc30415f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.159+0000 7fc30259c640 1 -- 172.21.15.106:0/2717127885 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3041451e0 con 0x7fc304144e00 2023-11-24T08:11:22.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.160+0000 7fc30259c640 1 --2- 172.21.15.106:0/2717127885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 0x7fc304070b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009e50 tx=0x7fc2f402bed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de930ddf969f178c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:22.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.161+0000 7fc30159a640 1 -- 172.21.15.106:0/2717127885 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f402ca70 con 0x7fc304144e00 2023-11-24T08:11:22.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.161+0000 7fc30159a640 1 -- 172.21.15.106:0/2717127885 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f402cbd0 con 0x7fc304144e00 2023-11-24T08:11:22.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.161+0000 7fc30159a640 1 -- 172.21.15.106:0/2717127885 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f40343f0 con 0x7fc304144e00 2023-11-24T08:11:22.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 -- 172.21.15.106:0/2717127885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 msgr2=0x7fc304070b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 --2- 172.21.15.106:0/2717127885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 0x7fc304070b30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4009e50 tx=0x7fc2f402bed0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 -- 172.21.15.106:0/2717127885 shutdown_connections 2023-11-24T08:11:22.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 --2- 172.21.15.106:0/2717127885 >> v2:172.21.15.156:3300/0 conn(0x7fc3040cfe50 0x7fc3040d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 --2- 172.21.15.106:0/2717127885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc304144e00 0x7fc304070b30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 -- 172.21.15.106:0/2717127885 >> 172.21.15.106:0/2717127885 conn(0x7fc30406e8a0 msgr2=0x7fc30406ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:22.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 -- 172.21.15.106:0/2717127885 shutdown_connections 2023-11-24T08:11:22.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.162+0000 7fc30889b640 1 -- 172.21.15.106:0/2717127885 wait complete. 2023-11-24T08:11:22.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.164+0000 7fc30889b640 1 Processor -- start 2023-11-24T08:11:22.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.164+0000 7fc30889b640 1 -- start start 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30889b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 0x7fc30413aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30889b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3040d16c0 0x7fc304153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30889b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304144e00 0x7fc3040b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30889b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3041539a0 con 0x7fc3040cfe50 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30889b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30406bd50 con 0x7fc304144e00 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30889b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc304153250 con 0x7fc3040d16c0 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 0x7fc30413aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 0x7fc30413aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34564/0 (socket says 172.21.15.106:34564) 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 -- 172.21.15.106:0/3844890821 learned_addr learned my addr 172.21.15.106:0/3844890821 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc301d9b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3040d16c0 0x7fc304153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc302d9d640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304144e00 0x7fc3040b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 -- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3040d16c0 msgr2=0x7fc304153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3040d16c0 0x7fc304153460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 -- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304144e00 msgr2=0x7fc3040b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304144e00 0x7fc3040b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.165+0000 7fc30259c640 1 -- 172.21.15.106:0/3844890821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f4009b30 con 0x7fc3040cfe50 2023-11-24T08:11:22.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.166+0000 7fc30259c640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 0x7fc30413aea0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000f80 tx=0x7fc2f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:22.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.166+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4040670 con 0x7fc3040cfe50 2023-11-24T08:11:22.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.166+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f40407d0 con 0x7fc3040cfe50 2023-11-24T08:11:22.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.166+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f4040b80 con 0x7fc3040cfe50 2023-11-24T08:11:22.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.166+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc30406b740 con 0x7fc3040cfe50 2023-11-24T08:11:22.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.166+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc30406b9e0 con 0x7fc3040cfe50 2023-11-24T08:11:22.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.167+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc30406bb70 con 0x7fc3040cfe50 2023-11-24T08:11:22.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.168+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fc2f4040ce0 con 0x7fc3040cfe50 2023-11-24T08:11:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.169+0000 7fc2faffd640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2e406b080 0x7fc2e406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.169+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc2f40ac820 con 0x7fc3040cfe50 2023-11-24T08:11:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.169+0000 7fc301d9b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2e406b080 0x7fc2e406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.170+0000 7fc301d9b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2e406b080 0x7fc2e406d540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3041f7670 tx=0x7fc2ec008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:22.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.173+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2f4077900 con 0x7fc3040cfe50 2023-11-24T08:11:22.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.450+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]} v 0) v1 -- 0x7fc304067b80 con 0x7fc3040cfe50 2023-11-24T08:11:22.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.487+0000 7fc2faffd640 1 -- 172.21.15.106:0/3844890821 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_clean_stale_subvolume_snapshot_metadata"]}]=0 v160) v1 ==== 172+0+0 (secure 0 0 0) 0x7fc2f4009510 con 0x7fc3040cfe50 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2e406b080 msgr2=0x7fc2e406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2e406b080 0x7fc2e406d540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc3041f7670 tx=0x7fc2ec008040 comp rx=0 tx=0).stop 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 msgr2=0x7fc30413aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 0x7fc30413aea0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fc2f4000f80 tx=0x7fc2f4004a40 comp rx=0 tx=0).stop 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 shutdown_connections 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc304144e00 0x7fc3040b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2e406b080 0x7fc2e406d540 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3040d16c0 0x7fc304153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 --2- 172.21.15.106:0/3844890821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3040cfe50 0x7fc30413aea0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 >> 172.21.15.106:0/3844890821 conn(0x7fc30406e8a0 msgr2=0x7fc304070000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:22.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 shutdown_connections 2023-11-24T08:11:22.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.490+0000 7fc30889b640 1 -- 172.21.15.106:0/3844890821 wait complete. 2023-11-24T08:11:22.512 INFO:tasks.cephfs_test_runner:Validate cleaning of stale subvolume snapshot metadata. ... ok 2023-11-24T08:11:22.512 INFO:tasks.cephfs_test_runner:Starting test: test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:11:22.512 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:22.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.725+0000 7fcd17fff640 1 Processor -- start 2023-11-24T08:11:22.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.725+0000 7fcd17fff640 1 -- start start 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.727+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd18144b30 0x7fcd1815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.727+0000 7fcd17fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd1815c410 0x7fcd180b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.727+0000 7fcd17fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd1815bf80 con 0x7fcd180b1de0 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.727+0000 7fcd17fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180b1560 con 0x7fcd1815c410 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.727+0000 7fcd17fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180b0fb0 con 0x7fcd18144b30 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd0e7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd1815c410 0x7fcd180b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd177fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcd180b1de0 0x7fcd1806c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52278/0 (socket says 172.21.15.106:52278) 2023-11-24T08:11:22.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd177fe640 1 -- 172.21.15.106:0/1099180723 learned_addr learned my addr 172.21.15.106:0/1099180723 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:22.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd16ffd640 1 --2- 172.21.15.106:0/1099180723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd18144b30 0x7fcd1815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/1099180723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd18144b30 msgr2=0x7fcd1815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:22.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd0e7fc640 1 --2- 172.21.15.106:0/1099180723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd18144b30 0x7fcd1815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd167fc640 1 -- 172.21.15.106:0/1099180723 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3450382806 0 0) 0x7fcd1815bf80 con 0x7fcd180b1de0 2023-11-24T08:11:22.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/1099180723 >> v1:172.21.15.106:6789/0 conn(0x7fcd180b1de0 legacy=0x7fcd1806c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.728+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/1099180723 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180b1280 con 0x7fcd1815c410 2023-11-24T08:11:22.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.730+0000 7fcd0e7fc640 1 --2- 172.21.15.106:0/1099180723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd1815c410 0x7fcd180b0a70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcd000097a0 tx=0x7fcd0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f898bb8b04222850 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:22.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.730+0000 7fcd167fc640 1 -- 172.21.15.106:0/1099180723 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00031830 con 0x7fcd1815c410 2023-11-24T08:11:22.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.730+0000 7fcd167fc640 1 -- 172.21.15.106:0/1099180723 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd00031990 con 0x7fcd1815c410 2023-11-24T08:11:22.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.730+0000 7fcd167fc640 1 -- 172.21.15.106:0/1099180723 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00031d20 con 0x7fcd1815c410 2023-11-24T08:11:22.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.730+0000 7fcd17fff640 1 -- 172.21.15.106:0/1099180723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd1815c410 msgr2=0x7fcd180b0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.730+0000 7fcd17fff640 1 --2- 172.21.15.106:0/1099180723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd1815c410 0x7fcd180b0a70 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcd000097a0 tx=0x7fcd0002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.731+0000 7fcd17fff640 1 -- 172.21.15.106:0/1099180723 shutdown_connections 2023-11-24T08:11:22.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.731+0000 7fcd17fff640 1 --2- 172.21.15.106:0/1099180723 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd1815c410 0x7fcd180b0a70 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.731+0000 7fcd17fff640 1 --2- 172.21.15.106:0/1099180723 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd18144b30 0x7fcd1815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.731+0000 7fcd17fff640 1 -- 172.21.15.106:0/1099180723 >> 172.21.15.106:0/1099180723 conn(0x7fcd180cb830 msgr2=0x7fcd180cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:22.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.731+0000 7fcd17fff640 1 -- 172.21.15.106:0/1099180723 shutdown_connections 2023-11-24T08:11:22.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.731+0000 7fcd17fff640 1 -- 172.21.15.106:0/1099180723 wait complete. 2023-11-24T08:11:22.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.732+0000 7fcd17fff640 1 Processor -- start 2023-11-24T08:11:22.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.733+0000 7fcd17fff640 1 -- start start 2023-11-24T08:11:22.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd180b1de0 0x7fcd1815a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 0x7fcd180c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1815c410 0x7fcd1813e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd17fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1813f5a0 con 0x7fcd18144b30 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd17fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180c8860 con 0x7fcd1815c410 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd17fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1813ec70 con 0x7fcd180b1de0 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd0e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 0x7fcd180c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd0e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 0x7fcd180c8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34592/0 (socket says 172.21.15.106:34592) 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/2284379964 learned_addr learned my addr 172.21.15.106:0/2284379964 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:22.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd16ffd640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd180b1de0 0x7fcd1815a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd180b1de0 msgr2=0x7fcd1815a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd0e7fc640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd180b1de0 0x7fcd1815a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1815c410 msgr2=0x7fcd1813e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.734+0000 7fcd177fe640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1815c410 0x7fcd1813e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd0e7fc640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1815c410 0x7fcd1813e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd0e7fc640 1 -- 172.21.15.106:0/2284379964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00009400 con 0x7fcd18144b30 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd16ffd640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd180b1de0 0x7fcd1815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd177fe640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1815c410 0x7fcd1813e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:22.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd0e7fc640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 0x7fcd180c8160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcd00000c00 tx=0x7fcd000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:22.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00004230 con 0x7fcd18144b30 2023-11-24T08:11:22.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd000043b0 con 0x7fcd18144b30 2023-11-24T08:11:22.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1813ef50 con 0x7fcd18144b30 2023-11-24T08:11:22.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd00004520 con 0x7fcd18144b30 2023-11-24T08:11:22.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.735+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1813fa30 con 0x7fcd18144b30 2023-11-24T08:11:22.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.736+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1804e840 con 0x7fcd18144b30 2023-11-24T08:11:22.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.737+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fcd0003a870 con 0x7fcd18144b30 2023-11-24T08:11:22.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.738+0000 7fcd14ff9640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fccf806b080 0x7fccf806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:22.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.738+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcd000abaa0 con 0x7fcd18144b30 2023-11-24T08:11:22.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.739+0000 7fcd16ffd640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fccf806b080 0x7fccf806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:22.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.739+0000 7fcd16ffd640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fccf806b080 0x7fccf806d540 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd1813e1b0 tx=0x7fcd0800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:22.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.743+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd000769d0 con 0x7fcd18144b30 2023-11-24T08:11:22.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.934+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd180b3610 con 0x7fcd18144b30 2023-11-24T08:11:22.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.935+0000 7fcd14ff9640 1 -- 172.21.15.106:0/2284379964 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+13036 (secure 0 0 0) 0x7fcd180b3610 con 0x7fcd18144b30 2023-11-24T08:11:22.937 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:22.937 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":53,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:21.467922+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/1634060253":"2023-11-25T08:11:18.231868+0000","172.21.15.106:6836/1634060253":"2023-11-25T08:11:18.231868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:22.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fccf806b080 msgr2=0x7fccf806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fccf806b080 0x7fccf806d540 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd1813e1b0 tx=0x7fcd0800b040 comp rx=0 tx=0).stop 2023-11-24T08:11:22.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 msgr2=0x7fcd180c8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:22.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 0x7fcd180c8160 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcd00000c00 tx=0x7fcd000046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 shutdown_connections 2023-11-24T08:11:22.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd1815c410 0x7fcd1813e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd18144b30 0x7fcd180c8160 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fccf806b080 0x7fccf806d540 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 --2- 172.21.15.106:0/2284379964 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd180b1de0 0x7fcd1815a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:22.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 >> 172.21.15.106:0/2284379964 conn(0x7fcd180cb830 msgr2=0x7fcd180ccc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:22.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 shutdown_connections 2023-11-24T08:11:22.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:22.938+0000 7fcd17fff640 1 -- 172.21.15.106:0/2284379964 wait complete. 2023-11-24T08:11:22.959 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:22.959 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:23.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.168+0000 7f7b5359e640 1 Processor -- start 2023-11-24T08:11:23.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.168+0000 7f7b5359e640 1 -- start start 2023-11-24T08:11:23.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.170+0000 7f7b5359e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7b5415d7d0 0x7f7b540cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.170+0000 7f7b5359e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540d15e0 con 0x7f7b5415d3f0 2023-11-24T08:11:23.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.170+0000 7f7b5359e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b540d0280 con 0x7f7b540d1f40 2023-11-24T08:11:23.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.170+0000 7f7b5359e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b540b0eb0 con 0x7f7b5415d7d0 2023-11-24T08:11:23.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.170+0000 7f7b5259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7b5415d3f0 0x7f7b540d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52280/0 (socket says 172.21.15.106:52280) 2023-11-24T08:11:23.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.170+0000 7f7b5259c640 1 -- 172.21.15.106:0/559946372 learned_addr learned my addr 172.21.15.106:0/559946372 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:23.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/559946372 >> v2:172.21.15.156:3300/0 conn(0x7f7b5415d7d0 0x7f7b540cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206697750 0 0) 0x7f7b540d15e0 con 0x7f7b5415d3f0 2023-11-24T08:11:23.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b40003520 con 0x7f7b5415d3f0 2023-11-24T08:11:23.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1880359700 0 0) 0x7f7b40003520 con 0x7f7b5415d3f0 2023-11-24T08:11:23.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 >> v2:172.21.15.156:3300/0 conn(0x7f7b5415d7d0 msgr2=0x7f7b540cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:23.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 --2- 172.21.15.106:0/559946372 >> v2:172.21.15.156:3300/0 conn(0x7f7b5415d7d0 0x7f7b540cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 >> v1:172.21.15.156:6789/0 conn(0x7f7b540d1f40 legacy=0x7f7b540ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540d1300 con 0x7f7b5415d3f0 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f7b44001670 con 0x7f7b5415d3f0 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 >> v1:172.21.15.106:6789/0 conn(0x7f7b5415d3f0 legacy=0x7f7b540d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b40003970 0x7f7b40003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b400047c0 0x7f7b40004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.171+0000 7f7b51d9b640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b40005180 0x7f7b4000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540b0eb0 con 0x7f7b400047c0 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4000c020 con 0x7f7b40003970 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b4000c1e0 con 0x7f7b40005180 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b400047c0 0x7f7b40004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b52d9d640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b40005180 0x7f7b4000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b5259c640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b40003970 0x7f7b40003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b4bfff640 1 -- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b40005180 msgr2=0x7f7b4000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b40005180 0x7f7b4000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b4bfff640 1 -- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b40003970 msgr2=0x7f7b40003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b40003970 0x7f7b40003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b4bfff640 1 -- 172.21.15.106:0/559946372 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b40003520 con 0x7f7b400047c0 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b52d9d640 1 -- 172.21.15.106:0/559946372 reap_dead start 2023-11-24T08:11:23.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.172+0000 7f7b5259c640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b40003970 0x7f7b40003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:23.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.173+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b400047c0 0x7f7b40004c40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009a90 tx=0x7f7b400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1240714041c63d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:23.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c02c6b0 con 0x7f7b400047c0 2023-11-24T08:11:23.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c02c810 con 0x7f7b400047c0 2023-11-24T08:11:23.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b51d9b640 1 -- 172.21.15.106:0/559946372 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c02cba0 con 0x7f7b400047c0 2023-11-24T08:11:23.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b5359e640 1 -- 172.21.15.106:0/559946372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b400047c0 msgr2=0x7f7b40004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b5359e640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b400047c0 0x7f7b40004c40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c009a90 tx=0x7f7b400057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b5359e640 1 -- 172.21.15.106:0/559946372 shutdown_connections 2023-11-24T08:11:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b5359e640 1 --2- 172.21.15.106:0/559946372 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b400047c0 0x7f7b40004c40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b5359e640 1 -- 172.21.15.106:0/559946372 >> 172.21.15.106:0/559946372 conn(0x7f7b5409c5b0 msgr2=0x7f7b540d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.174+0000 7f7b5359e640 1 -- 172.21.15.106:0/559946372 shutdown_connections 2023-11-24T08:11:23.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.175+0000 7f7b5359e640 1 -- 172.21.15.106:0/559946372 wait complete. 2023-11-24T08:11:23.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.176+0000 7f7b5359e640 1 Processor -- start 2023-11-24T08:11:23.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.176+0000 7f7b5359e640 1 -- start start 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b5359e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400047c0 0x7f7b54158640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b5359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 0x7f7b540b4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b5359e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b540c8720 0x7f7b540b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b5359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c8cf0 con 0x7f7b540c82a0 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b5359e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c8fd0 con 0x7f7b400047c0 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b5359e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b540c92b0 con 0x7f7b540c8720 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.177+0000 7f7b4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 0x7f7b540b4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 0x7f7b540b4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34604/0 (socket says 172.21.15.106:34604) 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 -- 172.21.15.106:0/2777486635 learned_addr learned my addr 172.21.15.106:0/2777486635 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b5259c640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400047c0 0x7f7b54158640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b52d9d640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b540c8720 0x7f7b540b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 -- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b540c8720 msgr2=0x7f7b540b5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b540c8720 0x7f7b540b5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 -- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400047c0 msgr2=0x7f7b54158640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400047c0 0x7f7b54158640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 -- 172.21.15.106:0/2777486635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b540c9590 con 0x7f7b540c82a0 2023-11-24T08:11:23.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.178+0000 7f7b4bfff640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 0x7f7b540b4b40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c02c250 tx=0x7f7b3c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.179+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c003810 con 0x7f7b540c82a0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.179+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3c003970 con 0x7f7b540c82a0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.179+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3c003ce0 con 0x7f7b540c82a0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.179+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b540c9870 con 0x7f7b540c82a0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.179+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b540cae60 con 0x7f7b540c82a0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.180+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b540c9a30 con 0x7f7b540c82a0 2023-11-24T08:11:23.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.181+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7b3c040450 con 0x7f7b540c82a0 2023-11-24T08:11:23.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.182+0000 7f7b4affd640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b3406b0b0 0x7f7b3406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.182+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7b3c0ab6c0 con 0x7f7b540c82a0 2023-11-24T08:11:23.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.182+0000 7f7b5259c640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b3406b0b0 0x7f7b3406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.183+0000 7f7b5259c640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b3406b0b0 0x7f7b3406d570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b44001eb0 tx=0x7f7b44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:23.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.186+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7b3c076670 con 0x7f7b540c82a0 2023-11-24T08:11:23.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.374+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b5404e840 con 0x7f7b540c82a0 2023-11-24T08:11:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.375+0000 7f7b4affd640 1 -- 172.21.15.106:0/2777486635 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v53) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b3c0044e0 con 0x7f7b540c82a0 2023-11-24T08:11:23.377 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b3406b0b0 msgr2=0x7f7b3406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b3406b0b0 0x7f7b3406d570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7b44001eb0 tx=0x7f7b44009040 comp rx=0 tx=0).stop 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 msgr2=0x7f7b540b4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 0x7f7b540b4b40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7b3c02c250 tx=0x7f7b3c004730 comp rx=0 tx=0).stop 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 shutdown_connections 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7b3406b0b0 0x7f7b3406d570 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7b540c8720 0x7f7b540b5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7b540c82a0 0x7f7b540b4b40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 --2- 172.21.15.106:0/2777486635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7b400047c0 0x7f7b54158640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 >> 172.21.15.106:0/2777486635 conn(0x7f7b5409c5b0 msgr2=0x7f7b54160360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 shutdown_connections 2023-11-24T08:11:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.378+0000 7f7b5359e640 1 -- 172.21.15.106:0/2777486635 wait complete. 2023-11-24T08:11:23.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T08:11:23.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.598+0000 7fb33839e640 1 Processor -- start 2023-11-24T08:11:23.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.599+0000 7fb33839e640 1 -- start start 2023-11-24T08:11:23.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33839e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3300ce640 0x7fb3300abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33839e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3300ce270 con 0x7fb3300cc470 2023-11-24T08:11:23.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33839e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33006d0a0 con 0x7fb3300cfb60 2023-11-24T08:11:23.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33839e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33006c560 con 0x7fb3300ce640 2023-11-24T08:11:23.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33613a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb3300cc470 0x7fb3300cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:52284/0 (socket says 172.21.15.106:52284) 2023-11-24T08:11:23.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33613a640 1 -- 172.21.15.106:0/1326838945 learned_addr learned my addr 172.21.15.106:0/1326838945 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:23.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.600+0000 7fb33693b640 1 --2- 172.21.15.106:0/1326838945 >> v2:172.21.15.106:3300/0 conn(0x7fb3300ce640 0x7fb3300abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.601+0000 7fb335138640 1 -- 172.21.15.106:0/1326838945 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078684695 0 0) 0x7fb3300ce270 con 0x7fb3300cc470 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.601+0000 7fb335138640 1 -- 172.21.15.106:0/1326838945 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb320003520 con 0x7fb3300cc470 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.601+0000 7fb33693b640 1 -- 172.21.15.106:0/1326838945 >> v1:172.21.15.156:6789/0 conn(0x7fb3300cfb60 legacy=0x7fb33006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.601+0000 7fb33693b640 1 -- 172.21.15.106:0/1326838945 >> v1:172.21.15.106:6789/0 conn(0x7fb3300cc470 legacy=0x7fb3300cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.601+0000 7fb33693b640 1 -- 172.21.15.106:0/1326838945 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33006c770 con 0x7fb3300ce640 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.602+0000 7fb33693b640 1 --2- 172.21.15.106:0/1326838945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300ce640 0x7fb3300abfc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb3280097a0 tx=0x7fb32802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eeab312e97bd40a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.602+0000 7fb335138640 1 -- 172.21.15.106:0/1326838945 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328031830 con 0x7fb3300ce640 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.602+0000 7fb335138640 1 -- 172.21.15.106:0/1326838945 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb328031990 con 0x7fb3300ce640 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.602+0000 7fb335138640 1 -- 172.21.15.106:0/1326838945 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb328031d00 con 0x7fb3300ce640 2023-11-24T08:11:23.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 -- 172.21.15.106:0/1326838945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300ce640 msgr2=0x7fb3300abfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 --2- 172.21.15.106:0/1326838945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300ce640 0x7fb3300abfc0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb3280097a0 tx=0x7fb32802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 -- 172.21.15.106:0/1326838945 shutdown_connections 2023-11-24T08:11:23.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 --2- 172.21.15.106:0/1326838945 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300ce640 0x7fb3300abfc0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 -- 172.21.15.106:0/1326838945 >> 172.21.15.106:0/1326838945 conn(0x7fb33006a920 msgr2=0x7fb33006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:23.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 -- 172.21.15.106:0/1326838945 shutdown_connections 2023-11-24T08:11:23.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.603+0000 7fb33839e640 1 -- 172.21.15.106:0/1326838945 wait complete. 2023-11-24T08:11:23.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.605+0000 7fb33839e640 1 Processor -- start 2023-11-24T08:11:23.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.605+0000 7fb33839e640 1 -- start start 2023-11-24T08:11:23.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33839e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3300cc470 0x7fb33015a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33839e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3300ce640 0x7fb33015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33839e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 0x7fb33015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33839e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015ec30 con 0x7fb3300cfb60 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33839e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015ef10 con 0x7fb3300ce640 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33839e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33015f1f0 con 0x7fb3300cc470 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33693b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 0x7fb33015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33693b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 0x7fb33015d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:34618/0 (socket says 172.21.15.106:34618) 2023-11-24T08:11:23.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.606+0000 7fb33693b640 1 -- 172.21.15.106:0/4115494300 learned_addr learned my addr 172.21.15.106:0/4115494300 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:23.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33613a640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3300cc470 0x7fb33015a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb335939640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3300ce640 0x7fb33015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33693b640 1 -- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3300cc470 msgr2=0x7fb33015a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33693b640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3300cc470 0x7fb33015a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33693b640 1 -- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3300ce640 msgr2=0x7fb33015fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33693b640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3300ce640 0x7fb33015fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33693b640 1 -- 172.21.15.106:0/4115494300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb328009400 con 0x7fb3300cfb60 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33613a640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3300cc470 0x7fb33015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb335939640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3300ce640 0x7fb33015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33693b640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 0x7fb33015d350 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb32802b910 tx=0x7fb3280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32803a630 con 0x7fb3300cfb60 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb32803a790 con 0x7fb3300cfb60 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33015f4d0 con 0x7fb3300cfb60 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.607+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb32803ab40 con 0x7fb3300cfb60 2023-11-24T08:11:23.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.608+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33015e140 con 0x7fb3300cfb60 2023-11-24T08:11:23.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.609+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33015f9f0 con 0x7fb3300cfb60 2023-11-24T08:11:23.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.610+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb32803aca0 con 0x7fb3300cfb60 2023-11-24T08:11:23.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.610+0000 7fb32effd640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31406b080 0x7fb31406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:23.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.611+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb3280ab510 con 0x7fb3300cfb60 2023-11-24T08:11:23.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.611+0000 7fb33613a640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31406b080 0x7fb31406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:23.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.611+0000 7fb33613a640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31406b080 0x7fb31406d540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb324005c00 tx=0x7fb324009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:23.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.615+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb33015f9f0 con 0x7fb3300cfb60 2023-11-24T08:11:23.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:23.858+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7fb33015f9f0 con 0x7fb3300cfb60 2023-11-24T08:11:24.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.492+0000 7fb32effd640 1 -- 172.21.15.106:0/4115494300 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v162) v1 ==== 169+0+0 (secure 0 0 0) 0x7fb33015f9f0 con 0x7fb3300cfb60 2023-11-24T08:11:24.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.496+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31406b080 msgr2=0x7fb31406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.496+0000 7fb33839e640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31406b080 0x7fb31406d540 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fb324005c00 tx=0x7fb324009040 comp rx=0 tx=0).stop 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.496+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 msgr2=0x7fb33015d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.496+0000 7fb33839e640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 0x7fb33015d350 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb32802b910 tx=0x7fb3280046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.496+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 shutdown_connections 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3300cfb60 0x7fb33015d350 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3300ce640 0x7fb33015fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb31406b080 0x7fb31406d540 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 --2- 172.21.15.106:0/4115494300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3300cc470 0x7fb33015a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 >> 172.21.15.106:0/4115494300 conn(0x7fb33006a920 msgr2=0x7fb3300aec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 shutdown_connections 2023-11-24T08:11:24.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.497+0000 7fb33839e640 1 -- 172.21.15.106:0/4115494300 wait complete. 2023-11-24T08:11:24.518 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:11:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.734+0000 7f823cd88640 1 Processor -- start 2023-11-24T08:11:24.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.734+0000 7f823cd88640 1 -- start start 2023-11-24T08:11:24.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.735+0000 7f823cd88640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f823806c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.735+0000 7f823cd88640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82380ccdf0 con 0x7f82380cede0 2023-11-24T08:11:24.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.735+0000 7f823cd88640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f823815f470 con 0x7f823815ef00 2023-11-24T08:11:24.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.735+0000 7f823cd88640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f823815f750 con 0x7f8238141b30 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8237fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f82380cede0 0x7f82380cd2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40876/0 (socket says 172.21.15.106:40876) 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8237fff640 1 -- 172.21.15.106:0/1287141088 learned_addr learned my addr 172.21.15.106:0/1287141088 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f82377fe640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f823806c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190866836 0 0) 0x7f82380ccdf0 con 0x7f82380cede0 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8224003520 con 0x7f82380cede0 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3323839674 0 0) 0x7f8224003520 con 0x7f82380cede0 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 msgr2=0x7f823806c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:24.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f823806c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 >> v1:172.21.15.156:6789/0 conn(0x7f823815ef00 legacy=0x7f8238071700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f823815fa30 con 0x7f82380cede0 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f82300019f0 con 0x7f82380cede0 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.736+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 >> v1:172.21.15.106:6789/0 conn(0x7f82380cede0 legacy=0x7f82380cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8236ffd640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82240039a0 0x7f8224003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8236ffd640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f82240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8236ffd640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8224004d00 0x7f8224005160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f823815f750 con 0x7f82240039a0 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822400bb90 con 0x7f8224004d00 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822400bd50 con 0x7f8238141b30 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f82377fe640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82240039a0 0x7f8224003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8237fff640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8224004d00 0x7f8224005160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f82377fe640 1 -- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 msgr2=0x7f82240047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f822ffff640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f82240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f82377fe640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f82240047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f82377fe640 1 -- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8224004d00 msgr2=0x7f8224005160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f82377fe640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8224004d00 0x7f8224005160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f82377fe640 1 -- 172.21.15.106:0/1287141088 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8224003520 con 0x7f82240039a0 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f8237fff640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8224004d00 0x7f8224005160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:24.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.737+0000 7f822ffff640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8238141b30 0x7f82240047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:24.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.738+0000 7f82377fe640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82240039a0 0x7f8224003dc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8228009ac0 tx=0x7f82240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76b6bcc5d3945f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:24.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822802c6b0 con 0x7f82240039a0 2023-11-24T08:11:24.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f822802c810 con 0x7f82240039a0 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f8236ffd640 1 -- 172.21.15.106:0/1287141088 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f822802cba0 con 0x7f82240039a0 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f823cd88640 1 -- 172.21.15.106:0/1287141088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82240039a0 msgr2=0x7f8224003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f823cd88640 1 --2- 172.21.15.106:0/1287141088 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82240039a0 0x7f8224003dc0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f8228009ac0 tx=0x7f82240057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f8237fff640 1 -- 172.21.15.106:0/1287141088 reap_dead start 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f823cd88640 1 -- 172.21.15.106:0/1287141088 shutdown_connections 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f823cd88640 1 -- 172.21.15.106:0/1287141088 >> 172.21.15.106:0/1287141088 conn(0x7f823806a500 msgr2=0x7f823806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f823cd88640 1 -- 172.21.15.106:0/1287141088 shutdown_connections 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.739+0000 7f823cd88640 1 -- 172.21.15.106:0/1287141088 wait complete. 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.741+0000 7f823cd88640 1 Processor -- start 2023-11-24T08:11:24.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.741+0000 7f823cd88640 1 -- start start 2023-11-24T08:11:24.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f823cd88640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82240039a0 0x7f8238158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f823cd88640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8238143a30 0x7f8238142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f823cd88640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 0x7f82380b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f823cd88640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8238144120 con 0x7f82381f1de0 2023-11-24T08:11:24.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f823cd88640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8238144400 con 0x7f8238143a30 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f823cd88640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82381446e0 con 0x7f82240039a0 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f8237fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 0x7f82380b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f8237fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 0x7f82380b0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43378/0 (socket says 172.21.15.106:43378) 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.742+0000 7f8237fff640 1 -- 172.21.15.106:0/2093954800 learned_addr learned my addr 172.21.15.106:0/2093954800 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f822ffff640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8238143a30 0x7f8238142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f82377fe640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82240039a0 0x7f8238158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8237fff640 1 -- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82240039a0 msgr2=0x7f8238158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8237fff640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82240039a0 0x7f8238158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8237fff640 1 -- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8238143a30 msgr2=0x7f8238142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8237fff640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8238143a30 0x7f8238142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8237fff640 1 -- 172.21.15.106:0/2093954800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82381449c0 con 0x7f82381f1de0 2023-11-24T08:11:24.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f822ffff640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8238143a30 0x7f8238142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8237fff640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 0x7f82380b0680 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f82380b1c80 tx=0x7f823000b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000b8d0 con 0x7f82381f1de0 2023-11-24T08:11:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823000ba50 con 0x7f82381f1de0 2023-11-24T08:11:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823000be00 con 0x7f82381f1de0 2023-11-24T08:11:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.743+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8238144ca0 con 0x7f82381f1de0 2023-11-24T08:11:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.744+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82380b0bc0 con 0x7f82381f1de0 2023-11-24T08:11:24.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.745+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8238144e90 con 0x7f82381f1de0 2023-11-24T08:11:24.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.746+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8230007110 con 0x7f82381f1de0 2023-11-24T08:11:24.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.746+0000 7f8234ff9640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f821806b080 0x7f821806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:24.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.746+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8230089bc0 con 0x7f82381f1de0 2023-11-24T08:11:24.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.747+0000 7f82377fe640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f821806b080 0x7f821806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:24.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.747+0000 7f82377fe640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f821806b080 0x7f821806d540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8228036000 tx=0x7f8228030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.751+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8230054af0 con 0x7f82381f1de0 2023-11-24T08:11:24.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.939+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8238143e70 con 0x7f82381f1de0 2023-11-24T08:11:24.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.944+0000 7f8234ff9640 1 -- 172.21.15.106:0/2093954800 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8230058110 con 0x7f82381f1de0 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.947+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f821806b080 msgr2=0x7f821806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f821806b080 0x7f821806d540 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8228036000 tx=0x7f8228030000 comp rx=0 tx=0).stop 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 msgr2=0x7f82380b0680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 0x7f82380b0680 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f82380b1c80 tx=0x7f823000b0d0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 shutdown_connections 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82381f1de0 0x7f82380b0680 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8238143a30 0x7f8238142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f821806b080 0x7f821806d540 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 --2- 172.21.15.106:0/2093954800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82240039a0 0x7f8238158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 >> 172.21.15.106:0/2093954800 conn(0x7f823806a500 msgr2=0x7f823806c750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:24.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 shutdown_connections 2023-11-24T08:11:24.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:24.948+0000 7f823cd88640 1 -- 172.21.15.106:0/2093954800 wait complete. 2023-11-24T08:11:24.969 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:11:24.969 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:11:25.085 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:11:25.204 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:11:25.204 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:25.204 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:25.263 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:11:25.263 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:25.263 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:25.325 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:11:25.325 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:25.325 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:25.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:11:25.385 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:25.385 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:25.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:11:25.445 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:11:25.445 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.661+0000 7f8e72705640 1 Processor -- start 2023-11-24T08:11:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.661+0000 7f8e72705640 1 -- start start 2023-11-24T08:11:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.662+0000 7f8e72705640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e6c15c7f0 0x7f8e6c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.662+0000 7f8e72705640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.662+0000 7f8e72705640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e6c0cece0 con 0x7f8e6c0cf450 2023-11-24T08:11:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.662+0000 7f8e72705640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e6c0cfe40 con 0x7f8e6c15c7f0 2023-11-24T08:11:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.662+0000 7f8e72705640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c156010 con 0x7f8e6c0ce6c0 2023-11-24T08:11:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e6bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8e6c15c7f0 0x7f8e6c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e63fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8e6c0cf450 0x7f8e6c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40890/0 (socket says 172.21.15.106:40890) 2023-11-24T08:11:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e63fff640 1 -- 172.21.15.106:0/1128705912 learned_addr learned my addr 172.21.15.106:0/1128705912 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e6bfff640 1 -- 172.21.15.106:0/1128705912 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 msgr2=0x7f8e6c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e70ca2640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e6bfff640 1 -- 172.21.15.106:0/1128705912 >> v1:172.21.15.106:6789/0 conn(0x7f8e6c0cf450 legacy=0x7f8e6c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e6bfff640 1 -- 172.21.15.106:0/1128705912 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e6c06d040 con 0x7f8e6c15c7f0 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e6b7fe640 1 -- 172.21.15.106:0/1128705912 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081724072 0 0) 0x7f8e6c0cece0 con 0x7f8e6c0cf450 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.663+0000 7f8e70ca2640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.664+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c15c7f0 0x7f8e6c06db20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0097a0 tx=0x7f8e5c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb70f193fa4864d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:25.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e6b7fe640 1 -- 172.21.15.106:0/1128705912 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c02d040 con 0x7f8e6c15c7f0 2023-11-24T08:11:25.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e6b7fe640 1 -- 172.21.15.106:0/1128705912 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e5c031810 con 0x7f8e6c15c7f0 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e6b7fe640 1 -- 172.21.15.106:0/1128705912 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e5c031ba0 con 0x7f8e6c15c7f0 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e72705640 1 -- 172.21.15.106:0/1128705912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c15c7f0 msgr2=0x7f8e6c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e72705640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c15c7f0 0x7f8e6c06db20 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c0097a0 tx=0x7f8e5c02b8b0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e72705640 1 -- 172.21.15.106:0/1128705912 shutdown_connections 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e72705640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c0cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e72705640 1 --2- 172.21.15.106:0/1128705912 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c15c7f0 0x7f8e6c06db20 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.665+0000 7f8e72705640 1 -- 172.21.15.106:0/1128705912 >> 172.21.15.106:0/1128705912 conn(0x7f8e6c06a920 msgr2=0x7f8e6c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.666+0000 7f8e72705640 1 -- 172.21.15.106:0/1128705912 shutdown_connections 2023-11-24T08:11:25.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.666+0000 7f8e72705640 1 -- 172.21.15.106:0/1128705912 wait complete. 2023-11-24T08:11:25.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.667+0000 7f8e72705640 1 Processor -- start 2023-11-24T08:11:25.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.667+0000 7f8e72705640 1 -- start start 2023-11-24T08:11:25.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.668+0000 7f8e72705640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:25.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.668+0000 7f8e72705640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 0x7f8e6c155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.668+0000 7f8e72705640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e6c15c7f0 0x7f8e6c0ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.668+0000 7f8e72705640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c154f30 con 0x7f8e6c0cf450 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.668+0000 7f8e72705640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0b40f0 con 0x7f8e6c15c7f0 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.668+0000 7f8e72705640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e6c0b3e10 con 0x7f8e6c0ce6c0 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 0x7f8e6c155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 0x7f8e6c155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43390/0 (socket says 172.21.15.106:43390) 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 -- 172.21.15.106:0/3268839931 learned_addr learned my addr 172.21.15.106:0/3268839931 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 -- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 msgr2=0x7f8e6c150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:25.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 -- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e6c15c7f0 msgr2=0x7f8e6c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:25.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e6c15c7f0 0x7f8e6c0ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 -- 172.21.15.106:0/3268839931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c009400 con 0x7f8e6c0cf450 2023-11-24T08:11:25.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.669+0000 7f8e63fff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 0x7f8e6c155350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c4d0 tx=0x7f8e5400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.670+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e540074e0 con 0x7f8e6c0cf450 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.670+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e54007660 con 0x7f8e6c0cf450 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.670+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54004a20 con 0x7f8e6c0cf450 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.670+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e6c154870 con 0x7f8e6c0cf450 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.670+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e6c150ec0 con 0x7f8e6c0cf450 2023-11-24T08:11:25.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.671+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e6c155070 con 0x7f8e6c0cf450 2023-11-24T08:11:25.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.672+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f8e54004b80 con 0x7f8e6c0cf450 2023-11-24T08:11:25.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.673+0000 7f8e697fa640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e4c06b0b0 0x7f8e4c06d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:25.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.673+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8e54088890 con 0x7f8e6c0cf450 2023-11-24T08:11:25.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.673+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e4c06b0b0 0x7f8e4c06d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:25.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.674+0000 7f8e6bfff640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e4c06b0b0 0x7f8e4c06d570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c036000 tx=0x7f8e5c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.677+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8e6c155070 con 0x7f8e6c0cf450 2023-11-24T08:11:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.907+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e6c155070 con 0x7f8e6c0cf450 2023-11-24T08:11:25.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.908+0000 7f8e697fa640 1 -- 172.21.15.106:0/3268839931 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f8e6c155070 con 0x7f8e6c0cf450 2023-11-24T08:11:25.910 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:25.910 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/1559795601","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1559795601},{"type":"v1","addr":"172.21.15.156:6833","nonce":1559795601}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":18},{"gid":4864,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/1947011990","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":1947011990},{"type":"v1","addr":"172.21.15.156:6835","nonce":1947011990}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5553,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1746575681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1746575681},{"type":"v1","addr":"172.21.15.106:6835","nonce":1746575681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":19},{"gid":5973,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2463317484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":2463317484},{"type":"v1","addr":"172.21.15.106:6837","nonce":2463317484}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":28}],"filesystems":[]} 2023-11-24T08:11:25.910 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 28 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e4c06b0b0 msgr2=0x7f8e4c06d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e4c06b0b0 0x7f8e4c06d570 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c036000 tx=0x7f8e5c03c040 comp rx=0 tx=0).stop 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 msgr2=0x7f8e6c155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 0x7f8e6c155350 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f8e5400c4d0 tx=0x7f8e5400c990 comp rx=0 tx=0).stop 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 shutdown_connections 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e6c15c7f0 0x7f8e6c0ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8e6c0cf450 0x7f8e6c155350 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8e4c06b0b0 0x7f8e4c06d570 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 --2- 172.21.15.106:0/3268839931 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8e6c0ce6c0 0x7f8e6c150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 >> 172.21.15.106:0/3268839931 conn(0x7f8e6c06a920 msgr2=0x7f8e6c0cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 shutdown_connections 2023-11-24T08:11:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:25.911+0000 7f8e72705640 1 -- 172.21.15.106:0/3268839931 wait complete. 2023-11-24T08:11:25.931 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:11:25.931 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:11:25.931 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:11:25.931 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:11:25.931 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:11:25.931 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:11:25.932 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:11:25.932 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:11:25.932 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:11:25.932 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:11:25.932 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:11:25.932 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:11:25.932 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:11:25.933 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:11:25.933 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:11:25.933 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:11:25.948 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:11:25.948 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:11:25.949 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:11:25.949 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:11:25.955 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:11:25.955 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:11:25.956 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:11:25.956 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:11:25.997 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:11:25.997 DEBUG:teuthology.parallel:result is None 2023-11-24T08:11:26.004 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:11:26.004 DEBUG:teuthology.parallel:result is None 2023-11-24T08:11:26.005 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:11:26.005 DEBUG:teuthology.parallel:result is None 2023-11-24T08:11:26.011 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:11:26.011 DEBUG:teuthology.parallel:result is None 2023-11-24T08:11:26.011 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:11:26.159 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:11:26.166 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:11:26.168 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-11-24T08:11:26.168 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:11:26.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.199+0000 7f3ce359e640 1 Processor -- start 2023-11-24T08:11:26.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.199+0000 7f3ce359e640 1 -- start start 2023-11-24T08:11:26.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.200+0000 7f3ce359e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3ce415d480 0x7f3ce40af190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.200+0000 7f3ce359e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce406b170 con 0x7f3ce40af6d0 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.200+0000 7f3ce359e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ce406a5a0 con 0x7f3ce40b1880 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.200+0000 7f3ce359e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ce406a840 con 0x7f3ce415d480 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.200+0000 7f3ce2d9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3ce40af6d0 0x7f3ce40afc10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40914/0 (socket says 172.21.15.106:40914) 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce2d9d640 1 -- 172.21.15.106:0/3137143413 learned_addr learned my addr 172.21.15.106:0/3137143413 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce259c640 1 --2- 172.21.15.106:0/3137143413 >> v2:172.21.15.156:3300/0 conn(0x7f3ce415d480 0x7f3ce40af190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168261793 0 0) 0x7f3ce406b170 con 0x7f3ce40af6d0 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd0003520 con 0x7f3ce40af6d0 2023-11-24T08:11:26.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194859417 0 0) 0x7f3ce406a5a0 con 0x7f3ce40b1880 2023-11-24T08:11:26.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ce406b170 con 0x7f3ce40b1880 2023-11-24T08:11:26.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce259c640 1 -- 172.21.15.106:0/3137143413 >> v1:172.21.15.156:6789/0 conn(0x7f3ce40b1880 legacy=0x7f3ce40cf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:26.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce259c640 1 -- 172.21.15.106:0/3137143413 >> v1:172.21.15.106:6789/0 conn(0x7f3ce40af6d0 legacy=0x7f3ce40afc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:26.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.201+0000 7f3ce259c640 1 -- 172.21.15.106:0/3137143413 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce406ab20 con 0x7f3ce415d480 2023-11-24T08:11:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.202+0000 7f3ce259c640 1 --2- 172.21.15.106:0/3137143413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40af190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=278487c4bdc97822 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4031830 con 0x7f3ce415d480 2023-11-24T08:11:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd4031990 con 0x7f3ce415d480 2023-11-24T08:11:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce1d9b640 1 -- 172.21.15.106:0/3137143413 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4031d20 con 0x7f3ce415d480 2023-11-24T08:11:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce359e640 1 -- 172.21.15.106:0/3137143413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 msgr2=0x7f3ce40af190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:26.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3137143413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40af190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f3cd40097a0 tx=0x7f3cd402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce359e640 1 -- 172.21.15.106:0/3137143413 shutdown_connections 2023-11-24T08:11:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3137143413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40af190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce359e640 1 -- 172.21.15.106:0/3137143413 >> 172.21.15.106:0/3137143413 conn(0x7f3ce40999d0 msgr2=0x7f3ce40d01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.203+0000 7f3ce359e640 1 -- 172.21.15.106:0/3137143413 shutdown_connections 2023-11-24T08:11:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.204+0000 7f3ce359e640 1 -- 172.21.15.106:0/3137143413 wait complete. 2023-11-24T08:11:26.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.205+0000 7f3ce359e640 1 Processor -- start 2023-11-24T08:11:26.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.205+0000 7f3ce359e640 1 -- start start 2023-11-24T08:11:26.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.206+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ce40af6d0 0x7f3ce4156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:26.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.206+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 0x7f3ce40b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:26.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.206+0000 7f3ce359e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40b59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:26.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.206+0000 7f3ce359e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce4156ba0 con 0x7f3ce40b1880 2023-11-24T08:11:26.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.206+0000 7f3ce359e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce41595f0 con 0x7f3ce415d480 2023-11-24T08:11:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.206+0000 7f3ce359e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ce41598d0 con 0x7f3ce40af6d0 2023-11-24T08:11:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 0x7f3ce40b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 0x7f3ce40b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43452/0 (socket says 172.21.15.106:43452) 2023-11-24T08:11:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 -- 172.21.15.106:0/3984562610 learned_addr learned my addr 172.21.15.106:0/3984562610 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3ce2d9d640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40b59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:26.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3ce259c640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ce40af6d0 0x7f3ce4156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 -- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ce40af6d0 msgr2=0x7f3ce4156660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ce40af6d0 0x7f3ce4156660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 -- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 msgr2=0x7f3ce40b59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:26.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40b59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 -- 172.21.15.106:0/3984562610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd4009400 con 0x7f3ce40b1880 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.207+0000 7f3cdbfff640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 0x7f3ce40b5490 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00c450 tx=0x7f3ccc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.208+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc0052d0 con 0x7f3ce40b1880 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.208+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ccc005430 con 0x7f3ce40b1880 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.208+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ccc012870 con 0x7f3ce40b1880 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.208+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ce4159bb0 con 0x7f3ce40b1880 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.208+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ce415a170 con 0x7f3ce40b1880 2023-11-24T08:11:26.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.209+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ce40b3810 con 0x7f3ce40b1880 2023-11-24T08:11:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.210+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f3ccc0129d0 con 0x7f3ce40b1880 2023-11-24T08:11:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.211+0000 7f3cdaffd640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3cc406b080 0x7f3cc406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:26.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.211+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3ccc089180 con 0x7f3ce40b1880 2023-11-24T08:11:26.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.211+0000 7f3ce259c640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3cc406b080 0x7f3cc406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:26.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.212+0000 7f3ce259c640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3cc406b080 0x7f3cc406d540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4036000 tx=0x7f3cd403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:26.217 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.216+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3ccc054270 con 0x7f3ce40b1880 2023-11-24T08:11:26.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:26.412+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3ce404e840 con 0x7f3ce40b1880 2023-11-24T08:11:27.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.527+0000 7f3cdaffd640 1 -- 172.21.15.106:0/3984562610 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v58) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3ccc057890 con 0x7f3ce40b1880 2023-11-24T08:11:27.529 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2023-11-24T08:11:27.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3cc406b080 msgr2=0x7f3cc406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3cc406b080 0x7f3cc406d540 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3cd4036000 tx=0x7f3cd403c040 comp rx=0 tx=0).stop 2023-11-24T08:11:27.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 msgr2=0x7f3ce40b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 0x7f3ce40b5490 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3ccc00c450 tx=0x7f3ccc00c910 comp rx=0 tx=0).stop 2023-11-24T08:11:27.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 shutdown_connections 2023-11-24T08:11:27.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ce415d480 0x7f3ce40b59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3ce40b1880 0x7f3ce40b5490 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3cc406b080 0x7f3cc406d540 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 --2- 172.21.15.106:0/3984562610 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3ce40af6d0 0x7f3ce4156660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 >> 172.21.15.106:0/3984562610 conn(0x7f3ce40999d0 msgr2=0x7f3ce406e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:27.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 shutdown_connections 2023-11-24T08:11:27.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.531+0000 7f3ce359e640 1 -- 172.21.15.106:0/3984562610 wait complete. 2023-11-24T08:11:27.555 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:11:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.764+0000 7f92d4ae6640 1 Processor -- start 2023-11-24T08:11:27.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.764+0000 7f92d4ae6640 1 -- start start 2023-11-24T08:11:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.766+0000 7f92d4ae6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 0x7f92d006c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.766+0000 7f92d4ae6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d015b7a0 con 0x7f92d0070a80 2023-11-24T08:11:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.766+0000 7f92d4ae6640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d015ba60 con 0x7f92d006cee0 2023-11-24T08:11:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.766+0000 7f92d4ae6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006d360 con 0x7f92d013e0e0 2023-11-24T08:11:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.766+0000 7f92cffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f92d0070a80 0x7f92d013e810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40926/0 (socket says 172.21.15.106:40926) 2023-11-24T08:11:27.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.766+0000 7f92cffff640 1 -- 172.21.15.106:0/1990891483 learned_addr learned my addr 172.21.15.106:0/1990891483 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92cf7fe640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 0x7f92d006c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657931975 0 0) 0x7f92d015b7a0 con 0x7f92d0070a80 2023-11-24T08:11:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92bc003520 con 0x7f92d0070a80 2023-11-24T08:11:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369805608 0 0) 0x7f92bc003520 con 0x7f92d0070a80 2023-11-24T08:11:27.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 msgr2=0x7f92d006c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 0x7f92d006c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 >> v1:172.21.15.156:6789/0 conn(0x7f92d006cee0 legacy=0x7f92d0070370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d015bd40 con 0x7f92d0070a80 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f92c80019f0 con 0x7f92d0070a80 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 >> v1:172.21.15.106:6789/0 conn(0x7f92d0070a80 legacy=0x7f92d013e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.767+0000 7f92ceffd640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 0x7f92bc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92ceffd640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92bc003f70 0x7f92bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92ceffd640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc004d00 0x7f92bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d006d360 con 0x7f92bc003f70 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc00bac0 con 0x7f92bc004d00 2023-11-24T08:11:27.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc00bc80 con 0x7f92d013e0e0 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92c7fff640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92bc003f70 0x7f92bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92cf7fe640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 0x7f92bc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92cffff640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc004d00 0x7f92bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92c7fff640 1 -- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 msgr2=0x7f92bc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92c7fff640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92d013e0e0 0x7f92bc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92c7fff640 1 -- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc004d00 msgr2=0x7f92bc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92c7fff640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92bc004d00 0x7f92bc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.768+0000 7f92c7fff640 1 -- 172.21.15.106:0/1990891483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc003520 con 0x7f92bc003f70 2023-11-24T08:11:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.769+0000 7f92c7fff640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92bc003f70 0x7f92bc0047c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f92b8001750 tx=0x7f92bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f45766035aca94 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b80052d0 con 0x7f92bc003f70 2023-11-24T08:11:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b802e050 con 0x7f92bc003f70 2023-11-24T08:11:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92ceffd640 1 -- 172.21.15.106:0/1990891483 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b8003830 con 0x7f92bc003f70 2023-11-24T08:11:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92d4ae6640 1 -- 172.21.15.106:0/1990891483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92bc003f70 msgr2=0x7f92bc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/1990891483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92bc003f70 0x7f92bc0047c0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f92b8001750 tx=0x7f92bc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92cffff640 1 -- 172.21.15.106:0/1990891483 reap_dead start 2023-11-24T08:11:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92d4ae6640 1 -- 172.21.15.106:0/1990891483 shutdown_connections 2023-11-24T08:11:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92d4ae6640 1 -- 172.21.15.106:0/1990891483 >> 172.21.15.106:0/1990891483 conn(0x7f92d00b4630 msgr2=0x7f92d00b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92d4ae6640 1 -- 172.21.15.106:0/1990891483 shutdown_connections 2023-11-24T08:11:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.770+0000 7f92d4ae6640 1 -- 172.21.15.106:0/1990891483 wait complete. 2023-11-24T08:11:27.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.772+0000 7f92d4ae6640 1 Processor -- start 2023-11-24T08:11:27.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.772+0000 7f92d4ae6640 1 -- start start 2023-11-24T08:11:27.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92d4ae6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92bc003f70 0x7f92d0154cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92d4ae6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d01e3930 0x7f92d00b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92d4ae6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 0x7f92d01f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92d4ae6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0155210 con 0x7f92d01f31d0 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92d4ae6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01e3ea0 con 0x7f92d01e3930 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92d4ae6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d01e4180 con 0x7f92bc003f70 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 0x7f92d01f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 0x7f92d01f3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43470/0 (socket says 172.21.15.106:43470) 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.773+0000 7f92cffff640 1 -- 172.21.15.106:0/3135662062 learned_addr learned my addr 172.21.15.106:0/3135662062 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cf7fe640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92bc003f70 0x7f92d0154cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92c7fff640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d01e3930 0x7f92d00b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cffff640 1 -- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92bc003f70 msgr2=0x7f92d0154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cffff640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92bc003f70 0x7f92d0154cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cffff640 1 -- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d01e3930 msgr2=0x7f92d00b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cffff640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d01e3930 0x7f92d00b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cffff640 1 -- 172.21.15.106:0/3135662062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d01e4460 con 0x7f92d01f31d0 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92cffff640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 0x7f92d01f3650 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f92d006e550 tx=0x7f92c800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8007ae0 con 0x7f92d01f31d0 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c800ee20 con 0x7f92d01f31d0 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d01e4740 con 0x7f92d01f31d0 2023-11-24T08:11:27.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.774+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8013680 con 0x7f92d01f31d0 2023-11-24T08:11:27.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.775+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d00b0e40 con 0x7f92d01f31d0 2023-11-24T08:11:27.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.776+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d004e840 con 0x7f92d01f31d0 2023-11-24T08:11:27.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.777+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f92c80137e0 con 0x7f92d01f31d0 2023-11-24T08:11:27.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.778+0000 7f92ccff9640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f92b006b0b0 0x7f92b006d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:27.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.778+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f92c8089a20 con 0x7f92d01f31d0 2023-11-24T08:11:27.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.778+0000 7f92cf7fe640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f92b006b0b0 0x7f92b006d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:27.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.779+0000 7f92cf7fe640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f92b006b0b0 0x7f92b006d570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009950 tx=0x7f92c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:27.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:27.782+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f92c8054c20 con 0x7f92d01f31d0 2023-11-24T08:11:28.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.040+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f92d00cbc40 con 0x7f92d01f31d0 2023-11-24T08:11:28.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.041+0000 7f92ccff9640 1 -- 172.21.15.106:0/3135662062 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f92c8058240 con 0x7f92d01f31d0 2023-11-24T08:11:28.043 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:28.043 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:11:28.043 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2023-11-24T08:11:28.043 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.043 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A==", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg==", 2023-11-24T08:11:28.044 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q==", 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.045 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw==", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:11:28.046 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA==", 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.047 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA==", 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.048 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA==", 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2023-11-24T08:11:28.049 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw==", 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw==", 2023-11-24T08:11:28.050 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw==", 2023-11-24T08:11:28.051 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA==", 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.052 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug==", 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:11:28.053 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ==", 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2023-11-24T08:11:28.054 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg==", 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:11:28.055 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:11:28.056 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:11:28.056 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.056 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.056 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.056 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2023-11-24T08:11:28.056 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg==", 2023-11-24T08:11:28.057 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.057 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.057 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:11:28.057 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:11:28.057 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:11:28.057 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w==", 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:11:28.058 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBludbKAhAARxKqsuG6+yx4M/G/4IloBw==", 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:11:28.059 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2023-11-24T08:11:28.060 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ==", 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:28.061 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2023-11-24T08:11:28.062 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA==", 2023-11-24T08:11:28.062 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:11:28.062 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:11:28.062 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:11:28.063 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:11:28.063 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.063 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:28.063 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2023-11-24T08:11:28.063 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:11:28.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f92b006b0b0 msgr2=0x7f92b006d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f92b006b0b0 0x7f92b006d570 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f92c0009950 tx=0x7f92c0008040 comp rx=0 tx=0).stop 2023-11-24T08:11:28.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 msgr2=0x7f92d01f3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 0x7f92d01f3650 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f92d006e550 tx=0x7f92c800e530 comp rx=0 tx=0).stop 2023-11-24T08:11:28.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 shutdown_connections 2023-11-24T08:11:28.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f92d01f31d0 0x7f92d01f3650 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92d01e3930 0x7f92d00b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f92b006b0b0 0x7f92b006d570 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 --2- 172.21.15.106:0/3135662062 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f92bc003f70 0x7f92d0154cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 >> 172.21.15.106:0/3135662062 conn(0x7f92d00b4630 msgr2=0x7f92d015d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:28.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 shutdown_connections 2023-11-24T08:11:28.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.044+0000 7f92d4ae6640 1 -- 172.21.15.106:0/3135662062 wait complete. 2023-11-24T08:11:28.067 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.278+0000 7f0cf7051640 1 Processor -- start 2023-11-24T08:11:28.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.278+0000 7f0cf7051640 1 -- start start 2023-11-24T08:11:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.279+0000 7f0cf7051640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0cf015c7f0 0x7f0cf006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.279+0000 7f0cf7051640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0cf00cf450 0x7f0cf00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0cf7051640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0cf00cece0 con 0x7f0cf00cdfb0 2023-11-24T08:11:28.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0cf7051640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00cfe40 con 0x7f0cf00cf450 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0cf7051640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cf00ceaf0 con 0x7f0cf015c7f0 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0ceffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0cf00cf450 0x7f0cf00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0ceffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0cf00cf450 0x7f0cf00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43486/0 (socket says 172.21.15.106:43486) 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0ceffff640 1 -- 172.21.15.106:0/4193672144 learned_addr learned my addr 172.21.15.106:0/4193672144 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.280+0000 7f0cf4ded640 1 --2- 172.21.15.106:0/4193672144 >> v2:172.21.15.156:3300/0 conn(0x7f0cf015c7f0 0x7f0cf006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.281+0000 7f0ceffff640 1 -- 172.21.15.106:0/4193672144 >> v2:172.21.15.156:3300/0 conn(0x7f0cf015c7f0 msgr2=0x7f0cf006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.281+0000 7f0ceffff640 1 --2- 172.21.15.106:0/4193672144 >> v2:172.21.15.156:3300/0 conn(0x7f0cf015c7f0 0x7f0cf006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.281+0000 7f0ceffff640 1 -- 172.21.15.106:0/4193672144 >> v1:172.21.15.156:6789/0 conn(0x7f0cf00cdfb0 legacy=0x7f0cf00ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:28.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.281+0000 7f0ceffff640 1 -- 172.21.15.106:0/4193672144 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf006d040 con 0x7f0cf00cf450 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.282+0000 7f0ceffff640 1 --2- 172.21.15.106:0/4193672144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00cf900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00d0130 tx=0x7f0cdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9c4faa2bf354bfc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.282+0000 7f0cef7fe640 1 -- 172.21.15.106:0/4193672144 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc031870 con 0x7f0cf00cf450 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.282+0000 7f0cef7fe640 1 -- 172.21.15.106:0/4193672144 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc0319d0 con 0x7f0cf00cf450 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.282+0000 7f0cef7fe640 1 -- 172.21.15.106:0/4193672144 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc031d60 con 0x7f0cf00cf450 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 -- 172.21.15.106:0/4193672144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 msgr2=0x7f0cf00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 --2- 172.21.15.106:0/4193672144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00cf900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00d0130 tx=0x7f0cdc02b910 comp rx=0 tx=0).stop 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 -- 172.21.15.106:0/4193672144 shutdown_connections 2023-11-24T08:11:28.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 --2- 172.21.15.106:0/4193672144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00cf900 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 --2- 172.21.15.106:0/4193672144 >> v2:172.21.15.156:3300/0 conn(0x7f0cf015c7f0 0x7f0cf006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 -- 172.21.15.106:0/4193672144 >> 172.21.15.106:0/4193672144 conn(0x7f0cf006a920 msgr2=0x7f0cf006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:28.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 -- 172.21.15.106:0/4193672144 shutdown_connections 2023-11-24T08:11:28.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.283+0000 7f0cf7051640 1 -- 172.21.15.106:0/4193672144 wait complete. 2023-11-24T08:11:28.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.285+0000 7f0cf7051640 1 Processor -- start 2023-11-24T08:11:28.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.285+0000 7f0cf7051640 1 -- start start 2023-11-24T08:11:28.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0cf7051640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0cf00cdfb0 0x7f0cf01473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0cf7051640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0cf7051640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf015c7f0 0x7f0cf00ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0cf7051640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf009bd70 con 0x7f0cf00cf450 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0cf7051640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf009cbb0 con 0x7f0cf015c7f0 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0cf7051640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cf009bfa0 con 0x7f0cf00cdfb0 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0ceffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43492/0 (socket says 172.21.15.106:43492) 2023-11-24T08:11:28.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0ceffff640 1 -- 172.21.15.106:0/3178687940 learned_addr learned my addr 172.21.15.106:0/3178687940 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.286+0000 7f0ceffff640 1 -- 172.21.15.106:0/3178687940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0cf00cdfb0 msgr2=0x7f0cf01473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0cf00cdfb0 0x7f0cf01473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ceffff640 1 -- 172.21.15.106:0/3178687940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf015c7f0 msgr2=0x7f0cf00ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf015c7f0 0x7f0cf00ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ceffff640 1 -- 172.21.15.106:0/3178687940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc0094d0 con 0x7f0cf00cf450 2023-11-24T08:11:28.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ceffff640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00ad2b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000c00 tx=0x7f0cdc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc03abe0 con 0x7f0cf00cf450 2023-11-24T08:11:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cdc03ad40 con 0x7f0cf00cf450 2023-11-24T08:11:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cf009c280 con 0x7f0cf00cf450 2023-11-24T08:11:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cdc044610 con 0x7f0cf00cf450 2023-11-24T08:11:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.287+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cf009c840 con 0x7f0cf00cf450 2023-11-24T08:11:28.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.288+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cf0067470 con 0x7f0cf00cf450 2023-11-24T08:11:28.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.289+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0cdc04d450 con 0x7f0cf00cf450 2023-11-24T08:11:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.291+0000 7f0ced7fa640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0cd406b080 0x7f0cd406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.291+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0cdc02d040 con 0x7f0cf00cf450 2023-11-24T08:11:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.291+0000 7f0cf4ded640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0cd406b080 0x7f0cd406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.292+0000 7f0cf4ded640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0cd406b080 0x7f0cd406d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006fd0 tx=0x7f0ce4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:28.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.296+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0cdc076de0 con 0x7f0cf00cf450 2023-11-24T08:11:28.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.484+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cf00cd730 con 0x7f0cf00cf450 2023-11-24T08:11:28.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.486+0000 7f0ced7fa640 1 -- 172.21.15.106:0/3178687940 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f0cdc035090 con 0x7f0cf00cf450 2023-11-24T08:11:28.488 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:28.489 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":58,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:27.524959+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":46,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.488+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0cd406b080 msgr2=0x7f0cd406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.488+0000 7f0cf7051640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0cd406b080 0x7f0cd406d540 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0ce4006fd0 tx=0x7f0ce4008040 comp rx=0 tx=0).stop 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.488+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 msgr2=0x7f0cf00ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.488+0000 7f0cf7051640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00ad2b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc000c00 tx=0x7f0cdc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 shutdown_connections 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0cf015c7f0 0x7f0cf00ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0cf00cf450 0x7f0cf00ad2b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0cd406b080 0x7f0cd406d540 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 --2- 172.21.15.106:0/3178687940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0cf00cdfb0 0x7f0cf01473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 >> 172.21.15.106:0/3178687940 conn(0x7f0cf006a920 msgr2=0x7f0cf006c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:28.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 shutdown_connections 2023-11-24T08:11:28.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.489+0000 7f0cf7051640 1 -- 172.21.15.106:0/3178687940 wait complete. 2023-11-24T08:11:28.508 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:28.508 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.719+0000 7fe852cd3640 1 Processor -- start 2023-11-24T08:11:28.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.720+0000 7fe852cd3640 1 -- start start 2023-11-24T08:11:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe852cd3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe84c0d4010 0x7fe84c0cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe852cd3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0cfc80 0x7fe84c0d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe852cd3640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84c0d05a0 con 0x7fe84c15d580 2023-11-24T08:11:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe852cd3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe84c0cf300 con 0x7fe84c0d4010 2023-11-24T08:11:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe852cd3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0d1090 con 0x7fe84c0cfc80 2023-11-24T08:11:28.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe84bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe84c0d4010 0x7fe84c0cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe84bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe84c0d4010 0x7fe84c0cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43496/0 (socket says 172.21.15.106:43496) 2023-11-24T08:11:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.721+0000 7fe84bfff640 1 -- 172.21.15.106:0/2868361704 learned_addr learned my addr 172.21.15.106:0/2868361704 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.722+0000 7fe84bfff640 1 -- 172.21.15.106:0/2868361704 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0cfc80 msgr2=0x7fe84c0d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.722+0000 7fe84bfff640 1 --2- 172.21.15.106:0/2868361704 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0cfc80 0x7fe84c0d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.722+0000 7fe84bfff640 1 -- 172.21.15.106:0/2868361704 >> v1:172.21.15.156:6789/0 conn(0x7fe84c15d580 legacy=0x7fe84c0d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:28.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.722+0000 7fe84bfff640 1 -- 172.21.15.106:0/2868361704 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe84c0d1370 con 0x7fe84c0d4010 2023-11-24T08:11:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.723+0000 7fe84bfff640 1 --2- 172.21.15.106:0/2868361704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0d4010 0x7fe84c0cead0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0b9430 tx=0x7fe83402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c25e255a5c6e464 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.723+0000 7fe84b7fe640 1 -- 172.21.15.106:0/2868361704 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834031640 con 0x7fe84c0d4010 2023-11-24T08:11:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.723+0000 7fe84b7fe640 1 -- 172.21.15.106:0/2868361704 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8340317a0 con 0x7fe84c0d4010 2023-11-24T08:11:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.723+0000 7fe84b7fe640 1 -- 172.21.15.106:0/2868361704 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe834031b30 con 0x7fe84c0d4010 2023-11-24T08:11:28.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.723+0000 7fe852cd3640 1 -- 172.21.15.106:0/2868361704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0d4010 msgr2=0x7fe84c0cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.723+0000 7fe852cd3640 1 --2- 172.21.15.106:0/2868361704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0d4010 0x7fe84c0cead0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fe84c0b9430 tx=0x7fe83402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.724+0000 7fe852cd3640 1 -- 172.21.15.106:0/2868361704 shutdown_connections 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.724+0000 7fe852cd3640 1 --2- 172.21.15.106:0/2868361704 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0cfc80 0x7fe84c0d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.724+0000 7fe852cd3640 1 --2- 172.21.15.106:0/2868361704 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0d4010 0x7fe84c0cead0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.724+0000 7fe852cd3640 1 -- 172.21.15.106:0/2868361704 >> 172.21.15.106:0/2868361704 conn(0x7fe84c09c770 msgr2=0x7fe84c0d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.724+0000 7fe852cd3640 1 -- 172.21.15.106:0/2868361704 shutdown_connections 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.724+0000 7fe852cd3640 1 -- 172.21.15.106:0/2868361704 wait complete. 2023-11-24T08:11:28.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.725+0000 7fe852cd3640 1 Processor -- start 2023-11-24T08:11:28.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.725+0000 7fe852cd3640 1 -- start start 2023-11-24T08:11:28.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.726+0000 7fe852cd3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 0x7fe84c0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.726+0000 7fe852cd3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0d4010 0x7fe84c0c9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.726+0000 7fe852cd3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe84c15d580 0x7fe84c0cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.726+0000 7fe852cd3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c158a30 con 0x7fe84c0cfc80 2023-11-24T08:11:28.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.726+0000 7fe852cd3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0c7380 con 0x7fe84c15d580 2023-11-24T08:11:28.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.726+0000 7fe852cd3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84c0c7660 con 0x7fe84c0d4010 2023-11-24T08:11:28.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 0x7fe84c0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 0x7fe84c0c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43506/0 (socket says 172.21.15.106:43506) 2023-11-24T08:11:28.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 -- 172.21.15.106:0/168583146 learned_addr learned my addr 172.21.15.106:0/168583146 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 -- 172.21.15.106:0/168583146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0d4010 msgr2=0x7fe84c0c9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0d4010 0x7fe84c0c9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 -- 172.21.15.106:0/168583146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe84c15d580 msgr2=0x7fe84c0cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe84c15d580 0x7fe84c0cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 -- 172.21.15.106:0/168583146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8340094d0 con 0x7fe84c0cfc80 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.727+0000 7fe850a6f640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 0x7fe84c0c6cf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c7a0 tx=0x7fe83c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.728+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c003d80 con 0x7fe84c0cfc80 2023-11-24T08:11:28.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.728+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83c005070 con 0x7fe84c0cfc80 2023-11-24T08:11:28.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.728+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83c005460 con 0x7fe84c0cfc80 2023-11-24T08:11:28.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.728+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe84c0c7940 con 0x7fe84c0cfc80 2023-11-24T08:11:28.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.728+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84c0c7f00 con 0x7fe84c0cfc80 2023-11-24T08:11:28.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.729+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84c0c80f0 con 0x7fe84c0cfc80 2023-11-24T08:11:28.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.730+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe83c0055c0 con 0x7fe84c0cfc80 2023-11-24T08:11:28.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.731+0000 7fe8497fa640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe82c06b080 0x7fe82c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:28.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.731+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe83c088cf0 con 0x7fe84c0cfc80 2023-11-24T08:11:28.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.731+0000 7fe84bfff640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe82c06b080 0x7fe82c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:28.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.732+0000 7fe84bfff640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe82c06b080 0x7fe82c06d540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe834036000 tx=0x7fe834030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:28.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.736+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe83c053cf0 con 0x7fe84c0cfc80 2023-11-24T08:11:28.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.926+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe84c04e8b0 con 0x7fe84c0cfc80 2023-11-24T08:11:28.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.927+0000 7fe8497fa640 1 -- 172.21.15.106:0/168583146 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v58) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe83c017020 con 0x7fe84c0cfc80 2023-11-24T08:11:28.928 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:28.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe82c06b080 msgr2=0x7fe82c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe82c06b080 0x7fe82c06d540 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe834036000 tx=0x7fe834030000 comp rx=0 tx=0).stop 2023-11-24T08:11:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 msgr2=0x7fe84c0c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 0x7fe84c0c6cf0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fe83c00c7a0 tx=0x7fe83c00cc60 comp rx=0 tx=0).stop 2023-11-24T08:11:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 shutdown_connections 2023-11-24T08:11:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe84c15d580 0x7fe84c0cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe82c06b080 0x7fe82c06d540 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe84c0d4010 0x7fe84c0c9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 --2- 172.21.15.106:0/168583146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe84c0cfc80 0x7fe84c0c6cf0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:28.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 >> 172.21.15.106:0/168583146 conn(0x7fe84c09c770 msgr2=0x7fe84c0b2e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:28.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 shutdown_connections 2023-11-24T08:11:28.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:28.930+0000 7fe852cd3640 1 -- 172.21.15.106:0/168583146 wait complete. 2023-11-24T08:11:28.951 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:11:29.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.160+0000 7f169f59e640 1 Processor -- start 2023-11-24T08:11:29.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.160+0000 7f169f59e640 1 -- start start 2023-11-24T08:11:29.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.161+0000 7f169f59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a0144b30 0x7f16a015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.161+0000 7f169f59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f16a015c410 0x7f16a00b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.161+0000 7f169f59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16a00b1de0 0x7f16a00b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.161+0000 7f169f59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00b27b0 con 0x7f16a00b1de0 2023-11-24T08:11:29.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.161+0000 7f169f59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a015bf80 con 0x7f16a015c410 2023-11-24T08:11:29.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.161+0000 7f169f59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b1560 con 0x7f16a0144b30 2023-11-24T08:11:29.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16a00b1de0 0x7f16a00b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16a00b1de0 0x7f16a00b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43510/0 (socket says 172.21.15.106:43510) 2023-11-24T08:11:29.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 -- 172.21.15.106:0/3249552327 learned_addr learned my addr 172.21.15.106:0/3249552327 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:29.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169e59c640 1 --2- 172.21.15.106:0/3249552327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a0144b30 0x7f16a015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f1697fff640 1 --2- 172.21.15.106:0/3249552327 >> v2:172.21.15.156:3300/0 conn(0x7f16a015c410 0x7f16a00b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 -- 172.21.15.106:0/3249552327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a0144b30 msgr2=0x7f16a015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:29.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 --2- 172.21.15.106:0/3249552327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a0144b30 0x7f16a015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 -- 172.21.15.106:0/3249552327 >> v2:172.21.15.156:3300/0 conn(0x7f16a015c410 msgr2=0x7f16a00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:29.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 --2- 172.21.15.106:0/3249552327 >> v2:172.21.15.156:3300/0 conn(0x7f16a015c410 0x7f16a00b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.162+0000 7f169ed9d640 1 -- 172.21.15.106:0/3249552327 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a00b0fb0 con 0x7f16a00b1de0 2023-11-24T08:11:29.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.163+0000 7f169ed9d640 1 --2- 172.21.15.106:0/3249552327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a00b2270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f16a006e820 tx=0x7f169802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e532f190525a40dc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169dd9b640 1 -- 172.21.15.106:0/3249552327 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169802ca70 con 0x7f16a00b1de0 2023-11-24T08:11:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169dd9b640 1 -- 172.21.15.106:0/3249552327 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169802cbd0 con 0x7f16a00b1de0 2023-11-24T08:11:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169dd9b640 1 -- 172.21.15.106:0/3249552327 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16980343f0 con 0x7f16a00b1de0 2023-11-24T08:11:29.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 -- 172.21.15.106:0/3249552327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 msgr2=0x7f16a00b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 --2- 172.21.15.106:0/3249552327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a00b2270 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f16a006e820 tx=0x7f169802bed0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 -- 172.21.15.106:0/3249552327 shutdown_connections 2023-11-24T08:11:29.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 --2- 172.21.15.106:0/3249552327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a00b2270 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 --2- 172.21.15.106:0/3249552327 >> v2:172.21.15.156:3300/0 conn(0x7f16a015c410 0x7f16a00b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 --2- 172.21.15.106:0/3249552327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a0144b30 0x7f16a015f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.164+0000 7f169f59e640 1 -- 172.21.15.106:0/3249552327 >> 172.21.15.106:0/3249552327 conn(0x7f16a00cb830 msgr2=0x7f16a00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.165+0000 7f169f59e640 1 -- 172.21.15.106:0/3249552327 shutdown_connections 2023-11-24T08:11:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.165+0000 7f169f59e640 1 -- 172.21.15.106:0/3249552327 wait complete. 2023-11-24T08:11:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.166+0000 7f169f59e640 1 Processor -- start 2023-11-24T08:11:29.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.166+0000 7f169f59e640 1 -- start start 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.167+0000 7f169f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a0142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.167+0000 7f169f59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16a0144b30 0x7f16a0136500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.167+0000 7f169f59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a015c410 0x7f16a015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.167+0000 7f169f59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01428a0 con 0x7f16a00b1de0 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.167+0000 7f169f59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00a8650 con 0x7f16a0144b30 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.167+0000 7f169f59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0141c20 con 0x7f16a015c410 2023-11-24T08:11:29.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a0142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a0142cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43514/0 (socket says 172.21.15.106:43514) 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 -- 172.21.15.106:0/3695382195 learned_addr learned my addr 172.21.15.106:0/3695382195 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 -- 172.21.15.106:0/3695382195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a015c410 msgr2=0x7f16a015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a015c410 0x7f16a015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 -- 172.21.15.106:0/3695382195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16a0144b30 msgr2=0x7f16a0136500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16a0144b30 0x7f16a0136500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 -- 172.21.15.106:0/3695382195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698009b30 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f169e59c640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a0142cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f169000d800 tx=0x7f169000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.168+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16900096a0 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.169+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690009800 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.169+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0141f00 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.169+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690009b90 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.169+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a01424c0 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.170+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00b3610 con 0x7f16a00b1de0 2023-11-24T08:11:29.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.171+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f1690009cf0 con 0x7f16a00b1de0 2023-11-24T08:11:29.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.171+0000 7f1696ffd640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f168006b080 0x7f168006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.172+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1690089c10 con 0x7f16a00b1de0 2023-11-24T08:11:29.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.172+0000 7f1697fff640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f168006b080 0x7f168006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.172+0000 7f1697fff640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f168006b080 0x7f168006d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1688006fd0 tx=0x7f1688008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:29.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.176+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f16a00b3610 con 0x7f16a00b1de0 2023-11-24T08:11:29.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.370+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f16a00b3610 con 0x7f16a00b1de0 2023-11-24T08:11:29.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.371+0000 7f1696ffd640 1 -- 172.21.15.106:0/3695382195 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v58) v1 ==== 84+0+65 (secure 0 0 0) 0x7f16a00b3610 con 0x7f16a00b1de0 2023-11-24T08:11:29.372 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:29.372 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:11:29.372 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:29.373 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2023-11-24T08:11:29.373 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2023-11-24T08:11:29.373 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:29.373 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:11:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f168006b080 msgr2=0x7f168006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f168006b080 0x7f168006d540 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f1688006fd0 tx=0x7f1688008040 comp rx=0 tx=0).stop 2023-11-24T08:11:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 msgr2=0x7f16a0142cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a0142cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f169000d800 tx=0x7f169000dcc0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 shutdown_connections 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f168006b080 0x7f168006d540 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16a015c410 0x7f16a015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16a0144b30 0x7f16a0136500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 --2- 172.21.15.106:0/3695382195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16a00b1de0 0x7f16a0142cc0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.373+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 >> 172.21.15.106:0/3695382195 conn(0x7f16a00cb830 msgr2=0x7f16a00722e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.374+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 shutdown_connections 2023-11-24T08:11:29.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.374+0000 7f169f59e640 1 -- 172.21.15.106:0/3695382195 wait complete. 2023-11-24T08:11:29.394 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:11:29.394 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:11:29.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.609+0000 7f530e5c5640 1 Processor -- start 2023-11-24T08:11:29.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.609+0000 7f530e5c5640 1 -- start start 2023-11-24T08:11:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.611+0000 7f530e5c5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53081461d0 0x7f53080b4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.611+0000 7f530e5c5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f5308167110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.611+0000 7f530e5c5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530806d180 con 0x7f53080b4fe0 2023-11-24T08:11:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.611+0000 7f530e5c5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530806ba70 con 0x7f53081461d0 2023-11-24T08:11:29.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.611+0000 7f530e5c5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530806bd50 con 0x7f53080b3a60 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.611+0000 7f5307fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f53081461d0 0x7f53080b4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f53077fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f53080b4fe0 0x7f53080b3350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40942/0 (socket says 172.21.15.106:40942) 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f53077fe640 1 -- 172.21.15.106:0/2337286579 learned_addr learned my addr 172.21.15.106:0/2337286579 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f530cb62640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f5308167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f5307fff640 1 -- 172.21.15.106:0/2337286579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 msgr2=0x7f5308167110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f5307fff640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f5308167110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f5307fff640 1 -- 172.21.15.106:0/2337286579 >> v1:172.21.15.106:6789/0 conn(0x7f53080b4fe0 legacy=0x7f53080b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f5307fff640 1 -- 172.21.15.106:0/2337286579 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530806c030 con 0x7f53081461d0 2023-11-24T08:11:29.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f5306ffd640 1 -- 172.21.15.106:0/2337286579 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1988015949 0 0) 0x7f530806d180 con 0x7f53080b4fe0 2023-11-24T08:11:29.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.612+0000 7f530cb62640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f5308167110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:29.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.613+0000 7f5307fff640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53081461d0 0x7f53080b4aa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009dc0 tx=0x7f52f802be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4a088865c4710d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:29.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f5306ffd640 1 -- 172.21.15.106:0/2337286579 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f802cc30 con 0x7f53081461d0 2023-11-24T08:11:29.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f5306ffd640 1 -- 172.21.15.106:0/2337286579 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f802cd90 con 0x7f53081461d0 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f5306ffd640 1 -- 172.21.15.106:0/2337286579 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f8034660 con 0x7f53081461d0 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 -- 172.21.15.106:0/2337286579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53081461d0 msgr2=0x7f53080b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53081461d0 0x7f53080b4aa0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f52f8009dc0 tx=0x7f52f802be80 comp rx=0 tx=0).stop 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 -- 172.21.15.106:0/2337286579 shutdown_connections 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f5308167110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 --2- 172.21.15.106:0/2337286579 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53081461d0 0x7f53080b4aa0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 -- 172.21.15.106:0/2337286579 >> 172.21.15.106:0/2337286579 conn(0x7f5308061340 msgr2=0x7f5308061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:29.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 -- 172.21.15.106:0/2337286579 shutdown_connections 2023-11-24T08:11:29.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.614+0000 7f530e5c5640 1 -- 172.21.15.106:0/2337286579 wait complete. 2023-11-24T08:11:29.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.616+0000 7f530e5c5640 1 Processor -- start 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.616+0000 7f530e5c5640 1 -- start start 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f530e5c5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f53081412a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f530e5c5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 0x7f53080cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f530e5c5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53081461d0 0x7f53080a8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f530e5c5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080cc980 con 0x7f53080b4fe0 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f530e5c5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53080ccc60 con 0x7f53081461d0 2023-11-24T08:11:29.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f530e5c5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53080ccf40 con 0x7f53080b3a60 2023-11-24T08:11:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f53077fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 0x7f53080cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f53077fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 0x7f53080cda20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43532/0 (socket says 172.21.15.106:43532) 2023-11-24T08:11:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.617+0000 7f53077fe640 1 -- 172.21.15.106:0/67363902 learned_addr learned my addr 172.21.15.106:0/67363902 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f53077fe640 1 -- 172.21.15.106:0/67363902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 msgr2=0x7f53081412a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f53077fe640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f53081412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f53077fe640 1 -- 172.21.15.106:0/67363902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53081461d0 msgr2=0x7f53080a8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f53077fe640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53081461d0 0x7f53080a8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f53077fe640 1 -- 172.21.15.106:0/67363902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f8009a70 con 0x7f53080b4fe0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f53077fe640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 0x7f53080cda20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c4d0 tx=0x7f52f000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00074e0 con 0x7f53080b4fe0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f0007660 con 0x7f53080b4fe0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5308146730 con 0x7f53080b4fe0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.618+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f0004a20 con 0x7f53080b4fe0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.619+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53080cd500 con 0x7f53080b4fe0 2023-11-24T08:11:29.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.619+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5308067470 con 0x7f53080b4fe0 2023-11-24T08:11:29.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.621+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f52f0004b80 con 0x7f53080b4fe0 2023-11-24T08:11:29.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.621+0000 7f5304ff9640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f52e806b080 0x7f52e806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:29.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.621+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52f0088cc0 con 0x7f53080b4fe0 2023-11-24T08:11:29.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.622+0000 7f5307fff640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f52e806b080 0x7f52e806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:29.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.622+0000 7f5307fff640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f52e806b080 0x7f52e806d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52f8036000 tx=0x7f52f802e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:29.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.626+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f52f008f080 con 0x7f53080b4fe0 2023-11-24T08:11:29.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:29.821+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f53080cd6c0 con 0x7f53080b4fe0 2023-11-24T08:11:30.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.555+0000 7f5304ff9640 1 -- 172.21.15.106:0/67363902 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f52f0018020 con 0x7f53080b4fe0 2023-11-24T08:11:30.557 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2023-11-24T08:11:30.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.558+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f52e806b080 msgr2=0x7f52e806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:30.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.558+0000 7f530e5c5640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f52e806b080 0x7f52e806d540 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f52f8036000 tx=0x7f52f802e000 comp rx=0 tx=0).stop 2023-11-24T08:11:30.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.558+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 msgr2=0x7f53080cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:30.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.558+0000 7f530e5c5640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 0x7f53080cda20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f52f000c4d0 tx=0x7f52f000c990 comp rx=0 tx=0).stop 2023-11-24T08:11:30.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 shutdown_connections 2023-11-24T08:11:30.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f53081461d0 0x7f53080a8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53080b4fe0 0x7f53080cda20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f52e806b080 0x7f52e806d540 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 --2- 172.21.15.106:0/67363902 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f53080b3a60 0x7f53081412a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 >> 172.21.15.106:0/67363902 conn(0x7f5308061340 msgr2=0x7f530815d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:30.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 shutdown_connections 2023-11-24T08:11:30.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.559+0000 7f530e5c5640 1 -- 172.21.15.106:0/67363902 wait complete. 2023-11-24T08:11:30.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:11:30.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.785+0000 7f49d3836640 1 Processor -- start 2023-11-24T08:11:30.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.786+0000 7f49d3836640 1 -- start start 2023-11-24T08:11:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d3836640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc15e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d3836640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49cc0c49f0 0x7f49cc06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d3836640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc15ec90 con 0x7f49cc06dc40 2023-11-24T08:11:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d3836640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc06e0f0 con 0x7f49cc0c49f0 2023-11-24T08:11:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d3836640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0cd530 con 0x7f49cc072160 2023-11-24T08:11:30.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d0dd1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49cc0c49f0 0x7f49cc06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d0dd1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f49cc0c49f0 0x7f49cc06d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43542/0 (socket says 172.21.15.106:43542) 2023-11-24T08:11:30.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.787+0000 7f49d0dd1640 1 -- 172.21.15.106:0/3599398821 learned_addr learned my addr 172.21.15.106:0/3599398821 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.788+0000 7f49d15d2640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc15e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.788+0000 7f49d0dd1640 1 -- 172.21.15.106:0/3599398821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 msgr2=0x7f49cc15e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.788+0000 7f49d0dd1640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc15e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.788+0000 7f49d0dd1640 1 -- 172.21.15.106:0/3599398821 >> v1:172.21.15.156:6789/0 conn(0x7f49cc06dc40 legacy=0x7f49cc0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.788+0000 7f49d0dd1640 1 -- 172.21.15.106:0/3599398821 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0c4ed0 con 0x7f49cc0c49f0 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.788+0000 7f49d15d2640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:30.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.789+0000 7f49d0dd1640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc0c49f0 0x7f49cc06d700 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f49c800bb10 tx=0x7f49c802e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7fd765a22d7f3b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.789+0000 7f49bffff640 1 -- 172.21.15.106:0/3599398821 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c802ee70 con 0x7f49cc0c49f0 2023-11-24T08:11:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.789+0000 7f49bffff640 1 -- 172.21.15.106:0/3599398821 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c8036440 con 0x7f49cc0c49f0 2023-11-24T08:11:30.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.789+0000 7f49bffff640 1 -- 172.21.15.106:0/3599398821 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c80367d0 con 0x7f49cc0c49f0 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 -- 172.21.15.106:0/3599398821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc0c49f0 msgr2=0x7f49cc06d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc0c49f0 0x7f49cc06d700 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f49c800bb10 tx=0x7f49c802e0c0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 -- 172.21.15.106:0/3599398821 shutdown_connections 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc0c49f0 0x7f49cc06d700 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 --2- 172.21.15.106:0/3599398821 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc15e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 -- 172.21.15.106:0/3599398821 >> 172.21.15.106:0/3599398821 conn(0x7f49cc0b0c50 msgr2=0x7f49cc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:30.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 -- 172.21.15.106:0/3599398821 shutdown_connections 2023-11-24T08:11:30.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.790+0000 7f49d3836640 1 -- 172.21.15.106:0/3599398821 wait complete. 2023-11-24T08:11:30.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.792+0000 7f49d3836640 1 Processor -- start 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.792+0000 7f49d3836640 1 -- start start 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d3836640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 0x7f49cc0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d3836640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc0cc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d3836640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49cc0c49f0 0x7f49cc152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d3836640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0caba0 con 0x7f49cc06dc40 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d3836640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0c6c80 con 0x7f49cc0c49f0 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d3836640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0c6f60 con 0x7f49cc072160 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d15d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 0x7f49cc0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:30.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d15d2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 0x7f49cc0c62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43544/0 (socket says 172.21.15.106:43544) 2023-11-24T08:11:30.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.793+0000 7f49d15d2640 1 -- 172.21.15.106:0/3780521652 learned_addr learned my addr 172.21.15.106:0/3780521652 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:30.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d1dd3640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49cc0c49f0 0x7f49cc152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:30.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d15d2640 1 -- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 msgr2=0x7f49cc0cc450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:30.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d15d2640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc0cc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d15d2640 1 -- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49cc0c49f0 msgr2=0x7f49cc152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:30.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d15d2640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49cc0c49f0 0x7f49cc152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:30.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d15d2640 1 -- 172.21.15.106:0/3780521652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c800b770 con 0x7f49cc06dc40 2023-11-24T08:11:30.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49d15d2640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 0x7f49cc0c62e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c7f0 tx=0x7f49c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:30.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c0007a30 con 0x7f49cc06dc40 2023-11-24T08:11:30.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.794+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c0007b90 con 0x7f49cc06dc40 2023-11-24T08:11:30.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.795+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c000ce40 con 0x7f49cc06dc40 2023-11-24T08:11:30.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.795+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc0ca550 con 0x7f49cc06dc40 2023-11-24T08:11:30.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.795+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc0c8560 con 0x7f49cc06dc40 2023-11-24T08:11:30.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.796+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc04e8b0 con 0x7f49cc06dc40 2023-11-24T08:11:30.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.798+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f49c0024480 con 0x7f49cc06dc40 2023-11-24T08:11:30.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.799+0000 7f49bdffb640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006b080 0x7f49b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:30.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.799+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f49c0089f80 con 0x7f49cc06dc40 2023-11-24T08:11:30.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.800+0000 7f49d0dd1640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006b080 0x7f49b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:30.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.800+0000 7f49d0dd1640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006b080 0x7f49b006d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49c8038000 tx=0x7f49c8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:30.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.803+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f49c0054df0 con 0x7f49cc06dc40 2023-11-24T08:11:30.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:30.996+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f49cc06cde0 con 0x7f49cc06dc40 2023-11-24T08:11:31.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.571+0000 7f49bdffb640 1 -- 172.21.15.106:0/3780521652 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v60) v1 ==== 170+0+0 (secure 0 0 0) 0x7f49c001f020 con 0x7f49cc06dc40 2023-11-24T08:11:31.573 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2023-11-24T08:11:31.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.576+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006b080 msgr2=0x7f49b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:31.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.576+0000 7f49d3836640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006b080 0x7f49b006d540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49c8038000 tx=0x7f49c8030000 comp rx=0 tx=0).stop 2023-11-24T08:11:31.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.576+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 msgr2=0x7f49cc0c62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:31.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.576+0000 7f49d3836640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 0x7f49cc0c62e0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f49c000c7f0 tx=0x7f49c000cbf0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 shutdown_connections 2023-11-24T08:11:31.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49cc0c49f0 0x7f49cc152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f49b006b080 0x7f49b006d540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f49cc072160 0x7f49cc0cc450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 --2- 172.21.15.106:0/3780521652 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f49cc06dc40 0x7f49cc0c62e0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 >> 172.21.15.106:0/3780521652 conn(0x7f49cc0b0c50 msgr2=0x7f49cc15e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:31.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 shutdown_connections 2023-11-24T08:11:31.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.577+0000 7f49d3836640 1 -- 172.21.15.106:0/3780521652 wait complete. 2023-11-24T08:11:31.616 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:11:31.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.802+0000 7f74e123d640 1 Processor -- start 2023-11-24T08:11:31.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.802+0000 7f74e123d640 1 -- start start 2023-11-24T08:11:31.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74e123d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74dc15d7f0 0x7f74dc0c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74e123d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0cabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74e123d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74dc15e400 con 0x7f74dc15dbd0 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74e123d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74dc0ce880 con 0x7f74dc15d7f0 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74e123d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc0c5ed0 con 0x7f74dc0c4220 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74dad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f74dc15d7f0 0x7f74dc0c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74db59e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f74dc15dbd0 0x7f74dc0cb0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40948/0 (socket says 172.21.15.106:40948) 2023-11-24T08:11:31.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.804+0000 7f74db59e640 1 -- 172.21.15.106:0/3547609571 learned_addr learned my addr 172.21.15.106:0/3547609571 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:31.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74da59c640 1 --2- 172.21.15.106:0/3547609571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0cabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:31.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74d9d9b640 1 -- 172.21.15.106:0/3547609571 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 732464689 0 0) 0x7f74dc15e400 con 0x7f74dc15dbd0 2023-11-24T08:11:31.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74d9d9b640 1 -- 172.21.15.106:0/3547609571 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c8003520 con 0x7f74dc15dbd0 2023-11-24T08:11:31.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74dad9d640 1 -- 172.21.15.106:0/3547609571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 msgr2=0x7f74dc0cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:31.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74dad9d640 1 --2- 172.21.15.106:0/3547609571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0cabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74dad9d640 1 -- 172.21.15.106:0/3547609571 >> v1:172.21.15.106:6789/0 conn(0x7f74dc15dbd0 legacy=0x7f74dc0cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:31.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.805+0000 7f74dad9d640 1 -- 172.21.15.106:0/3547609571 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc0c46e0 con 0x7f74dc15d7f0 2023-11-24T08:11:31.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.806+0000 7f74dad9d640 1 --2- 172.21.15.106:0/3547609571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0c5990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009d90 tx=0x7f74cc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b022eddacbf7e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.806+0000 7f74d9d9b640 1 -- 172.21.15.106:0/3547609571 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc02cc50 con 0x7f74dc15d7f0 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.806+0000 7f74d9d9b640 1 -- 172.21.15.106:0/3547609571 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc02cdb0 con 0x7f74dc15d7f0 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.806+0000 7f74d9d9b640 1 -- 172.21.15.106:0/3547609571 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc034660 con 0x7f74dc15d7f0 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 -- 172.21.15.106:0/3547609571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15d7f0 msgr2=0x7f74dc0c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 --2- 172.21.15.106:0/3547609571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0c5990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f74cc009d90 tx=0x7f74cc02bea0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 -- 172.21.15.106:0/3547609571 shutdown_connections 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 --2- 172.21.15.106:0/3547609571 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0cabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 --2- 172.21.15.106:0/3547609571 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0c5990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 -- 172.21.15.106:0/3547609571 >> 172.21.15.106:0/3547609571 conn(0x7f74dc0b0c50 msgr2=0x7f74dc0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:31.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 -- 172.21.15.106:0/3547609571 shutdown_connections 2023-11-24T08:11:31.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.807+0000 7f74e123d640 1 -- 172.21.15.106:0/3547609571 wait complete. 2023-11-24T08:11:31.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.809+0000 7f74e123d640 1 Processor -- start 2023-11-24T08:11:31.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.809+0000 7f74e123d640 1 -- start start 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74e123d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74e123d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74e123d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 0x7f74dc1f4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74e123d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc0b2de0 con 0x7f74dc15dbd0 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74e123d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc154500 con 0x7f74dc15d7f0 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74e123d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc0c8c10 con 0x7f74dc0c4220 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74db59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 0x7f74dc1f4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74db59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 0x7f74dc1f4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43556/0 (socket says 172.21.15.106:43556) 2023-11-24T08:11:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.810+0000 7f74db59e640 1 -- 172.21.15.106:0/3545114623 learned_addr learned my addr 172.21.15.106:0/3545114623 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74db59e640 1 -- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 msgr2=0x7f74dc0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74db59e640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74db59e640 1 -- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc15d7f0 msgr2=0x7f74dc0b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74da59c640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74db59e640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74db59e640 1 -- 172.21.15.106:0/3545114623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc009a70 con 0x7f74dc15dbd0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74db59e640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 0x7f74dc1f4d10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c430 tx=0x7f74d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4005ce0 con 0x7f74dc15dbd0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d4005e40 con 0x7f74dc15dbd0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc0c8ef0 con 0x7f74dc15dbd0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4007310 con 0x7f74dc15dbd0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.811+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc0b26e0 con 0x7f74dc15dbd0 2023-11-24T08:11:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.812+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74dc04e8b0 con 0x7f74dc15dbd0 2023-11-24T08:11:31.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.814+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f74d4007470 con 0x7f74dc15dbd0 2023-11-24T08:11:31.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.815+0000 7f74d37fe640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f74bc06b080 0x7f74bc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:31.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.815+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f74d40894e0 con 0x7f74dc15dbd0 2023-11-24T08:11:31.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.815+0000 7f74dad9d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f74bc06b080 0x7f74bc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:31.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.816+0000 7f74dad9d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f74bc06b080 0x7f74bc06d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74cc036000 tx=0x7f74cc02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:31.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:31.818+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f74d4054270 con 0x7f74dc15dbd0 2023-11-24T08:11:32.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.051+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f74dc166810 con 0x7f74dc15dbd0 2023-11-24T08:11:32.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.586+0000 7f74d37fe640 1 -- 172.21.15.106:0/3545114623 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v30) v1 ==== 172+0+0 (secure 0 0 0) 0x7f74d4057890 con 0x7f74dc15dbd0 2023-11-24T08:11:32.592 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 8 and data pool 9 2023-11-24T08:11:32.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f74bc06b080 msgr2=0x7f74bc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:32.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f74bc06b080 0x7f74bc06d540 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f74cc036000 tx=0x7f74cc02e000 comp rx=0 tx=0).stop 2023-11-24T08:11:32.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 msgr2=0x7f74dc1f4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:32.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 0x7f74dc1f4d10 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c430 tx=0x7f74d400c8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 shutdown_connections 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f74dc15dbd0 0x7f74dc1f4d10 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74dc15d7f0 0x7f74dc0b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f74bc06b080 0x7f74bc06d540 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 --2- 172.21.15.106:0/3545114623 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f74dc0c4220 0x7f74dc0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 >> 172.21.15.106:0/3545114623 conn(0x7f74dc0b0c50 msgr2=0x7f74dc1589a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 shutdown_connections 2023-11-24T08:11:32.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.594+0000 7f74e123d640 1 -- 172.21.15.106:0/3545114623 wait complete. 2023-11-24T08:11:32.627 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:32.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.799+0000 7f0a30f8e640 1 Processor -- start 2023-11-24T08:11:32.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.799+0000 7f0a30f8e640 1 -- start start 2023-11-24T08:11:32.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a30f8e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0a2c15c7f0 0x7f0a2c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a30f8e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c0cf450 0x7f0a2c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a30f8e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0a2c0cdfb0 0x7f0a2c0ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a30f8e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2c156010 con 0x7f0a2c0cdfb0 2023-11-24T08:11:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a30f8e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a2c06ce30 con 0x7f0a2c15c7f0 2023-11-24T08:11:32.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a30f8e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cfe10 con 0x7f0a2c0cf450 2023-11-24T08:11:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0a2c0cdfb0 0x7f0a2c0ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0a2c0cdfb0 0x7f0a2c0ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43558/0 (socket says 172.21.15.106:43558) 2023-11-24T08:11:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2ad9d640 1 -- 172.21.15.106:0/2346267388 learned_addr learned my addr 172.21.15.106:0/2346267388 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a29d9b640 1 --2- 172.21.15.106:0/2346267388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c0cf450 0x7f0a2c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2ad9d640 1 -- 172.21.15.106:0/2346267388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c0cf450 msgr2=0x7f0a2c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:32.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2a59c640 1 --2- 172.21.15.106:0/2346267388 >> v2:172.21.15.156:3300/0 conn(0x7f0a2c15c7f0 0x7f0a2c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2ad9d640 1 --2- 172.21.15.106:0/2346267388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c0cf450 0x7f0a2c0cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.801+0000 7f0a2ad9d640 1 -- 172.21.15.106:0/2346267388 >> v2:172.21.15.156:3300/0 conn(0x7f0a2c15c7f0 msgr2=0x7f0a2c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.802+0000 7f0a2ad9d640 1 --2- 172.21.15.106:0/2346267388 >> v2:172.21.15.156:3300/0 conn(0x7f0a2c15c7f0 0x7f0a2c06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.802+0000 7f0a2ad9d640 1 -- 172.21.15.106:0/2346267388 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c0cece0 con 0x7f0a2c0cdfb0 2023-11-24T08:11:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.803+0000 7f0a2ad9d640 1 --2- 172.21.15.106:0/2346267388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c0ce440 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0d0130 tx=0x7f0a2402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e79f1e5360acaf1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.803+0000 7f0a2959a640 1 -- 172.21.15.106:0/2346267388 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24031870 con 0x7f0a2c0cdfb0 2023-11-24T08:11:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.803+0000 7f0a2959a640 1 -- 172.21.15.106:0/2346267388 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a240319d0 con 0x7f0a2c0cdfb0 2023-11-24T08:11:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.803+0000 7f0a2959a640 1 -- 172.21.15.106:0/2346267388 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24031d60 con 0x7f0a2c0cdfb0 2023-11-24T08:11:32.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.803+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2346267388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cdfb0 msgr2=0x7f0a2c0ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.803+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2346267388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c0ce440 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c0d0130 tx=0x7f0a2402b910 comp rx=0 tx=0).stop 2023-11-24T08:11:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2346267388 shutdown_connections 2023-11-24T08:11:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2346267388 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c0ce440 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2346267388 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c0cf450 0x7f0a2c0cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2346267388 >> v2:172.21.15.156:3300/0 conn(0x7f0a2c15c7f0 0x7f0a2c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2346267388 >> 172.21.15.106:0/2346267388 conn(0x7f0a2c06a920 msgr2=0x7f0a2c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2346267388 shutdown_connections 2023-11-24T08:11:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.804+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2346267388 wait complete. 2023-11-24T08:11:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.805+0000 7f0a30f8e640 1 Processor -- start 2023-11-24T08:11:32.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 -- start start 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c1475e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 0x7f0a2c1f3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c15c7f0 0x7f0a2c0b22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c150b70 con 0x7f0a2c0cf450 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c150e50 con 0x7f0a2c0cdfb0 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.806+0000 7f0a30f8e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0b2b60 con 0x7f0a2c15c7f0 2023-11-24T08:11:32.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 0x7f0a2c1f3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 0x7f0a2c1f3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43560/0 (socket says 172.21.15.106:43560) 2023-11-24T08:11:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 -- 172.21.15.106:0/2334238555 learned_addr learned my addr 172.21.15.106:0/2334238555 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a2a59c640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c1475e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a2ad9d640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c15c7f0 0x7f0a2c0b22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 -- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c15c7f0 msgr2=0x7f0a2c0b22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c15c7f0 0x7f0a2c0b22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 -- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a2c0cdfb0 msgr2=0x7f0a2c1475e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c1475e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 -- 172.21.15.106:0/2334238555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240094d0 con 0x7f0a2c0cf450 2023-11-24T08:11:32.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a2a59c640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.807+0000 7f0a29d9b640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 0x7f0a2c1f3500 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c7f0 tx=0x7f0a1400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.808+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14004220 con 0x7f0a2c0cf450 2023-11-24T08:11:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.808+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a14004380 con 0x7f0a2c0cf450 2023-11-24T08:11:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.808+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2c0b2e40 con 0x7f0a2c0cf450 2023-11-24T08:11:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.808+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a14007860 con 0x7f0a2c0cf450 2023-11-24T08:11:32.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.808+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2c151130 con 0x7f0a2c0cf450 2023-11-24T08:11:32.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.809+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2c067470 con 0x7f0a2c0cf450 2023-11-24T08:11:32.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.810+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f0a14007bd0 con 0x7f0a2c0cf450 2023-11-24T08:11:32.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.811+0000 7f0a22ffd640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0a0c06b080 0x7f0a0c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:32.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.811+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a14089be0 con 0x7f0a2c0cf450 2023-11-24T08:11:32.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.811+0000 7f0a2a59c640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0a0c06b080 0x7f0a0c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:32.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.811+0000 7f0a2a59c640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0a0c06b080 0x7f0a0c06d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c14f4b0 tx=0x7f0a1c007420 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:32.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:32.815+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0a140546f0 con 0x7f0a2c0cf450 2023-11-24T08:11:33.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.005+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a2c15ea70 con 0x7f0a2c0cf450 2023-11-24T08:11:33.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.007+0000 7f0a22ffd640 1 -- 172.21.15.106:0/2334238555 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16626 (secure 0 0 0) 0x7f0a1401e070 con 0x7f0a2c0cf450 2023-11-24T08:11:33.009 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:33.009 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":61,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:32.052746+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:33.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.009+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0a0c06b080 msgr2=0x7f0a0c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.009+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0a0c06b080 0x7f0a0c06d540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f0a2c14f4b0 tx=0x7f0a1c007420 comp rx=0 tx=0).stop 2023-11-24T08:11:33.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 msgr2=0x7f0a2c1f3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 0x7f0a2c1f3500 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0a1400c7f0 tx=0x7f0a1400ccb0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 shutdown_connections 2023-11-24T08:11:33.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0a0c06b080 0x7f0a0c06d540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0a2c15c7f0 0x7f0a2c0b22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0a2c0cf450 0x7f0a2c1f3500 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 --2- 172.21.15.106:0/2334238555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a2c0cdfb0 0x7f0a2c1475e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 >> 172.21.15.106:0/2334238555 conn(0x7f0a2c06a920 msgr2=0x7f0a2c0b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:33.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 shutdown_connections 2023-11-24T08:11:33.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.010+0000 7f0a30f8e640 1 -- 172.21.15.106:0/2334238555 wait complete. 2023-11-24T08:11:33.031 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:33.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.238+0000 7f50c5f1d640 1 Processor -- start 2023-11-24T08:11:33.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.239+0000 7f50c5f1d640 1 -- start start 2023-11-24T08:11:33.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50c5f1d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f50c015c7f0 0x7f50c006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50c5f1d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f50c00cf450 0x7f50c00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50c5f1d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c00cdfb0 0x7f50c00ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50c5f1d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c00cfe40 con 0x7f50c015c7f0 2023-11-24T08:11:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50c5f1d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c0156010 con 0x7f50c00cf450 2023-11-24T08:11:33.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50c5f1d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c006ce30 con 0x7f50c00cdfb0 2023-11-24T08:11:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.240+0000 7f50bf7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f50c015c7f0 0x7f50c006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f50c015c7f0 0x7f50c006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43568/0 (socket says 172.21.15.106:43568) 2023-11-24T08:11:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 -- 172.21.15.106:0/2979614686 learned_addr learned my addr 172.21.15.106:0/2979614686 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50b7fff640 1 --2- 172.21.15.106:0/2979614686 >> v2:172.21.15.156:3300/0 conn(0x7f50c00cf450 0x7f50c00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bffff640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c00cdfb0 0x7f50c00ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 -- 172.21.15.106:0/2979614686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c00cdfb0 msgr2=0x7f50c00ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c00cdfb0 0x7f50c00ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 -- 172.21.15.106:0/2979614686 >> v2:172.21.15.156:3300/0 conn(0x7f50c00cf450 msgr2=0x7f50c00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 --2- 172.21.15.106:0/2979614686 >> v2:172.21.15.156:3300/0 conn(0x7f50c00cf450 0x7f50c00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bf7fe640 1 -- 172.21.15.106:0/2979614686 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00cece0 con 0x7f50c015c7f0 2023-11-24T08:11:33.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50bffff640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c00cdfb0 0x7f50c00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:33.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.241+0000 7f50b7fff640 1 --2- 172.21.15.106:0/2979614686 >> v2:172.21.15.156:3300/0 conn(0x7f50c00cf450 0x7f50c00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.242+0000 7f50bf7fe640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c015c7f0 0x7f50c006db20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f50c0070410 tx=0x7f50b002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9713ac71bcf09577 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50beffd640 1 -- 172.21.15.106:0/2979614686 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0031860 con 0x7f50c015c7f0 2023-11-24T08:11:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50beffd640 1 -- 172.21.15.106:0/2979614686 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00319c0 con 0x7f50c015c7f0 2023-11-24T08:11:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50beffd640 1 -- 172.21.15.106:0/2979614686 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0031d50 con 0x7f50c015c7f0 2023-11-24T08:11:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 -- 172.21.15.106:0/2979614686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c015c7f0 msgr2=0x7f50c006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c015c7f0 0x7f50c006db20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f50c0070410 tx=0x7f50b002b900 comp rx=0 tx=0).stop 2023-11-24T08:11:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 -- 172.21.15.106:0/2979614686 shutdown_connections 2023-11-24T08:11:33.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c00cdfb0 0x7f50c00ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/2979614686 >> v2:172.21.15.156:3300/0 conn(0x7f50c00cf450 0x7f50c00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/2979614686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c015c7f0 0x7f50c006db20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.243+0000 7f50c5f1d640 1 -- 172.21.15.106:0/2979614686 >> 172.21.15.106:0/2979614686 conn(0x7f50c006a920 msgr2=0x7f50c006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.244+0000 7f50c5f1d640 1 -- 172.21.15.106:0/2979614686 shutdown_connections 2023-11-24T08:11:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.244+0000 7f50c5f1d640 1 -- 172.21.15.106:0/2979614686 wait complete. 2023-11-24T08:11:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.245+0000 7f50c5f1d640 1 Processor -- start 2023-11-24T08:11:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.245+0000 7f50c5f1d640 1 -- start start 2023-11-24T08:11:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50c5f1d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 0x7f50c0157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50c5f1d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c00cf450 0x7f50c014d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50c5f1d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c015c7f0 0x7f50c00ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50c5f1d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0156b10 con 0x7f50c00cdfb0 2023-11-24T08:11:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50c5f1d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0156db0 con 0x7f50c00cf450 2023-11-24T08:11:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50c5f1d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c0157090 con 0x7f50c015c7f0 2023-11-24T08:11:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.246+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 0x7f50c0157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 0x7f50c0157e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43572/0 (socket says 172.21.15.106:43572) 2023-11-24T08:11:33.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 -- 172.21.15.106:0/3886218420 learned_addr learned my addr 172.21.15.106:0/3886218420 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 -- 172.21.15.106:0/3886218420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c015c7f0 msgr2=0x7f50c00ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c015c7f0 0x7f50c00ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 -- 172.21.15.106:0/3886218420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c00cf450 msgr2=0x7f50c014d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c00cf450 0x7f50c014d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 -- 172.21.15.106:0/3886218420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b00094d0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bf7fe640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 0x7f50c0157e50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50b002b940 tx=0x7f50b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0031850 con 0x7f50c00cdfb0 2023-11-24T08:11:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b00443f0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b0044780 con 0x7f50c00cdfb0 2023-11-24T08:11:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.247+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c0157370 con 0x7f50c00cdfb0 2023-11-24T08:11:33.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.248+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c0157930 con 0x7f50c00cdfb0 2023-11-24T08:11:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.249+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c01571d0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.250+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f50b004d450 con 0x7f50c00cdfb0 2023-11-24T08:11:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.250+0000 7f50bcff9640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f50a006b080 0x7f50a006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.251+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50b00ac2a0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.251+0000 7f50b7fff640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f50a006b080 0x7f50a006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.252+0000 7f50b7fff640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f50a006b080 0x7f50a006d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006fd0 tx=0x7f50a8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:33.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.255+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f50c01571d0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.438+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50c01571d0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.441+0000 7f50bcff9640 1 -- 172.21.15.106:0/3886218420 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+16626 (secure 0 0 0) 0x7f50c01571d0 con 0x7f50c00cdfb0 2023-11-24T08:11:33.443 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:33.443 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":61,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:32.052746+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:33.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.443+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f50a006b080 msgr2=0x7f50a006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.443+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f50a006b080 0x7f50a006d540 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f50a8006fd0 tx=0x7f50a8008040 comp rx=0 tx=0).stop 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.443+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 msgr2=0x7f50c0157e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.443+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 0x7f50c0157e50 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f50b002b940 tx=0x7f50b00046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 shutdown_connections 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f50a006b080 0x7f50a006d540 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50c015c7f0 0x7f50c00ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50c00cf450 0x7f50c014d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 --2- 172.21.15.106:0/3886218420 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f50c00cdfb0 0x7f50c0157e50 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 >> 172.21.15.106:0/3886218420 conn(0x7f50c006a920 msgr2=0x7f50c00afa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 shutdown_connections 2023-11-24T08:11:33.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.444+0000 7f50c5f1d640 1 -- 172.21.15.106:0/3886218420 wait complete. 2023-11-24T08:11:33.463 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:11:33.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.671+0000 7f93b551d640 1 Processor -- start 2023-11-24T08:11:33.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.672+0000 7f93b551d640 1 -- start start 2023-11-24T08:11:33.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93b551d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93b0072160 0x7f93b015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93b551d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f93b006de10 0x7f93b00705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93b551d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b015ec90 con 0x7f93b00c49f0 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93b551d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b00cd530 con 0x7f93b0072160 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93b551d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b00c4ed0 con 0x7f93b006de10 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93aeffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93b0072160 0x7f93b015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93aeffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f93b0072160 0x7f93b015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43580/0 (socket says 172.21.15.106:43580) 2023-11-24T08:11:33.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93aeffd640 1 -- 172.21.15.106:0/4053172187 learned_addr learned my addr 172.21.15.106:0/4053172187 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.674+0000 7f93af7fe640 1 --2- 172.21.15.106:0/4053172187 >> v2:172.21.15.156:3300/0 conn(0x7f93b006de10 0x7f93b00705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.675+0000 7f93aeffd640 1 -- 172.21.15.106:0/4053172187 >> v2:172.21.15.156:3300/0 conn(0x7f93b006de10 msgr2=0x7f93b00705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.675+0000 7f93aeffd640 1 --2- 172.21.15.106:0/4053172187 >> v2:172.21.15.156:3300/0 conn(0x7f93b006de10 0x7f93b00705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.675+0000 7f93aeffd640 1 -- 172.21.15.106:0/4053172187 >> v1:172.21.15.156:6789/0 conn(0x7f93b00c49f0 legacy=0x7f93b006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.675+0000 7f93aeffd640 1 -- 172.21.15.106:0/4053172187 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b00c51b0 con 0x7f93b0072160 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.675+0000 7f93af7fe640 1 --2- 172.21.15.106:0/4053172187 >> v2:172.21.15.156:3300/0 conn(0x7f93b006de10 0x7f93b00705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:33.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.676+0000 7f93aeffd640 1 --2- 172.21.15.106:0/4053172187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b0072160 0x7f93b015e750 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f93b006eab0 tx=0x7f93a002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365e6b91885747a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:33.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.676+0000 7f93adffb640 1 -- 172.21.15.106:0/4053172187 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a002ca80 con 0x7f93b0072160 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.676+0000 7f93adffb640 1 -- 172.21.15.106:0/4053172187 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a002cbe0 con 0x7f93b0072160 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.676+0000 7f93adffb640 1 -- 172.21.15.106:0/4053172187 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a00343f0 con 0x7f93b0072160 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 -- 172.21.15.106:0/4053172187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b0072160 msgr2=0x7f93b015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 --2- 172.21.15.106:0/4053172187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b0072160 0x7f93b015e750 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f93b006eab0 tx=0x7f93a002bee0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 -- 172.21.15.106:0/4053172187 shutdown_connections 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 --2- 172.21.15.106:0/4053172187 >> v2:172.21.15.156:3300/0 conn(0x7f93b006de10 0x7f93b00705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 --2- 172.21.15.106:0/4053172187 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b0072160 0x7f93b015e750 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 -- 172.21.15.106:0/4053172187 >> 172.21.15.106:0/4053172187 conn(0x7f93b00b0c50 msgr2=0x7f93b00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 -- 172.21.15.106:0/4053172187 shutdown_connections 2023-11-24T08:11:33.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.677+0000 7f93b551d640 1 -- 172.21.15.106:0/4053172187 wait complete. 2023-11-24T08:11:33.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.679+0000 7f93b551d640 1 Processor -- start 2023-11-24T08:11:33.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.679+0000 7f93b551d640 1 -- start start 2023-11-24T08:11:33.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93b551d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93b006de10 0x7f93b0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93b551d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93b0072160 0x7f93b00c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93b551d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 0x7f93b015c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93b551d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b015bee0 con 0x7f93b00c49f0 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93b551d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b015b360 con 0x7f93b006de10 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93b551d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b015b640 con 0x7f93b0072160 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 0x7f93b015c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93af7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 0x7f93b015c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43582/0 (socket says 172.21.15.106:43582) 2023-11-24T08:11:33.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93af7fe640 1 -- 172.21.15.106:0/1485216784 learned_addr learned my addr 172.21.15.106:0/1485216784 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:33.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93af7fe640 1 -- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93b0072160 msgr2=0x7f93b00c8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93aeffd640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93b006de10 0x7f93b0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93ae7fc640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93b0072160 0x7f93b00c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.680+0000 7f93af7fe640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93b0072160 0x7f93b00c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93af7fe640 1 -- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93b006de10 msgr2=0x7f93b0156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93af7fe640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93b006de10 0x7f93b0156c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93af7fe640 1 -- 172.21.15.106:0/1485216784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009b30 con 0x7f93b00c49f0 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93aeffd640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93b006de10 0x7f93b0156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93af7fe640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 0x7f93b015c520 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f93a800c7f0 tx=0x7f93a800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8004220 con 0x7f93b00c49f0 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a80043a0 con 0x7f93b00c49f0 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b015b920 con 0x7f93b00c49f0 2023-11-24T08:11:33.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a8007860 con 0x7f93b00c49f0 2023-11-24T08:11:33.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.681+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b0157160 con 0x7f93b00c49f0 2023-11-24T08:11:33.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.682+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b004e8b0 con 0x7f93b00c49f0 2023-11-24T08:11:33.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.684+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f93a8007bd0 con 0x7f93b00c49f0 2023-11-24T08:11:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.685+0000 7f93a77fe640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f939006b080 0x7f939006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:33.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.685+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93a8089ca0 con 0x7f93b00c49f0 2023-11-24T08:11:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.685+0000 7f93aeffd640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f939006b080 0x7f939006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:33.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.686+0000 7f93aeffd640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f939006b080 0x7f939006d540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f93a0035000 tx=0x7f93a0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:33.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.690+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f93a80547b0 con 0x7f93b00c49f0 2023-11-24T08:11:33.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:33.912+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f93b009cec0 con 0x7f93b00c49f0 2023-11-24T08:11:34.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.594+0000 7f93a77fe640 1 -- 172.21.15.106:0/1485216784 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f93a801e020 con 0x7f93b00c49f0 2023-11-24T08:11:34.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f939006b080 msgr2=0x7f939006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:34.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f939006b080 0x7f939006d540 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f93a0035000 tx=0x7f93a0030000 comp rx=0 tx=0).stop 2023-11-24T08:11:34.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 msgr2=0x7f93b015c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:34.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 0x7f93b015c520 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f93a800c7f0 tx=0x7f93a800ccb0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 shutdown_connections 2023-11-24T08:11:34.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93b00c49f0 0x7f93b015c520 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f939006b080 0x7f939006d540 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93b0072160 0x7f93b00c8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 --2- 172.21.15.106:0/1485216784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93b006de10 0x7f93b0156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 >> 172.21.15.106:0/1485216784 conn(0x7f93b00b0c50 msgr2=0x7f93b00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:34.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 shutdown_connections 2023-11-24T08:11:34.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.598+0000 7f93b551d640 1 -- 172.21.15.106:0/1485216784 wait complete. 2023-11-24T08:11:34.619 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:11:34.619 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:11:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.830+0000 7f43eda51640 1 Processor -- start 2023-11-24T08:11:34.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.831+0000 7f43eda51640 1 -- start start 2023-11-24T08:11:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.832+0000 7f43eda51640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43e813ea00 0x7f43e80cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.832+0000 7f43eda51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e80cf320 0x7f43e80700b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.832+0000 7f43eda51640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e815c630 con 0x7f43e813e3e0 2023-11-24T08:11:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.832+0000 7f43eda51640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e813e7c0 con 0x7f43e813ea00 2023-11-24T08:11:34.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.832+0000 7f43eda51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e80cf7d0 con 0x7f43e80cf320 2023-11-24T08:11:34.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.833+0000 7f43de7fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43e813ea00 0x7f43e80cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:34.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.833+0000 7f43e77fe640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e80cf320 0x7f43e80700b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:34.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.833+0000 7f43de7fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43e813ea00 0x7f43e80cede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:34982/0 (socket says 172.21.15.106:34982) 2023-11-24T08:11:34.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.833+0000 7f43de7fc640 1 -- 172.21.15.106:0/3929123476 learned_addr learned my addr 172.21.15.106:0/3929123476 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:34.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.834+0000 7f43de7fc640 1 -- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e80cf320 msgr2=0x7f43e80700b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.834+0000 7f43de7fc640 1 --2- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e80cf320 0x7f43e80700b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.834+0000 7f43de7fc640 1 -- 172.21.15.106:0/3929123476 >> v1:172.21.15.156:6789/0 conn(0x7f43e813e3e0 legacy=0x7f43e80ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.834+0000 7f43de7fc640 1 -- 172.21.15.106:0/3929123476 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e815ba20 con 0x7f43e813ea00 2023-11-24T08:11:34.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.835+0000 7f43de7fc640 1 --2- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e813ea00 0x7f43e80cede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43d00098a0 tx=0x7f43d002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17e15837b7488256 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43e67fc640 1 -- 172.21.15.106:0/3929123476 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0031640 con 0x7f43e813ea00 2023-11-24T08:11:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43e67fc640 1 -- 172.21.15.106:0/3929123476 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43d00317a0 con 0x7f43e813ea00 2023-11-24T08:11:34.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43e67fc640 1 -- 172.21.15.106:0/3929123476 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43d0031b30 con 0x7f43e813ea00 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 -- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e813ea00 msgr2=0x7f43e80cede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 --2- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e813ea00 0x7f43e80cede0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f43d00098a0 tx=0x7f43d002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 -- 172.21.15.106:0/3929123476 shutdown_connections 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 --2- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e80cf320 0x7f43e80700b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 --2- 172.21.15.106:0/3929123476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e813ea00 0x7f43e80cede0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 -- 172.21.15.106:0/3929123476 >> 172.21.15.106:0/3929123476 conn(0x7f43e80b2e40 msgr2=0x7f43e80b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:34.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 -- 172.21.15.106:0/3929123476 shutdown_connections 2023-11-24T08:11:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.836+0000 7f43eda51640 1 -- 172.21.15.106:0/3929123476 wait complete. 2023-11-24T08:11:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.838+0000 7f43eda51640 1 Processor -- start 2023-11-24T08:11:34.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.838+0000 7f43eda51640 1 -- start start 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43eda51640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e80cf320 0x7f43e806d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43eda51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e813e3e0 0x7f43e806ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43eda51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 0x7f43e8153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43eda51640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e8154e10 con 0x7f43e813ea00 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43eda51640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e806c100 con 0x7f43e80cf320 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43eda51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e806c3e0 con 0x7f43e813e3e0 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.839+0000 7f43e77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 0x7f43e8153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 0x7f43e8153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43842/0 (socket says 172.21.15.106:43842) 2023-11-24T08:11:34.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 -- 172.21.15.106:0/3780071455 learned_addr learned my addr 172.21.15.106:0/3780071455 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e6ffd640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e80cf320 0x7f43e806d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43de7fc640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e813e3e0 0x7f43e806ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 -- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e813e3e0 msgr2=0x7f43e806ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e813e3e0 0x7f43e806ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 -- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e80cf320 msgr2=0x7f43e806d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:34.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e80cf320 0x7f43e806d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 -- 172.21.15.106:0/3780071455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d00094d0 con 0x7f43e813ea00 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.840+0000 7f43e77fe640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 0x7f43e8153d60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f43e000efc0 tx=0x7f43e000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.841+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e0009ba0 con 0x7f43e813ea00 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.841+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43e000f040 con 0x7f43e813ea00 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.841+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e00146c0 con 0x7f43e813ea00 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.841+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e806c6c0 con 0x7f43e813ea00 2023-11-24T08:11:34.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.841+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e806cc80 con 0x7f43e813ea00 2023-11-24T08:11:34.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.842+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e806ce70 con 0x7f43e813ea00 2023-11-24T08:11:34.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.843+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f43e0014820 con 0x7f43e813ea00 2023-11-24T08:11:34.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.844+0000 7f43e4ff9640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c806b080 0x7f43c806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.844+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43e008a6c0 con 0x7f43e813ea00 2023-11-24T08:11:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.844+0000 7f43e6ffd640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c806b080 0x7f43c806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:34.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.845+0000 7f43e6ffd640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c806b080 0x7f43c806d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f43d8007fd0 tx=0x7f43d8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:34.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:34.848+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43e00552f0 con 0x7f43e813ea00 2023-11-24T08:11:35.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.080+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f43e804e8b0 con 0x7f43e813ea00 2023-11-24T08:11:35.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.604+0000 7f43e4ff9640 1 -- 172.21.15.106:0/3780071455 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f43e0058910 con 0x7f43e813ea00 2023-11-24T08:11:35.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c806b080 msgr2=0x7f43c806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:35.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c806b080 0x7f43c806d540 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f43d8007fd0 tx=0x7f43d8009040 comp rx=0 tx=0).stop 2023-11-24T08:11:35.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 msgr2=0x7f43e8153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:35.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 0x7f43e8153d60 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f43e000efc0 tx=0x7f43e000c490 comp rx=0 tx=0).stop 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 shutdown_connections 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e813ea00 0x7f43e8153d60 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c806b080 0x7f43c806d540 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e813e3e0 0x7f43e806ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.608+0000 7f43eda51640 1 --2- 172.21.15.106:0/3780071455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e80cf320 0x7f43e806d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.609+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 >> 172.21.15.106:0/3780071455 conn(0x7f43e80b2e40 msgr2=0x7f43e80b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.609+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 shutdown_connections 2023-11-24T08:11:35.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.609+0000 7f43eda51640 1 -- 172.21.15.106:0/3780071455 wait complete. 2023-11-24T08:11:35.630 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:11:35.630 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:35.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.836+0000 7f883441c640 1 Processor -- start 2023-11-24T08:11:35.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.836+0000 7f883441c640 1 -- start start 2023-11-24T08:11:35.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f883441c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f882c0cf450 0x7f882c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:35.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f883441c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f882c0cdfb0 0x7f882c0ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:35.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f883441c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f882c0cece0 con 0x7f882c15c7f0 2023-11-24T08:11:35.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f883441c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882c0cfe40 con 0x7f882c0cf450 2023-11-24T08:11:35.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f883441c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f882c156010 con 0x7f882c0cdfb0 2023-11-24T08:11:35.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f88319b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f882c0cf450 0x7f882c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:35.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f88319b7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f882c0cf450 0x7f882c0cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43850/0 (socket says 172.21.15.106:43850) 2023-11-24T08:11:35.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f88319b7640 1 -- 172.21.15.106:0/2917747270 learned_addr learned my addr 172.21.15.106:0/2917747270 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:35.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.838+0000 7f88329b9640 1 --2- 172.21.15.106:0/2917747270 >> v2:172.21.15.156:3300/0 conn(0x7f882c0cdfb0 0x7f882c0ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:35.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.839+0000 7f88319b7640 1 -- 172.21.15.106:0/2917747270 >> v2:172.21.15.156:3300/0 conn(0x7f882c0cdfb0 msgr2=0x7f882c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:35.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.839+0000 7f88319b7640 1 --2- 172.21.15.106:0/2917747270 >> v2:172.21.15.156:3300/0 conn(0x7f882c0cdfb0 0x7f882c0ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.839+0000 7f88319b7640 1 -- 172.21.15.106:0/2917747270 >> v1:172.21.15.156:6789/0 conn(0x7f882c15c7f0 legacy=0x7f882c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:35.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.839+0000 7f88319b7640 1 -- 172.21.15.106:0/2917747270 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c06d040 con 0x7f882c0cf450 2023-11-24T08:11:35.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.840+0000 7f88319b7640 1 --2- 172.21.15.106:0/2917747270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cf450 0x7f882c0cf900 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f882c0d00e0 tx=0x7f881c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864f71687a019705 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:35.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.840+0000 7f88311b6640 1 -- 172.21.15.106:0/2917747270 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c031870 con 0x7f882c0cf450 2023-11-24T08:11:35.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.840+0000 7f88311b6640 1 -- 172.21.15.106:0/2917747270 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881c0319d0 con 0x7f882c0cf450 2023-11-24T08:11:35.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.840+0000 7f88311b6640 1 -- 172.21.15.106:0/2917747270 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881c031d60 con 0x7f882c0cf450 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 -- 172.21.15.106:0/2917747270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cf450 msgr2=0x7f882c0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 --2- 172.21.15.106:0/2917747270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cf450 0x7f882c0cf900 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f882c0d00e0 tx=0x7f881c02b910 comp rx=0 tx=0).stop 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 -- 172.21.15.106:0/2917747270 shutdown_connections 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 --2- 172.21.15.106:0/2917747270 >> v2:172.21.15.156:3300/0 conn(0x7f882c0cdfb0 0x7f882c0ce3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 --2- 172.21.15.106:0/2917747270 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cf450 0x7f882c0cf900 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 -- 172.21.15.106:0/2917747270 >> 172.21.15.106:0/2917747270 conn(0x7f882c06a920 msgr2=0x7f882c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 -- 172.21.15.106:0/2917747270 shutdown_connections 2023-11-24T08:11:35.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.841+0000 7f883441c640 1 -- 172.21.15.106:0/2917747270 wait complete. 2023-11-24T08:11:35.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.843+0000 7f883441c640 1 Processor -- start 2023-11-24T08:11:35.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.843+0000 7f883441c640 1 -- start start 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f883441c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 0x7f882c09cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f883441c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f882c0cf450 0x7f882c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f883441c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f882c15c7f0 0x7f882c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f883441c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c143430 con 0x7f882c0cdfb0 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f883441c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f882c1439b0 con 0x7f882c0cf450 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f883441c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882c15aba0 con 0x7f882c15c7f0 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f88321b8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 0x7f882c09cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f88321b8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 0x7f882c09cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43864/0 (socket says 172.21.15.106:43864) 2023-11-24T08:11:35.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.844+0000 7f88321b8640 1 -- 172.21.15.106:0/1102281992 learned_addr learned my addr 172.21.15.106:0/1102281992 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88329b9640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f882c15c7f0 0x7f882c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:35.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88319b7640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f882c0cf450 0x7f882c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88321b8640 1 -- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f882c15c7f0 msgr2=0x7f882c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88321b8640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f882c15c7f0 0x7f882c158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88321b8640 1 -- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f882c0cf450 msgr2=0x7f882c15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88321b8640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f882c0cf450 0x7f882c15a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88321b8640 1 -- 172.21.15.106:0/1102281992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f881c0094d0 con 0x7f882c0cdfb0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.845+0000 7f88321b8640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 0x7f882c09cbb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f882800b420 tx=0x7f882800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.846+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828011830 con 0x7f882c0cdfb0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.846+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8828011990 con 0x7f882c0cdfb0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.846+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8828011d40 con 0x7f882c0cdfb0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.846+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f882c15ae80 con 0x7f882c0cdfb0 2023-11-24T08:11:35.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.846+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f882c09d0f0 con 0x7f882c0cdfb0 2023-11-24T08:11:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.847+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f882c15b070 con 0x7f882c0cdfb0 2023-11-24T08:11:35.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.848+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f882801b460 con 0x7f882c0cdfb0 2023-11-24T08:11:35.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.849+0000 7f8822ffd640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f881006b080 0x7f881006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:35.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.849+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f88280896c0 con 0x7f882c0cdfb0 2023-11-24T08:11:35.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.849+0000 7f88319b7640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f881006b080 0x7f881006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:35.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.850+0000 7f88319b7640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f881006b080 0x7f881006d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f881c036000 tx=0x7f881c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:35.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:35.853+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f882c15b070 con 0x7f882c0cdfb0 2023-11-24T08:11:36.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.086+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f882c15b070 con 0x7f882c0cdfb0 2023-11-24T08:11:36.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.087+0000 7f8822ffd640 1 -- 172.21.15.106:0/1102281992 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f882c15b070 con 0x7f882c0cdfb0 2023-11-24T08:11:36.089 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:36.090 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:11:32.053291+0000","modified":"2023-11-24T08:11:35.081758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/180703350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":180703350},{"type":"v1","addr":"172.21.15.106:6837","nonce":180703350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:11:36.090 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 33 2023-11-24T08:11:36.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.089+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f881006b080 msgr2=0x7f881006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.089+0000 7f883441c640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f881006b080 0x7f881006d540 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f881c036000 tx=0x7f881c03c040 comp rx=0 tx=0).stop 2023-11-24T08:11:36.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.089+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 msgr2=0x7f882c09cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.089+0000 7f883441c640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 0x7f882c09cbb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f882800b420 tx=0x7f882800b8e0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 shutdown_connections 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f881006b080 0x7f881006d540 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f882c15c7f0 0x7f882c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f882c0cf450 0x7f882c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 --2- 172.21.15.106:0/1102281992 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f882c0cdfb0 0x7f882c09cbb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 >> 172.21.15.106:0/1102281992 conn(0x7f882c06a920 msgr2=0x7f882c15f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 shutdown_connections 2023-11-24T08:11:36.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.090+0000 7f883441c640 1 -- 172.21.15.106:0/1102281992 wait complete. 2023-11-24T08:11:36.110 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:36.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.324+0000 7feadac09640 1 Processor -- start 2023-11-24T08:11:36.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.325+0000 7feadac09640 1 -- start start 2023-11-24T08:11:36.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.326+0000 7feadac09640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 0x7fead40d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.326+0000 7feadac09640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead40d0240 0x7fead40b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.326+0000 7feadac09640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead40ce990 con 0x7fead40d38e0 2023-11-24T08:11:36.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.326+0000 7feadac09640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead40d4110 con 0x7fead415d4e0 2023-11-24T08:11:36.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.326+0000 7feadac09640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40cec70 con 0x7fead40d0240 2023-11-24T08:11:36.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.327+0000 7fead9c07640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 0x7fead40d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.327+0000 7fead9c07640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 0x7fead40d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:35014/0 (socket says 172.21.15.106:35014) 2023-11-24T08:11:36.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.327+0000 7fead9c07640 1 -- 172.21.15.106:0/2248520319 learned_addr learned my addr 172.21.15.106:0/2248520319 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:36.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.327+0000 7feada408640 1 --2- 172.21.15.106:0/2248520319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead40d0240 0x7fead40b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.328+0000 7feada408640 1 -- 172.21.15.106:0/2248520319 >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 msgr2=0x7fead40d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:36.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.328+0000 7feada408640 1 --2- 172.21.15.106:0/2248520319 >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 0x7fead40d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.328+0000 7feada408640 1 -- 172.21.15.106:0/2248520319 >> v1:172.21.15.156:6789/0 conn(0x7fead40d38e0 legacy=0x7fead40cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.328+0000 7feada408640 1 -- 172.21.15.106:0/2248520319 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead40cef50 con 0x7fead40d0240 2023-11-24T08:11:36.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.328+0000 7fead9c07640 1 --2- 172.21.15.106:0/2248520319 >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 0x7fead40d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:11:36.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.329+0000 7feada408640 1 --2- 172.21.15.106:0/2248520319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead40d0240 0x7fead40b2e50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feacc0097a0 tx=0x7feacc02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=103c5c26949aa8ef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.330+0000 7fead8c05640 1 -- 172.21.15.106:0/2248520319 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc02d040 con 0x7fead40d0240 2023-11-24T08:11:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.330+0000 7fead8c05640 1 -- 172.21.15.106:0/2248520319 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feacc031810 con 0x7fead40d0240 2023-11-24T08:11:36.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.330+0000 7fead8c05640 1 -- 172.21.15.106:0/2248520319 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feacc031ba0 con 0x7fead40d0240 2023-11-24T08:11:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.330+0000 7feadac09640 1 -- 172.21.15.106:0/2248520319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead40d0240 msgr2=0x7fead40b2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.330+0000 7feadac09640 1 --2- 172.21.15.106:0/2248520319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead40d0240 0x7fead40b2e50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feacc0097a0 tx=0x7feacc02b8b0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.331+0000 7feadac09640 1 -- 172.21.15.106:0/2248520319 shutdown_connections 2023-11-24T08:11:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.331+0000 7feadac09640 1 --2- 172.21.15.106:0/2248520319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead40d0240 0x7fead40b2e50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.331+0000 7feadac09640 1 --2- 172.21.15.106:0/2248520319 >> v2:172.21.15.156:3300/0 conn(0x7fead415d4e0 0x7fead40d1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.331+0000 7feadac09640 1 -- 172.21.15.106:0/2248520319 >> 172.21.15.106:0/2248520319 conn(0x7fead409c510 msgr2=0x7fead40d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:36.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.331+0000 7feadac09640 1 -- 172.21.15.106:0/2248520319 shutdown_connections 2023-11-24T08:11:36.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.331+0000 7feadac09640 1 -- 172.21.15.106:0/2248520319 wait complete. 2023-11-24T08:11:36.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.332+0000 7feadac09640 1 Processor -- start 2023-11-24T08:11:36.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.333+0000 7feadac09640 1 -- start start 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.333+0000 7feadac09640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 0x7fead41f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7feadac09640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead40d38e0 0x7fead40b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7feadac09640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead415d4e0 0x7fead4160160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7feadac09640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b0b60 con 0x7fead40d0240 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7feadac09640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b16e0 con 0x7fead40d38e0 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7feadac09640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead40b0db0 con 0x7fead415d4e0 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 0x7fead41f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 0x7fead41f2560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43876/0 (socket says 172.21.15.106:43876) 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 -- 172.21.15.106:0/3888742072 learned_addr learned my addr 172.21.15.106:0/3888742072 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7feada408640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead415d4e0 0x7fead4160160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9406640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead40d38e0 0x7fead40b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 -- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead415d4e0 msgr2=0x7fead4160160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead415d4e0 0x7fead4160160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 -- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead40d38e0 msgr2=0x7fead40b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead40d38e0 0x7fead40b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9c07640 1 -- 172.21.15.106:0/3888742072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac4009b30 con 0x7fead40d0240 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.334+0000 7fead9406640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead40d38e0 0x7fead40b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:36.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.335+0000 7fead9c07640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 0x7fead41f2560 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feac4009e80 tx=0x7feac400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:36.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.335+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac400ce60 con 0x7fead40d0240 2023-11-24T08:11:36.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.335+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac400f040 con 0x7fead40d0240 2023-11-24T08:11:36.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.335+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc009400 con 0x7fead40d0240 2023-11-24T08:11:36.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.335+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac4014680 con 0x7fead40d0240 2023-11-24T08:11:36.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.335+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead40b1370 con 0x7fead40d0240 2023-11-24T08:11:36.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.336+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead404e840 con 0x7fead40d0240 2023-11-24T08:11:36.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.337+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7feac40147e0 con 0x7fead40d0240 2023-11-24T08:11:36.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.338+0000 7fead27fc640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac006b080 0x7feac006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.339 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.338+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feac408a320 con 0x7fead40d0240 2023-11-24T08:11:36.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.338+0000 7fead9406640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac006b080 0x7feac006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.339+0000 7fead9406640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac006b080 0x7feac006d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fead415a280 tx=0x7feac8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:36.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.342+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7feac4054ff0 con 0x7fead40d0240 2023-11-24T08:11:36.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.532+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fead40d3480 con 0x7fead40d0240 2023-11-24T08:11:36.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.536+0000 7fead27fc640 1 -- 172.21.15.106:0/3888742072 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+16613 (secure 0 0 0) 0x7feac4058610 con 0x7fead40d0240 2023-11-24T08:11:36.538 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:36.538 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":62,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:33.580022+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:36.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.538+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac006b080 msgr2=0x7feac006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.538+0000 7feadac09640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac006b080 0x7feac006d540 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fead415a280 tx=0x7feac8009040 comp rx=0 tx=0).stop 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.538+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 msgr2=0x7fead41f2560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.538+0000 7feadac09640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 0x7fead41f2560 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7feac4009e80 tx=0x7feac400c490 comp rx=0 tx=0).stop 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 shutdown_connections 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7feac006b080 0x7feac006d540 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fead415d4e0 0x7fead4160160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fead40d38e0 0x7fead40b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 --2- 172.21.15.106:0/3888742072 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fead40d0240 0x7fead41f2560 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 >> 172.21.15.106:0/3888742072 conn(0x7fead409c510 msgr2=0x7fead40cc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 shutdown_connections 2023-11-24T08:11:36.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.539+0000 7feadac09640 1 -- 172.21.15.106:0/3888742072 wait complete. 2023-11-24T08:11:36.559 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:11:36.559 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:11:36.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.765+0000 7ff7256df640 1 Processor -- start 2023-11-24T08:11:36.767 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.765+0000 7ff7256df640 1 -- start start 2023-11-24T08:11:36.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.767+0000 7ff7256df640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 0x7ff7200d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.767+0000 7ff7256df640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7200ce990 con 0x7ff7200d38e0 2023-11-24T08:11:36.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.767+0000 7ff7256df640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7200cec70 con 0x7ff7200d0240 2023-11-24T08:11:36.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.767+0000 7ff7256df640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200d4110 con 0x7ff72015d4e0 2023-11-24T08:11:36.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.767+0000 7ff71f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff7200d38e0 0x7ff7200cfb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37568/0 (socket says 172.21.15.106:37568) 2023-11-24T08:11:36.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.767+0000 7ff71f7fe640 1 -- 172.21.15.106:0/4281224165 learned_addr learned my addr 172.21.15.106:0/4281224165 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1848820595 0 0) 0x7ff7200ce990 con 0x7ff7200d38e0 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71ffff640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 0x7ff7200d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff708003520 con 0x7ff7200d38e0 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3163240 0 0) 0x7ff708003520 con 0x7ff7200d38e0 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 msgr2=0x7ff7200d1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 0x7ff7200d1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 >> v1:172.21.15.156:6789/0 conn(0x7ff7200d0240 legacy=0x7ff7200b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7200cef50 con 0x7ff7200d38e0 2023-11-24T08:11:36.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7ff714001690 con 0x7ff7200d38e0 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 >> v1:172.21.15.106:6789/0 conn(0x7ff7200d38e0 legacy=0x7ff7200cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 0x7ff7080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff708003f20 0x7ff7080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708004d00 0x7ff708005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200d4110 con 0x7ff708003f20 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800ba60 con 0x7ff708004d00 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.768+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70800bc20 con 0x7ff72015d4e0 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff708003f20 0x7ff7080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71ffff640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 0x7ff7080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff724ede640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708004d00 0x7ff708005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71f7fe640 1 -- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 msgr2=0x7ff7080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:36.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff72015d4e0 0x7ff7080039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71f7fe640 1 -- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708004d00 msgr2=0x7ff708005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:36.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff708004d00 0x7ff708005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.769+0000 7ff71f7fe640 1 -- 172.21.15.106:0/4281224165 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff708003520 con 0x7ff708003f20 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.770+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff708003f20 0x7ff7080047c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c210 tx=0x7ff7080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=865d64cca1961dfb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff714005ad0 con 0x7ff708003f20 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff714005c30 con 0x7ff708003f20 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff71effd640 1 -- 172.21.15.106:0/4281224165 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71402c870 con 0x7ff708003f20 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff7256df640 1 -- 172.21.15.106:0/4281224165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff708003f20 msgr2=0x7ff7080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff7256df640 1 --2- 172.21.15.106:0/4281224165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff708003f20 0x7ff7080047c0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c210 tx=0x7ff7080057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff724ede640 1 -- 172.21.15.106:0/4281224165 reap_dead start 2023-11-24T08:11:36.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff7256df640 1 -- 172.21.15.106:0/4281224165 shutdown_connections 2023-11-24T08:11:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff7256df640 1 -- 172.21.15.106:0/4281224165 >> 172.21.15.106:0/4281224165 conn(0x7ff72009c510 msgr2=0x7ff7200d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff7256df640 1 -- 172.21.15.106:0/4281224165 shutdown_connections 2023-11-24T08:11:36.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.771+0000 7ff7256df640 1 -- 172.21.15.106:0/4281224165 wait complete. 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.773+0000 7ff7256df640 1 Processor -- start 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.773+0000 7ff7256df640 1 -- start start 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.774+0000 7ff7256df640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff708003f20 0x7ff7200c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.774+0000 7ff7256df640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 0x7ff720152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.774+0000 7ff7256df640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff72015fb00 0x7ff72015ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.774+0000 7ff7256df640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7201604c0 con 0x7ff72015f590 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.774+0000 7ff7256df640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff72015ef00 con 0x7ff72015fb00 2023-11-24T08:11:36.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.774+0000 7ff7256df640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7200c9900 con 0x7ff708003f20 2023-11-24T08:11:36.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 0x7ff720152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 0x7ff720152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43898/0 (socket says 172.21.15.106:43898) 2023-11-24T08:11:36.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 -- 172.21.15.106:0/3305287685 learned_addr learned my addr 172.21.15.106:0/3305287685 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff724ede640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff72015fb00 0x7ff72015ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 -- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff708003f20 msgr2=0x7ff7200c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff708003f20 0x7ff7200c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 -- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff72015fb00 msgr2=0x7ff72015ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff72015fb00 0x7ff72015ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 -- 172.21.15.106:0/3305287685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff72015e610 con 0x7ff72015f590 2023-11-24T08:11:36.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff724ede640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff72015fb00 0x7ff72015ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71f7fe640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 0x7ff720152760 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c210 tx=0x7ff7200cf230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.775+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff71402cc80 con 0x7ff72015f590 2023-11-24T08:11:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.776+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff71402ce00 con 0x7ff72015f590 2023-11-24T08:11:36.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.776+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff72015e8b0 con 0x7ff72015f590 2023-11-24T08:11:36.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.776+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7140072b0 con 0x7ff72015f590 2023-11-24T08:11:36.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.776+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7200cc6f0 con 0x7ff72015f590 2023-11-24T08:11:36.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.777+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff72004e840 con 0x7ff72015f590 2023-11-24T08:11:36.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.778+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7ff714032070 con 0x7ff72015f590 2023-11-24T08:11:36.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.779+0000 7ff71cff9640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff70c06b080 0x7ff70c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:36.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.779+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7140aab80 con 0x7ff72015f590 2023-11-24T08:11:36.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.779+0000 7ff71ffff640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff70c06b080 0x7ff70c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:36.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.780+0000 7ff71ffff640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff70c06b080 0x7ff70c06d540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff7200b2be0 tx=0x7ff710008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:36.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.783+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff714075a30 con 0x7ff72015f590 2023-11-24T08:11:36.967 INFO:tasks.ceph.mds.c.smithi106.stderr:2023-11-24T08:11:36.966+0000 7ffbd0bfa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:11:36.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.967+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7200d1e20 con 0x7ff70c06b080 2023-11-24T08:11:36.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.991+0000 7ff71cff9640 1 -- 172.21.15.106:0/3305287685 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+272317 (secure 0 0 0) 0x7ff710011860 con 0x7ff70c06b080 2023-11-24T08:11:36.997 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:36.999 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff70c06b080 msgr2=0x7ff70c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff70c06b080 0x7ff70c06d540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ff7200b2be0 tx=0x7ff710008040 comp rx=0 tx=0).stop 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 msgr2=0x7ff720152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 0x7ff720152760 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c210 tx=0x7ff7200cf230 comp rx=0 tx=0).stop 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 shutdown_connections 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff72015fb00 0x7ff72015ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff72015f590 0x7ff720152760 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.996+0000 7ff7256df640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff70c06b080 0x7ff70c06d540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.997+0000 7ff7256df640 1 --2- 172.21.15.106:0/3305287685 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff708003f20 0x7ff7200c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.997+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 >> 172.21.15.106:0/3305287685 conn(0x7ff72009c510 msgr2=0x7ff7200cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:37.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.997+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 shutdown_connections 2023-11-24T08:11:37.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:36.997+0000 7ff7256df640 1 -- 172.21.15.106:0/3305287685 wait complete. 2023-11-24T08:11:37.018 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2023-11-24T08:11:35.324791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":239408,"kb_used_data":6568,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754735312,"statfs":{"total":773094113280,"available":772848959488,"internally_reserved":0,"allocated":6725632,"data_stored":3455496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":238276144},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.766306"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:31.556545+0000","last_change":"2023-11-24T08:11:31.556545+0000","last_active":"2023-11-24T08:11:31.556545+0000","last_peered":"2023-11-24T08:11:31.556545+0000","last_clean":"2023-11-24T08:11:31.556545+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:31.556545+0000","last_undegraded":"2023-11-24T08:11:31.556545+0000","last_fullsized":"2023-11-24T08:11:31.556545+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:11:30.541099+0000","last_change":"2023-11-24T08:11:30.541099+0000","last_active":"2023-11-24T08:11:30.541099+0000","last_peered":"2023-11-24T08:11:30.541099+0000","last_clean":"2023-11-24T08:11:30.541099+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:11:30.541099+0000","last_undegraded":"2023-11-24T08:11:30.541099+0000","last_fullsized":"2023-11-24T08:11:30.541099+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":193,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-11-24T08:11:26.528241+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:11:26.528241+0000","last_peered":"2023-11-24T08:11:26.528241+0000","last_clean":"2023-11-24T08:11:26.528241+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:11:26.528241+0000","last_undegraded":"2023-11-24T08:11:26.528241+0000","last_fullsized":"2023-11-24T08:11:26.528241+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542182,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32436,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339404,"statfs":{"total":96636764160,"available":96603549696,"internally_reserved":0,"allocated":1286144,"data_stored":874713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.018,"15min":1.0129999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.002},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.014,"15min":0.99299999999999999},"min":{"1min":0.65300000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.51}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.141,"15min":1.129},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.179,"15min":1.167},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2250000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.2410000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.093,"15min":1.0589999999999999},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.147},{"interface":"front","average":{"1min":1.27,"5min":1.2150000000000001,"15min":1.206},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0699999999999998,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.296}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.25,"15min":1.222},"min":{"1min":0.97999999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.458}]},{"osd":4,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.2330000000000001,"15min":1.2110000000000001},"min":{"1min":0.95599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3959999999999999,"15min":1.3779999999999999},"min":{"1min":1.0920000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":0.96599999999999997}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.4399999999999999,"15min":1.4390000000000001},"min":{"1min":1.0449999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.931,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.371,"15min":1.355},"min":{"1min":1.0149999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.3460000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.494,"15min":1.4790000000000001},"min":{"1min":1.1890000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.581},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.482},"min":{"1min":1.016,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.8450000000000002,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.5449999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":284345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.091,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6220000000000001,"5min":2.085,"15min":2.085},"last":1.2729999999999999},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2390000000000001,"15min":1.2450000000000001},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.766,"5min":1.794,"15min":1.794},"last":1.4850000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.121,"15min":1.127},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.837,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.77400000000000002,"5min":0.749,"15min":0.749},"max":{"1min":1.615,"5min":1.98,"15min":1.98},"last":0.94799999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7729999999999999,"5min":1.778,"15min":1.778},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.105,"5min":1.1859999999999999,"15min":1.2},"min":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7649999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.53}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3340000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7010000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4399999999999999,"15min":1.448},"min":{"1min":0.83299999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.948,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.1930000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.359,"15min":1.3740000000000001},"min":{"1min":0.92200000000000004,"5min":0.622,"15min":0.622},"max":{"1min":1.9750000000000001,"5min":2.343,"15min":2.343},"last":1.651},{"interface":"front","average":{"1min":1.444,"5min":1.571,"15min":1.5920000000000001},"min":{"1min":0.79800000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9099999999999999,"5min":2.661,"15min":2.661},"last":1.782}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.6859999999999999,"15min":1.728},"min":{"1min":0.88400000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.012,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.5800000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.635,"15min":1.6659999999999999},"min":{"1min":0.70699999999999996,"5min":0.505,"15min":0.505},"max":{"1min":2.2610000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.708}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.7709999999999999,"15min":1.7949999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.3140000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.9139999999999999},{"interface":"front","average":{"1min":1.573,"5min":1.7669999999999999,"15min":1.7989999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.109,"5min":2.657,"15min":2.657},"last":1.8180000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542180,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31872,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339968,"statfs":{"total":96636764160,"available":96604127232,"internally_reserved":0,"allocated":692224,"data_stored":284345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.042,"15min":1.022},"min":{"1min":0.63600000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1639999999999999,"15min":1.1679999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.0740000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1850000000000001,"15min":1.1970000000000001},"min":{"1min":0.70199999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5469999999999999,"5min":1.722,"15min":1.722},"last":1.4119999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.76500000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.9179999999999999,"5min":2.085,"15min":2.085},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.304,"5min":1.3620000000000001,"15min":1.3720000000000001},"min":{"1min":0.748,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.75,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.1359999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4330000000000001,"15min":1.4450000000000001},"min":{"1min":0.85599999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7130000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.357},{"interface":"front","average":{"1min":1.395,"5min":1.512,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9510000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.0149999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":0.91800000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7849999999999999,"5min":3.577,"15min":3.577},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.694,"15min":1.7190000000000001},"min":{"1min":1.133,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.9750000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.5169999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6100000000000001,"15min":1.617},"min":{"1min":0.98299999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.121,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.546,"15min":1.54},"min":{"1min":1.1279999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.996,"5min":2.512,"15min":2.512},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.7130000000000001,"15min":1.726},"min":{"1min":1.2070000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.897,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.74,"15min":1.748},"min":{"1min":1.3220000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1989999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.462}]}]},{"osd":4,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":31856,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":692224,"data_stored":284345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.747,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.524,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.69599999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.73399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.135,"15min":1.1479999999999999},"min":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4790000000000001,"5min":1.859,"15min":1.859},"last":1.522},{"interface":"front","average":{"1min":1.254,"5min":1.171,"15min":1.157},"min":{"1min":0.83399999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.84499999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.871,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8759999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.417},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.92300000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.004,"5min":2.25,"15min":2.25},"last":1.306},{"interface":"front","average":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4199999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0899999999999999,"5min":2.746,"15min":2.746},"last":1.264}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.319,"15min":1.325},"min":{"1min":0.82399999999999995,"5min":0.627,"15min":0.627},"max":{"1min":1.7350000000000001,"5min":1.923,"15min":1.923},"last":1.056},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4530000000000001,"15min":1.458},"min":{"1min":0.93100000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.145,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.631}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.444,"15min":1.452},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.946,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.375},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5609999999999999,"15min":1.5649999999999999},"min":{"1min":0.84799999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9239999999999999,"5min":2.641,"15min":2.641},"last":1.577}]},{"osd":7,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5169999999999999,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.3769999999999998,"5min":2.548,"15min":2.548},"last":1.482},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.77100000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1989999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.2110000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27704,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":692224,"data_stored":284345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1439999999999999,"15min":1.177},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5389999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":1.127,"15min":1.165},"min":{"1min":0.61299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.333,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.54400000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.98699999999999999,"15min":1.0049999999999999},"min":{"1min":0.63100000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2529999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.5},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.131,"15min":1.1359999999999999},"min":{"1min":0.61399999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.466,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.222,"15min":1.242},"min":{"1min":0.67200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4930000000000001,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.60899999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.159,"15min":1.1599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.766,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.53200000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4630000000000001,"15min":1.484},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.9179999999999999,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":1.087,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.9950000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":0.64100000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":0.98499999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.853,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":0.72799999999999998},{"interface":"front","average":{"1min":1.494,"5min":1.548,"15min":1.5569999999999999},"min":{"1min":1.0129999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0339999999999998,"5min":3.339,"15min":3.339},"last":0.98399999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.657,"15min":1.698},"min":{"1min":1.0580000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.732,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.504,"5min":1.5609999999999999,"15min":1.571},"min":{"1min":1.081,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.966,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.0369999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.6859999999999999,"15min":1.702},"min":{"1min":1.248,"5min":1.048,"15min":1.048},"max":{"1min":2.028,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.048},{"interface":"front","average":{"1min":1.484,"5min":1.9830000000000001,"15min":2.0659999999999998},"min":{"1min":0.86099999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8420000000000001,"5min":16.952999999999999,"15min":16.952999999999999},"last":0.82299999999999995}]}]},{"osd":2,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27708,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":692224,"data_stored":284345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.78900000000000003,"15min":0.76700000000000002},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.6950000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85999999999999999,"15min":0.874},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.1599999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.97199999999999998,"15min":0.96999999999999997},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.466,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.387},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.93200000000000005}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.752,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5960000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.452},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2150000000000001,"15min":1.2190000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.79}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.4179999999999999,"15min":1.45},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.895,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.28,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.752}]},{"osd":6,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4039999999999999,"15min":1.423},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8440000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":1.5469999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4650000000000001,"15min":1.4870000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.9219999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4350000000000001,"15min":1.446},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9910000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.633}]}]},{"osd":1,"up_from":14,"seq":60129542181,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27696,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344144,"statfs":{"total":96636764160,"available":96608403456,"internally_reserved":0,"allocated":692224,"data_stored":284345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.175,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.125,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.748}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85899999999999999,"15min":0.85499999999999998},"min":{"1min":0.46700000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.52,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0209999999999999,"15min":1.01},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4219999999999999,"5min":1.621,"15min":1.621},"last":0.70999999999999996}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.157,"15min":1.163},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5509999999999999,"5min":1.907,"15min":1.907},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.999,"5min":1.048,"15min":1.056},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.069,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.98299999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.91700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.7270000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.425},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3380000000000001,"15min":1.335},"min":{"1min":0.86299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8280000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.089}]},{"osd":5,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.625,"5min":2.661,"15min":2.661},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.3300000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6279999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.514}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3580000000000001,"15min":1.361},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.337,"15min":1.333},"min":{"1min":0.98299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.704,"5min":2.105,"15min":2.105},"last":1.47}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4139999999999999,"15min":1.4059999999999999},"min":{"1min":1.0569999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.0339999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.351},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3580000000000001,"15min":1.365},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7709999999999999,"5min":2.008,"15min":2.008},"last":1.494}]}]},{"osd":0,"up_from":14,"seq":60129542181,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28280,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343560,"statfs":{"total":96636764160,"available":96607805440,"internally_reserved":0,"allocated":1286144,"data_stored":874713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2909999999999999,"5min":1.403,"15min":1.403},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.494,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.82399999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.44,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92600000000000005,"15min":0.93400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.335,"5min":1.986,"15min":1.986},"last":0.71199999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0649999999999999,"15min":1.077},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.704,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.026},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95499999999999996,"15min":0.97399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.585,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.88900000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.163,"15min":1.1699999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.107},{"interface":"front","average":{"1min":1.288,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7849999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.2010000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.228},"min":{"1min":0.747,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7270000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.357},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5860000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.268}]},{"osd":6,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.288,"15min":1.2789999999999999},"min":{"1min":0.755,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.9139999999999999,"5min":2.149,"15min":2.149},"last":1.3300000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.85399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7969999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.423,"15min":1.425},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.0600000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.23}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:11:37.024 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:11:37.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.277+0000 7f09b1659640 1 Processor -- start 2023-11-24T08:11:37.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.277+0000 7f09b1659640 1 -- start start 2023-11-24T08:11:37.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09b1659640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac144b30 0x7f09ac15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09b1659640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09ac0b1de0 0x7f09ac0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09b1659640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ac15bf80 con 0x7f09ac15c410 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09b1659640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ac0b27b0 con 0x7f09ac0b1de0 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09b1659640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0b1560 con 0x7f09ac144b30 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09b0e58640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f09ac0b1de0 0x7f09ac0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09a3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f09ac15c410 0x7f09ac0b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37582/0 (socket says 172.21.15.106:37582) 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09a3fff640 1 -- 172.21.15.106:0/864466485 learned_addr learned my addr 172.21.15.106:0/864466485 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:37.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09abfff640 1 --2- 172.21.15.106:0/864466485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac144b30 0x7f09ac15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.279+0000 7f09ab7fe640 1 -- 172.21.15.106:0/864466485 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2902867539 0 0) 0x7f09ac15bf80 con 0x7f09ac15c410 2023-11-24T08:11:37.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.280+0000 7f09b0e58640 1 -- 172.21.15.106:0/864466485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac144b30 msgr2=0x7f09ac15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.280+0000 7f09b0e58640 1 --2- 172.21.15.106:0/864466485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac144b30 0x7f09ac15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.280+0000 7f09b0e58640 1 -- 172.21.15.106:0/864466485 >> v1:172.21.15.106:6789/0 conn(0x7f09ac15c410 legacy=0x7f09ac0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.280+0000 7f09b0e58640 1 -- 172.21.15.106:0/864466485 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0b1280 con 0x7f09ac0b1de0 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.281+0000 7f09b0e58640 1 --2- 172.21.15.106:0/864466485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac0b2270 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009d90 tx=0x7f09a402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28851e8255180470 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.281+0000 7f09ab7fe640 1 -- 172.21.15.106:0/864466485 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a402cc50 con 0x7f09ac0b1de0 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.281+0000 7f09ab7fe640 1 -- 172.21.15.106:0/864466485 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a402cdb0 con 0x7f09ac0b1de0 2023-11-24T08:11:37.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.281+0000 7f09ab7fe640 1 -- 172.21.15.106:0/864466485 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4034660 con 0x7f09ac0b1de0 2023-11-24T08:11:37.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.281+0000 7f09b1659640 1 -- 172.21.15.106:0/864466485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 msgr2=0x7f09ac0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.281+0000 7f09b1659640 1 --2- 172.21.15.106:0/864466485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac0b2270 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f09a4009d90 tx=0x7f09a402bea0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.282+0000 7f09b1659640 1 -- 172.21.15.106:0/864466485 shutdown_connections 2023-11-24T08:11:37.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.282+0000 7f09b1659640 1 --2- 172.21.15.106:0/864466485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac0b2270 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.282+0000 7f09b1659640 1 --2- 172.21.15.106:0/864466485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac144b30 0x7f09ac15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.282+0000 7f09b1659640 1 -- 172.21.15.106:0/864466485 >> 172.21.15.106:0/864466485 conn(0x7f09ac0cb830 msgr2=0x7f09ac0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:37.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.282+0000 7f09b1659640 1 -- 172.21.15.106:0/864466485 shutdown_connections 2023-11-24T08:11:37.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.282+0000 7f09b1659640 1 -- 172.21.15.106:0/864466485 wait complete. 2023-11-24T08:11:37.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.283+0000 7f09b1659640 1 Processor -- start 2023-11-24T08:11:37.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.284+0000 7f09b1659640 1 -- start start 2023-11-24T08:11:37.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.284+0000 7f09b1659640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac13bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.284+0000 7f09b1659640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ac144b30 0x7f09ac0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.284+0000 7f09b1659640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac15c410 0x7f09ac0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.284+0000 7f09b1659640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac1f4300 con 0x7f09ac0b1de0 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09b1659640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac1f36f0 con 0x7f09ac144b30 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09b1659640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac1f39d0 con 0x7f09ac15c410 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac13bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac13bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43912/0 (socket says 172.21.15.106:43912) 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 -- 172.21.15.106:0/2270641551 learned_addr learned my addr 172.21.15.106:0/2270641551 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09a3fff640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ac144b30 0x7f09ac0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 -- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac15c410 msgr2=0x7f09ac0ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac15c410 0x7f09ac0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 -- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ac144b30 msgr2=0x7f09ac0ca330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ac144b30 0x7f09ac0ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.285+0000 7f09abfff640 1 -- 172.21.15.106:0/2270641551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a4009a70 con 0x7f09ac0b1de0 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.286+0000 7f09abfff640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac13bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f099c00d800 tx=0x7f099c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.286+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00b6c0 con 0x7f09ac0b1de0 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.286+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f099c00b820 con 0x7f09ac0b1de0 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.286+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f099c00bbb0 con 0x7f09ac0b1de0 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.286+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac1f3cb0 con 0x7f09ac0b1de0 2023-11-24T08:11:37.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.286+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac13c530 con 0x7f09ac0b1de0 2023-11-24T08:11:37.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.287+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac04e840 con 0x7f09ac0b1de0 2023-11-24T08:11:37.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.288+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f099c00bd10 con 0x7f09ac0b1de0 2023-11-24T08:11:37.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.289+0000 7f09a97fa640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f098c06b080 0x7f098c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.289+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f099c08a110 con 0x7f09ac0b1de0 2023-11-24T08:11:37.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.289+0000 7f09a3fff640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f098c06b080 0x7f098c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.290+0000 7f09a3fff640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f098c06b080 0x7f098c06d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09ac152fc0 tx=0x7f0994009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:37.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.293+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f099c054d30 con 0x7f09ac0b1de0 2023-11-24T08:11:37.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.481+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09ac1f3b40 con 0x7f098c06b080 2023-11-24T08:11:37.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.490+0000 7f09a97fa640 1 -- 172.21.15.106:0/2270641551 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286914 (secure 0 0 0) 0x7f09ac1f3b40 con 0x7f098c06b080 2023-11-24T08:11:37.494 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:37.497 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:11:37.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f098c06b080 msgr2=0x7f098c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f098c06b080 0x7f098c06d540 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f09ac152fc0 tx=0x7f0994009040 comp rx=0 tx=0).stop 2023-11-24T08:11:37.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 msgr2=0x7f09ac13bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac13bff0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f099c00d800 tx=0x7f099c00dcc0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 shutdown_connections 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f098c06b080 0x7f098c06d540 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09ac15c410 0x7f09ac0ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09ac144b30 0x7f09ac0ca330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 --2- 172.21.15.106:0/2270641551 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09ac0b1de0 0x7f09ac13bff0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 >> 172.21.15.106:0/2270641551 conn(0x7f09ac0cb830 msgr2=0x7f09ac0cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:37.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 shutdown_connections 2023-11-24T08:11:37.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.497+0000 7f09b1659640 1 -- 172.21.15.106:0/2270641551 wait complete. 2023-11-24T08:11:37.519 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":144,"stamp":"2023-11-24T08:11:37.325355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":181,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243992,"kb_used_data":7016,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754730728,"statfs":{"total":773094113280,"available":772844265472,"internally_reserved":0,"allocated":7184384,"data_stored":3661588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001560"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.589380+0000","last_change":"2023-11-24T08:11:32.612196+0000","last_active":"2023-11-24T08:11:33.589380+0000","last_peered":"2023-11-24T08:11:33.589380+0000","last_clean":"2023-11-24T08:11:33.589380+0000","last_became_active":"2023-11-24T08:11:32.611781+0000","last_became_peered":"2023-11-24T08:11:32.611781+0000","last_unstale":"2023-11-24T08:11:33.589380+0000","last_undegraded":"2023-11-24T08:11:33.589380+0000","last_fullsized":"2023-11-24T08:11:33.589380+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:43.484801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360769+0000","last_change":"2023-11-24T08:11:32.596972+0000","last_active":"2023-11-24T08:11:34.360769+0000","last_peered":"2023-11-24T08:11:34.360769+0000","last_clean":"2023-11-24T08:11:34.360769+0000","last_became_active":"2023-11-24T08:11:32.596205+0000","last_became_peered":"2023-11-24T08:11:32.596205+0000","last_unstale":"2023-11-24T08:11:34.360769+0000","last_undegraded":"2023-11-24T08:11:34.360769+0000","last_fullsized":"2023-11-24T08:11:34.360769+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:55.370960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357708+0000","last_change":"2023-11-24T08:11:32.612160+0000","last_active":"2023-11-24T08:11:34.357708+0000","last_peered":"2023-11-24T08:11:34.357708+0000","last_clean":"2023-11-24T08:11:34.357708+0000","last_became_active":"2023-11-24T08:11:32.611837+0000","last_became_peered":"2023-11-24T08:11:32.611837+0000","last_unstale":"2023-11-24T08:11:34.357708+0000","last_undegraded":"2023-11-24T08:11:34.357708+0000","last_fullsized":"2023-11-24T08:11:34.357708+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:42.084881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356327+0000","last_change":"2023-11-24T08:11:32.607992+0000","last_active":"2023-11-24T08:11:34.356327+0000","last_peered":"2023-11-24T08:11:34.356327+0000","last_clean":"2023-11-24T08:11:34.356327+0000","last_became_active":"2023-11-24T08:11:32.607679+0000","last_became_peered":"2023-11-24T08:11:32.607679+0000","last_unstale":"2023-11-24T08:11:34.356327+0000","last_undegraded":"2023-11-24T08:11:34.356327+0000","last_fullsized":"2023-11-24T08:11:34.356327+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:50.618406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.588689+0000","last_change":"2023-11-24T08:11:32.607761+0000","last_active":"2023-11-24T08:11:33.588689+0000","last_peered":"2023-11-24T08:11:33.588689+0000","last_clean":"2023-11-24T08:11:33.588689+0000","last_became_active":"2023-11-24T08:11:32.604977+0000","last_became_peered":"2023-11-24T08:11:32.604977+0000","last_unstale":"2023-11-24T08:11:33.588689+0000","last_undegraded":"2023-11-24T08:11:33.588689+0000","last_fullsized":"2023-11-24T08:11:33.588689+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:26:33.374074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356081+0000","last_change":"2023-11-24T08:11:32.604371+0000","last_active":"2023-11-24T08:11:34.356081+0000","last_peered":"2023-11-24T08:11:34.356081+0000","last_clean":"2023-11-24T08:11:34.356081+0000","last_became_active":"2023-11-24T08:11:32.604107+0000","last_became_peered":"2023-11-24T08:11:32.604107+0000","last_unstale":"2023-11-24T08:11:34.356081+0000","last_undegraded":"2023-11-24T08:11:34.356081+0000","last_fullsized":"2023-11-24T08:11:34.356081+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:15.733058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.591306+0000","last_change":"2023-11-24T08:11:32.594543+0000","last_active":"2023-11-24T08:11:33.591306+0000","last_peered":"2023-11-24T08:11:33.591306+0000","last_clean":"2023-11-24T08:11:33.591306+0000","last_became_active":"2023-11-24T08:11:32.594135+0000","last_became_peered":"2023-11-24T08:11:32.594135+0000","last_unstale":"2023-11-24T08:11:33.591306+0000","last_undegraded":"2023-11-24T08:11:33.591306+0000","last_fullsized":"2023-11-24T08:11:33.591306+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:43:48.672109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359508+0000","last_change":"2023-11-24T08:11:32.601206+0000","last_active":"2023-11-24T08:11:34.359508+0000","last_peered":"2023-11-24T08:11:34.359508+0000","last_clean":"2023-11-24T08:11:34.359508+0000","last_became_active":"2023-11-24T08:11:32.600937+0000","last_became_peered":"2023-11-24T08:11:32.600937+0000","last_unstale":"2023-11-24T08:11:34.359508+0000","last_undegraded":"2023-11-24T08:11:34.359508+0000","last_fullsized":"2023-11-24T08:11:34.359508+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:12:17.052279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.593035+0000","last_change":"2023-11-24T08:11:32.611300+0000","last_active":"2023-11-24T08:11:33.593035+0000","last_peered":"2023-11-24T08:11:33.593035+0000","last_clean":"2023-11-24T08:11:33.593035+0000","last_became_active":"2023-11-24T08:11:32.611007+0000","last_became_peered":"2023-11-24T08:11:32.611007+0000","last_unstale":"2023-11-24T08:11:33.593035+0000","last_undegraded":"2023-11-24T08:11:33.593035+0000","last_fullsized":"2023-11-24T08:11:33.593035+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:22:27.540777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360033+0000","last_change":"2023-11-24T08:11:32.603559+0000","last_active":"2023-11-24T08:11:34.360033+0000","last_peered":"2023-11-24T08:11:34.360033+0000","last_clean":"2023-11-24T08:11:34.360033+0000","last_became_active":"2023-11-24T08:11:32.601889+0000","last_became_peered":"2023-11-24T08:11:32.601889+0000","last_unstale":"2023-11-24T08:11:34.360033+0000","last_undegraded":"2023-11-24T08:11:34.360033+0000","last_fullsized":"2023-11-24T08:11:34.360033+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:17.350019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359297+0000","last_change":"2023-11-24T08:11:32.591322+0000","last_active":"2023-11-24T08:11:34.359297+0000","last_peered":"2023-11-24T08:11:34.359297+0000","last_clean":"2023-11-24T08:11:34.359297+0000","last_became_active":"2023-11-24T08:11:32.591010+0000","last_became_peered":"2023-11-24T08:11:32.591010+0000","last_unstale":"2023-11-24T08:11:34.359297+0000","last_undegraded":"2023-11-24T08:11:34.359297+0000","last_fullsized":"2023-11-24T08:11:34.359297+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:58.380777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356632+0000","last_change":"2023-11-24T08:11:32.614474+0000","last_active":"2023-11-24T08:11:34.356632+0000","last_peered":"2023-11-24T08:11:34.356632+0000","last_clean":"2023-11-24T08:11:34.356632+0000","last_became_active":"2023-11-24T08:11:32.614150+0000","last_became_peered":"2023-11-24T08:11:32.614150+0000","last_unstale":"2023-11-24T08:11:34.356632+0000","last_undegraded":"2023-11-24T08:11:34.356632+0000","last_fullsized":"2023-11-24T08:11:34.356632+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:52:36.116848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592959+0000","last_change":"2023-11-24T08:11:32.611953+0000","last_active":"2023-11-24T08:11:33.592959+0000","last_peered":"2023-11-24T08:11:33.592959+0000","last_clean":"2023-11-24T08:11:33.592959+0000","last_became_active":"2023-11-24T08:11:32.611663+0000","last_became_peered":"2023-11-24T08:11:32.611663+0000","last_unstale":"2023-11-24T08:11:33.592959+0000","last_undegraded":"2023-11-24T08:11:33.592959+0000","last_fullsized":"2023-11-24T08:11:33.592959+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:31:38.261860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592663+0000","last_change":"2023-11-24T08:11:32.603018+0000","last_active":"2023-11-24T08:11:33.592663+0000","last_peered":"2023-11-24T08:11:33.592663+0000","last_clean":"2023-11-24T08:11:33.592663+0000","last_became_active":"2023-11-24T08:11:32.600318+0000","last_became_peered":"2023-11-24T08:11:32.600318+0000","last_unstale":"2023-11-24T08:11:33.592663+0000","last_undegraded":"2023-11-24T08:11:33.592663+0000","last_fullsized":"2023-11-24T08:11:33.592663+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:20:35.713448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592130+0000","last_change":"2023-11-24T08:11:32.591767+0000","last_active":"2023-11-24T08:11:33.592130+0000","last_peered":"2023-11-24T08:11:33.592130+0000","last_clean":"2023-11-24T08:11:33.592130+0000","last_became_active":"2023-11-24T08:11:32.591370+0000","last_became_peered":"2023-11-24T08:11:32.591370+0000","last_unstale":"2023-11-24T08:11:33.592130+0000","last_undegraded":"2023-11-24T08:11:33.592130+0000","last_fullsized":"2023-11-24T08:11:33.592130+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:40.425598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357378+0000","last_change":"2023-11-24T08:11:32.612417+0000","last_active":"2023-11-24T08:11:34.357378+0000","last_peered":"2023-11-24T08:11:34.357378+0000","last_clean":"2023-11-24T08:11:34.357378+0000","last_became_active":"2023-11-24T08:11:32.608499+0000","last_became_peered":"2023-11-24T08:11:32.608499+0000","last_unstale":"2023-11-24T08:11:34.357378+0000","last_undegraded":"2023-11-24T08:11:34.357378+0000","last_fullsized":"2023-11-24T08:11:34.357378+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:31.627907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360463+0000","last_change":"2023-11-24T08:11:32.594460+0000","last_active":"2023-11-24T08:11:34.360463+0000","last_peered":"2023-11-24T08:11:34.360463+0000","last_clean":"2023-11-24T08:11:34.360463+0000","last_became_active":"2023-11-24T08:11:32.593677+0000","last_became_peered":"2023-11-24T08:11:32.593677+0000","last_unstale":"2023-11-24T08:11:34.360463+0000","last_undegraded":"2023-11-24T08:11:34.360463+0000","last_fullsized":"2023-11-24T08:11:34.360463+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:25:50.596601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357899+0000","last_change":"2023-11-24T08:11:32.618494+0000","last_active":"2023-11-24T08:11:34.357899+0000","last_peered":"2023-11-24T08:11:34.357899+0000","last_clean":"2023-11-24T08:11:34.357899+0000","last_became_active":"2023-11-24T08:11:32.614007+0000","last_became_peered":"2023-11-24T08:11:32.614007+0000","last_unstale":"2023-11-24T08:11:34.357899+0000","last_undegraded":"2023-11-24T08:11:34.357899+0000","last_fullsized":"2023-11-24T08:11:34.357899+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:03.125308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.589014+0000","last_change":"2023-11-24T08:11:32.612931+0000","last_active":"2023-11-24T08:11:33.589014+0000","last_peered":"2023-11-24T08:11:33.589014+0000","last_clean":"2023-11-24T08:11:33.589014+0000","last_became_active":"2023-11-24T08:11:32.612627+0000","last_became_peered":"2023-11-24T08:11:32.612627+0000","last_unstale":"2023-11-24T08:11:33.589014+0000","last_undegraded":"2023-11-24T08:11:33.589014+0000","last_fullsized":"2023-11-24T08:11:33.589014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:47.022588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360238+0000","last_change":"2023-11-24T08:11:32.606201+0000","last_active":"2023-11-24T08:11:34.360238+0000","last_peered":"2023-11-24T08:11:34.360238+0000","last_clean":"2023-11-24T08:11:34.360238+0000","last_became_active":"2023-11-24T08:11:32.605921+0000","last_became_peered":"2023-11-24T08:11:32.605921+0000","last_unstale":"2023-11-24T08:11:34.360238+0000","last_undegraded":"2023-11-24T08:11:34.360238+0000","last_fullsized":"2023-11-24T08:11:34.360238+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:00:54.957877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358806+0000","last_change":"2023-11-24T08:11:32.603520+0000","last_active":"2023-11-24T08:11:34.358806+0000","last_peered":"2023-11-24T08:11:34.358806+0000","last_clean":"2023-11-24T08:11:34.358806+0000","last_became_active":"2023-11-24T08:11:32.602251+0000","last_became_peered":"2023-11-24T08:11:32.602251+0000","last_unstale":"2023-11-24T08:11:34.358806+0000","last_undegraded":"2023-11-24T08:11:34.358806+0000","last_fullsized":"2023-11-24T08:11:34.358806+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:12.385999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358835+0000","last_change":"2023-11-24T08:11:32.604622+0000","last_active":"2023-11-24T08:11:34.358835+0000","last_peered":"2023-11-24T08:11:34.358835+0000","last_clean":"2023-11-24T08:11:34.358835+0000","last_became_active":"2023-11-24T08:11:32.604375+0000","last_became_peered":"2023-11-24T08:11:32.604375+0000","last_unstale":"2023-11-24T08:11:34.358835+0000","last_undegraded":"2023-11-24T08:11:34.358835+0000","last_fullsized":"2023-11-24T08:11:34.358835+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:34.092909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358452+0000","last_change":"2023-11-24T08:11:32.608905+0000","last_active":"2023-11-24T08:11:34.358452+0000","last_peered":"2023-11-24T08:11:34.358452+0000","last_clean":"2023-11-24T08:11:34.358452+0000","last_became_active":"2023-11-24T08:11:32.608557+0000","last_became_peered":"2023-11-24T08:11:32.608557+0000","last_unstale":"2023-11-24T08:11:34.358452+0000","last_undegraded":"2023-11-24T08:11:34.358452+0000","last_fullsized":"2023-11-24T08:11:34.358452+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:33.231023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.591337+0000","last_change":"2023-11-24T08:11:32.601952+0000","last_active":"2023-11-24T08:11:33.591337+0000","last_peered":"2023-11-24T08:11:33.591337+0000","last_clean":"2023-11-24T08:11:33.591337+0000","last_became_active":"2023-11-24T08:11:32.601505+0000","last_became_peered":"2023-11-24T08:11:32.601505+0000","last_unstale":"2023-11-24T08:11:33.591337+0000","last_undegraded":"2023-11-24T08:11:33.591337+0000","last_fullsized":"2023-11-24T08:11:33.591337+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:45:50.421563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.589511+0000","last_change":"2023-11-24T08:11:32.608354+0000","last_active":"2023-11-24T08:11:33.589511+0000","last_peered":"2023-11-24T08:11:33.589511+0000","last_clean":"2023-11-24T08:11:33.589511+0000","last_became_active":"2023-11-24T08:11:32.608064+0000","last_became_peered":"2023-11-24T08:11:32.608064+0000","last_unstale":"2023-11-24T08:11:33.589511+0000","last_undegraded":"2023-11-24T08:11:33.589511+0000","last_fullsized":"2023-11-24T08:11:33.589511+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:18:28.394528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356589+0000","last_change":"2023-11-24T08:11:32.593775+0000","last_active":"2023-11-24T08:11:34.356589+0000","last_peered":"2023-11-24T08:11:34.356589+0000","last_clean":"2023-11-24T08:11:34.356589+0000","last_became_active":"2023-11-24T08:11:32.592828+0000","last_became_peered":"2023-11-24T08:11:32.592828+0000","last_unstale":"2023-11-24T08:11:34.356589+0000","last_undegraded":"2023-11-24T08:11:34.356589+0000","last_fullsized":"2023-11-24T08:11:34.356589+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:17.638609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357173+0000","last_change":"2023-11-24T08:11:32.608772+0000","last_active":"2023-11-24T08:11:34.357173+0000","last_peered":"2023-11-24T08:11:34.357173+0000","last_clean":"2023-11-24T08:11:34.357173+0000","last_became_active":"2023-11-24T08:11:32.605365+0000","last_became_peered":"2023-11-24T08:11:32.605365+0000","last_unstale":"2023-11-24T08:11:34.357173+0000","last_undegraded":"2023-11-24T08:11:34.357173+0000","last_fullsized":"2023-11-24T08:11:34.357173+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:13.407851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.589365+0000","last_change":"2023-11-24T08:11:32.603539+0000","last_active":"2023-11-24T08:11:33.589365+0000","last_peered":"2023-11-24T08:11:33.589365+0000","last_clean":"2023-11-24T08:11:33.589365+0000","last_became_active":"2023-11-24T08:11:32.601157+0000","last_became_peered":"2023-11-24T08:11:32.601157+0000","last_unstale":"2023-11-24T08:11:33.589365+0000","last_undegraded":"2023-11-24T08:11:33.589365+0000","last_fullsized":"2023-11-24T08:11:33.589365+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:12:10.042548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359943+0000","last_change":"2023-11-24T08:11:32.600578+0000","last_active":"2023-11-24T08:11:34.359943+0000","last_peered":"2023-11-24T08:11:34.359943+0000","last_clean":"2023-11-24T08:11:34.359943+0000","last_became_active":"2023-11-24T08:11:32.597621+0000","last_became_peered":"2023-11-24T08:11:32.597621+0000","last_unstale":"2023-11-24T08:11:34.359943+0000","last_undegraded":"2023-11-24T08:11:34.359943+0000","last_fullsized":"2023-11-24T08:11:34.359943+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:54.612369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357194+0000","last_change":"2023-11-24T08:11:32.603100+0000","last_active":"2023-11-24T08:11:34.357194+0000","last_peered":"2023-11-24T08:11:34.357194+0000","last_clean":"2023-11-24T08:11:34.357194+0000","last_became_active":"2023-11-24T08:11:32.602762+0000","last_became_peered":"2023-11-24T08:11:32.602762+0000","last_unstale":"2023-11-24T08:11:34.357194+0000","last_undegraded":"2023-11-24T08:11:34.357194+0000","last_fullsized":"2023-11-24T08:11:34.357194+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:59.243822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.588580+0000","last_change":"2023-11-24T08:11:32.609766+0000","last_active":"2023-11-24T08:11:33.588580+0000","last_peered":"2023-11-24T08:11:33.588580+0000","last_clean":"2023-11-24T08:11:33.588580+0000","last_became_active":"2023-11-24T08:11:32.607328+0000","last_became_peered":"2023-11-24T08:11:32.607328+0000","last_unstale":"2023-11-24T08:11:33.588580+0000","last_undegraded":"2023-11-24T08:11:33.588580+0000","last_fullsized":"2023-11-24T08:11:33.588580+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:03.587779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356298+0000","last_change":"2023-11-24T08:11:32.596560+0000","last_active":"2023-11-24T08:11:34.356298+0000","last_peered":"2023-11-24T08:11:34.356298+0000","last_clean":"2023-11-24T08:11:34.356298+0000","last_became_active":"2023-11-24T08:11:32.596302+0000","last_became_peered":"2023-11-24T08:11:32.596302+0000","last_unstale":"2023-11-24T08:11:34.356298+0000","last_undegraded":"2023-11-24T08:11:34.356298+0000","last_fullsized":"2023-11-24T08:11:34.356298+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:28:29.071108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357851+0000","last_change":"2023-11-24T08:11:32.616628+0000","last_active":"2023-11-24T08:11:34.357851+0000","last_peered":"2023-11-24T08:11:34.357851+0000","last_clean":"2023-11-24T08:11:34.357851+0000","last_became_active":"2023-11-24T08:11:32.616284+0000","last_became_peered":"2023-11-24T08:11:32.616284+0000","last_unstale":"2023-11-24T08:11:34.357851+0000","last_undegraded":"2023-11-24T08:11:34.357851+0000","last_fullsized":"2023-11-24T08:11:34.357851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:55:34.092909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"61'4","reported_seq":16,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357854+0000","last_change":"2023-11-24T08:11:31.609572+0000","last_active":"2023-11-24T08:11:34.357854+0000","last_peered":"2023-11-24T08:11:34.357854+0000","last_clean":"2023-11-24T08:11:34.357854+0000","last_became_active":"2023-11-24T08:11:31.609121+0000","last_became_peered":"2023-11-24T08:11:31.609121+0000","last_unstale":"2023-11-24T08:11:34.357854+0000","last_undegraded":"2023-11-24T08:11:34.357854+0000","last_fullsized":"2023-11-24T08:11:34.357854+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:36:12.520451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360942+0000","last_change":"2023-11-24T08:11:32.602524+0000","last_active":"2023-11-24T08:11:34.360942+0000","last_peered":"2023-11-24T08:11:34.360942+0000","last_clean":"2023-11-24T08:11:34.360942+0000","last_became_active":"2023-11-24T08:11:32.602170+0000","last_became_peered":"2023-11-24T08:11:32.602170+0000","last_unstale":"2023-11-24T08:11:34.360942+0000","last_undegraded":"2023-11-24T08:11:34.360942+0000","last_fullsized":"2023-11-24T08:11:34.360942+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:58.981331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360773+0000","last_change":"2023-11-24T08:11:31.608444+0000","last_active":"2023-11-24T08:11:34.360773+0000","last_peered":"2023-11-24T08:11:34.360773+0000","last_clean":"2023-11-24T08:11:34.360773+0000","last_became_active":"2023-11-24T08:11:31.608170+0000","last_became_peered":"2023-11-24T08:11:31.608170+0000","last_unstale":"2023-11-24T08:11:34.360773+0000","last_undegraded":"2023-11-24T08:11:34.360773+0000","last_fullsized":"2023-11-24T08:11:34.360773+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:53.655434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356851+0000","last_change":"2023-11-24T08:11:32.613441+0000","last_active":"2023-11-24T08:11:34.356851+0000","last_peered":"2023-11-24T08:11:34.356851+0000","last_clean":"2023-11-24T08:11:34.356851+0000","last_became_active":"2023-11-24T08:11:32.613190+0000","last_became_peered":"2023-11-24T08:11:32.613190+0000","last_unstale":"2023-11-24T08:11:34.356851+0000","last_undegraded":"2023-11-24T08:11:34.356851+0000","last_fullsized":"2023-11-24T08:11:34.356851+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:46:29.440774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357583+0000","last_change":"2023-11-24T08:11:31.599196+0000","last_active":"2023-11-24T08:11:34.357583+0000","last_peered":"2023-11-24T08:11:34.357583+0000","last_clean":"2023-11-24T08:11:34.357583+0000","last_became_active":"2023-11-24T08:11:31.598738+0000","last_became_peered":"2023-11-24T08:11:31.598738+0000","last_unstale":"2023-11-24T08:11:34.357583+0000","last_undegraded":"2023-11-24T08:11:34.357583+0000","last_fullsized":"2023-11-24T08:11:34.357583+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:39:58.228375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357334+0000","last_change":"2023-11-24T08:11:32.603325+0000","last_active":"2023-11-24T08:11:34.357334+0000","last_peered":"2023-11-24T08:11:34.357334+0000","last_clean":"2023-11-24T08:11:34.357334+0000","last_became_active":"2023-11-24T08:11:32.602346+0000","last_became_peered":"2023-11-24T08:11:32.602346+0000","last_unstale":"2023-11-24T08:11:34.357334+0000","last_undegraded":"2023-11-24T08:11:34.357334+0000","last_fullsized":"2023-11-24T08:11:34.357334+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:31.638552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356634+0000","last_change":"2023-11-24T08:11:31.610567+0000","last_active":"2023-11-24T08:11:34.356634+0000","last_peered":"2023-11-24T08:11:34.356634+0000","last_clean":"2023-11-24T08:11:34.356634+0000","last_became_active":"2023-11-24T08:11:31.610253+0000","last_became_peered":"2023-11-24T08:11:31.610253+0000","last_unstale":"2023-11-24T08:11:34.356634+0000","last_undegraded":"2023-11-24T08:11:34.356634+0000","last_fullsized":"2023-11-24T08:11:34.356634+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:44:15.076112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358307+0000","last_change":"2023-11-24T08:11:32.604544+0000","last_active":"2023-11-24T08:11:34.358307+0000","last_peered":"2023-11-24T08:11:34.358307+0000","last_clean":"2023-11-24T08:11:34.358307+0000","last_became_active":"2023-11-24T08:11:32.604265+0000","last_became_peered":"2023-11-24T08:11:32.604265+0000","last_unstale":"2023-11-24T08:11:34.358307+0000","last_undegraded":"2023-11-24T08:11:34.358307+0000","last_fullsized":"2023-11-24T08:11:34.358307+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:13.407851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.593161+0000","last_change":"2023-11-24T08:11:31.600011+0000","last_active":"2023-11-24T08:11:33.593161+0000","last_peered":"2023-11-24T08:11:33.593161+0000","last_clean":"2023-11-24T08:11:33.593161+0000","last_became_active":"2023-11-24T08:11:31.599071+0000","last_became_peered":"2023-11-24T08:11:31.599071+0000","last_unstale":"2023-11-24T08:11:33.593161+0000","last_undegraded":"2023-11-24T08:11:33.593161+0000","last_fullsized":"2023-11-24T08:11:33.593161+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:23:25.601533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358584+0000","last_change":"2023-11-24T08:11:32.611576+0000","last_active":"2023-11-24T08:11:34.358584+0000","last_peered":"2023-11-24T08:11:34.358584+0000","last_clean":"2023-11-24T08:11:34.358584+0000","last_became_active":"2023-11-24T08:11:32.611245+0000","last_became_peered":"2023-11-24T08:11:32.611245+0000","last_unstale":"2023-11-24T08:11:34.358584+0000","last_undegraded":"2023-11-24T08:11:34.358584+0000","last_fullsized":"2023-11-24T08:11:34.358584+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:19:31.638552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357314+0000","last_change":"2023-11-24T08:11:31.605803+0000","last_active":"2023-11-24T08:11:34.357314+0000","last_peered":"2023-11-24T08:11:34.357314+0000","last_clean":"2023-11-24T08:11:34.357314+0000","last_became_active":"2023-11-24T08:11:31.605481+0000","last_became_peered":"2023-11-24T08:11:31.605481+0000","last_unstale":"2023-11-24T08:11:34.357314+0000","last_undegraded":"2023-11-24T08:11:34.357314+0000","last_fullsized":"2023-11-24T08:11:34.357314+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:21.350368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.591741+0000","last_change":"2023-11-24T08:11:32.594345+0000","last_active":"2023-11-24T08:11:33.591741+0000","last_peered":"2023-11-24T08:11:33.591741+0000","last_clean":"2023-11-24T08:11:33.591741+0000","last_became_active":"2023-11-24T08:11:32.594037+0000","last_became_peered":"2023-11-24T08:11:32.594037+0000","last_unstale":"2023-11-24T08:11:33.591741+0000","last_undegraded":"2023-11-24T08:11:33.591741+0000","last_fullsized":"2023-11-24T08:11:33.591741+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:56:34.903605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.355735+0000","last_change":"2023-11-24T08:11:31.605202+0000","last_active":"2023-11-24T08:11:34.355735+0000","last_peered":"2023-11-24T08:11:34.355735+0000","last_clean":"2023-11-24T08:11:34.355735+0000","last_became_active":"2023-11-24T08:11:31.604872+0000","last_became_peered":"2023-11-24T08:11:31.604872+0000","last_unstale":"2023-11-24T08:11:34.355735+0000","last_undegraded":"2023-11-24T08:11:34.355735+0000","last_fullsized":"2023-11-24T08:11:34.355735+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:45:27.714412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358692+0000","last_change":"2023-11-24T08:11:32.600967+0000","last_active":"2023-11-24T08:11:34.358692+0000","last_peered":"2023-11-24T08:11:34.358692+0000","last_clean":"2023-11-24T08:11:34.358692+0000","last_became_active":"2023-11-24T08:11:32.600551+0000","last_became_peered":"2023-11-24T08:11:32.600551+0000","last_unstale":"2023-11-24T08:11:34.358692+0000","last_undegraded":"2023-11-24T08:11:34.358692+0000","last_fullsized":"2023-11-24T08:11:34.358692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:33.685432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.591618+0000","last_change":"2023-11-24T08:11:31.599773+0000","last_active":"2023-11-24T08:11:33.591618+0000","last_peered":"2023-11-24T08:11:33.591618+0000","last_clean":"2023-11-24T08:11:33.591618+0000","last_became_active":"2023-11-24T08:11:31.599506+0000","last_became_peered":"2023-11-24T08:11:31.599506+0000","last_unstale":"2023-11-24T08:11:33.591618+0000","last_undegraded":"2023-11-24T08:11:33.591618+0000","last_fullsized":"2023-11-24T08:11:33.591618+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:44.656508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592780+0000","last_change":"2023-11-24T08:11:32.595385+0000","last_active":"2023-11-24T08:11:33.592780+0000","last_peered":"2023-11-24T08:11:33.592780+0000","last_clean":"2023-11-24T08:11:33.592780+0000","last_became_active":"2023-11-24T08:11:32.594911+0000","last_became_peered":"2023-11-24T08:11:32.594911+0000","last_unstale":"2023-11-24T08:11:33.592780+0000","last_undegraded":"2023-11-24T08:11:33.592780+0000","last_fullsized":"2023-11-24T08:11:33.592780+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:23.315972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359008+0000","last_change":"2023-11-24T08:11:31.609788+0000","last_active":"2023-11-24T08:11:34.359008+0000","last_peered":"2023-11-24T08:11:34.359008+0000","last_clean":"2023-11-24T08:11:34.359008+0000","last_became_active":"2023-11-24T08:11:31.608773+0000","last_became_peered":"2023-11-24T08:11:31.608773+0000","last_unstale":"2023-11-24T08:11:34.359008+0000","last_undegraded":"2023-11-24T08:11:34.359008+0000","last_fullsized":"2023-11-24T08:11:34.359008+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:39.410151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359066+0000","last_change":"2023-11-24T08:11:32.618056+0000","last_active":"2023-11-24T08:11:34.359066+0000","last_peered":"2023-11-24T08:11:34.359066+0000","last_clean":"2023-11-24T08:11:34.359066+0000","last_became_active":"2023-11-24T08:11:32.615333+0000","last_became_peered":"2023-11-24T08:11:32.615333+0000","last_unstale":"2023-11-24T08:11:34.359066+0000","last_undegraded":"2023-11-24T08:11:34.359066+0000","last_fullsized":"2023-11-24T08:11:34.359066+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:22.365814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.361002+0000","last_change":"2023-11-24T08:11:31.601929+0000","last_active":"2023-11-24T08:11:34.361002+0000","last_peered":"2023-11-24T08:11:34.361002+0000","last_clean":"2023-11-24T08:11:34.361002+0000","last_became_active":"2023-11-24T08:11:31.601672+0000","last_became_peered":"2023-11-24T08:11:31.601672+0000","last_unstale":"2023-11-24T08:11:34.361002+0000","last_undegraded":"2023-11-24T08:11:34.361002+0000","last_fullsized":"2023-11-24T08:11:34.361002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:14.717611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358866+0000","last_change":"2023-11-24T08:11:32.600232+0000","last_active":"2023-11-24T08:11:34.358866+0000","last_peered":"2023-11-24T08:11:34.358866+0000","last_clean":"2023-11-24T08:11:34.358866+0000","last_became_active":"2023-11-24T08:11:32.599512+0000","last_became_peered":"2023-11-24T08:11:32.599512+0000","last_unstale":"2023-11-24T08:11:34.358866+0000","last_undegraded":"2023-11-24T08:11:34.358866+0000","last_fullsized":"2023-11-24T08:11:34.358866+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:36.909254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"61'3","reported_seq":15,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358178+0000","last_change":"2023-11-24T08:11:31.608919+0000","last_active":"2023-11-24T08:11:34.358178+0000","last_peered":"2023-11-24T08:11:34.358178+0000","last_clean":"2023-11-24T08:11:34.358178+0000","last_became_active":"2023-11-24T08:11:31.608619+0000","last_became_peered":"2023-11-24T08:11:31.608619+0000","last_unstale":"2023-11-24T08:11:34.358178+0000","last_undegraded":"2023-11-24T08:11:34.358178+0000","last_fullsized":"2023-11-24T08:11:34.358178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:41:22.300525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357829+0000","last_change":"2023-11-24T08:11:32.618204+0000","last_active":"2023-11-24T08:11:34.357829+0000","last_peered":"2023-11-24T08:11:34.357829+0000","last_clean":"2023-11-24T08:11:34.357829+0000","last_became_active":"2023-11-24T08:11:32.616497+0000","last_became_peered":"2023-11-24T08:11:32.616497+0000","last_unstale":"2023-11-24T08:11:34.357829+0000","last_undegraded":"2023-11-24T08:11:34.357829+0000","last_fullsized":"2023-11-24T08:11:34.357829+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:03:37.122838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"61'6","reported_seq":18,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592525+0000","last_change":"2023-11-24T08:11:31.605389+0000","last_active":"2023-11-24T08:11:33.592525+0000","last_peered":"2023-11-24T08:11:33.592525+0000","last_clean":"2023-11-24T08:11:33.592525+0000","last_became_active":"2023-11-24T08:11:31.602109+0000","last_became_peered":"2023-11-24T08:11:31.602109+0000","last_unstale":"2023-11-24T08:11:33.592525+0000","last_undegraded":"2023-11-24T08:11:33.592525+0000","last_fullsized":"2023-11-24T08:11:33.592525+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:46:54.509472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.355799+0000","last_change":"2023-11-24T08:11:32.604997+0000","last_active":"2023-11-24T08:11:34.355799+0000","last_peered":"2023-11-24T08:11:34.355799+0000","last_clean":"2023-11-24T08:11:34.355799+0000","last_became_active":"2023-11-24T08:11:32.604649+0000","last_became_peered":"2023-11-24T08:11:32.604649+0000","last_unstale":"2023-11-24T08:11:34.355799+0000","last_undegraded":"2023-11-24T08:11:34.355799+0000","last_fullsized":"2023-11-24T08:11:34.355799+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:46:49.472315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359723+0000","last_change":"2023-11-24T08:11:31.598537+0000","last_active":"2023-11-24T08:11:34.359723+0000","last_peered":"2023-11-24T08:11:34.359723+0000","last_clean":"2023-11-24T08:11:34.359723+0000","last_became_active":"2023-11-24T08:11:31.598257+0000","last_became_peered":"2023-11-24T08:11:31.598257+0000","last_unstale":"2023-11-24T08:11:34.359723+0000","last_undegraded":"2023-11-24T08:11:34.359723+0000","last_fullsized":"2023-11-24T08:11:34.359723+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:08:21.350368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358120+0000","last_change":"2023-11-24T08:11:32.603690+0000","last_active":"2023-11-24T08:11:34.358120+0000","last_peered":"2023-11-24T08:11:34.358120+0000","last_clean":"2023-11-24T08:11:34.358120+0000","last_became_active":"2023-11-24T08:11:32.602145+0000","last_became_peered":"2023-11-24T08:11:32.602145+0000","last_unstale":"2023-11-24T08:11:34.358120+0000","last_undegraded":"2023-11-24T08:11:34.358120+0000","last_fullsized":"2023-11-24T08:11:34.358120+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:53:27.246095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592141+0000","last_change":"2023-11-24T08:11:31.606140+0000","last_active":"2023-11-24T08:11:33.592141+0000","last_peered":"2023-11-24T08:11:33.592141+0000","last_clean":"2023-11-24T08:11:33.592141+0000","last_became_active":"2023-11-24T08:11:31.605797+0000","last_became_peered":"2023-11-24T08:11:31.605797+0000","last_unstale":"2023-11-24T08:11:33.592141+0000","last_undegraded":"2023-11-24T08:11:33.592141+0000","last_fullsized":"2023-11-24T08:11:33.592141+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:53:16.334573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.591621+0000","last_change":"2023-11-24T08:11:32.606948+0000","last_active":"2023-11-24T08:11:33.591621+0000","last_peered":"2023-11-24T08:11:33.591621+0000","last_clean":"2023-11-24T08:11:33.591621+0000","last_became_active":"2023-11-24T08:11:32.606381+0000","last_became_peered":"2023-11-24T08:11:32.606381+0000","last_unstale":"2023-11-24T08:11:33.591621+0000","last_undegraded":"2023-11-24T08:11:33.591621+0000","last_fullsized":"2023-11-24T08:11:33.591621+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:14.515895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.355737+0000","last_change":"2023-11-24T08:11:31.607926+0000","last_active":"2023-11-24T08:11:34.355737+0000","last_peered":"2023-11-24T08:11:34.355737+0000","last_clean":"2023-11-24T08:11:34.355737+0000","last_became_active":"2023-11-24T08:11:31.605279+0000","last_became_peered":"2023-11-24T08:11:31.605279+0000","last_unstale":"2023-11-24T08:11:34.355737+0000","last_undegraded":"2023-11-24T08:11:34.355737+0000","last_fullsized":"2023-11-24T08:11:34.355737+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:06:12.957350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357476+0000","last_change":"2023-11-24T08:11:32.608321+0000","last_active":"2023-11-24T08:11:34.357476+0000","last_peered":"2023-11-24T08:11:34.357476+0000","last_clean":"2023-11-24T08:11:34.357476+0000","last_became_active":"2023-11-24T08:11:32.605957+0000","last_became_peered":"2023-11-24T08:11:32.605957+0000","last_unstale":"2023-11-24T08:11:34.357476+0000","last_undegraded":"2023-11-24T08:11:34.357476+0000","last_fullsized":"2023-11-24T08:11:34.357476+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:15.676646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"61'8","reported_seq":20,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357209+0000","last_change":"2023-11-24T08:11:31.612361+0000","last_active":"2023-11-24T08:11:34.357209+0000","last_peered":"2023-11-24T08:11:34.357209+0000","last_clean":"2023-11-24T08:11:34.357209+0000","last_became_active":"2023-11-24T08:11:31.612067+0000","last_became_peered":"2023-11-24T08:11:31.612067+0000","last_unstale":"2023-11-24T08:11:34.357209+0000","last_undegraded":"2023-11-24T08:11:34.357209+0000","last_fullsized":"2023-11-24T08:11:34.357209+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:36:50.818659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357993+0000","last_change":"2023-11-24T08:11:32.614938+0000","last_active":"2023-11-24T08:11:34.357993+0000","last_peered":"2023-11-24T08:11:34.357993+0000","last_clean":"2023-11-24T08:11:34.357993+0000","last_became_active":"2023-11-24T08:11:32.613504+0000","last_became_peered":"2023-11-24T08:11:32.613504+0000","last_unstale":"2023-11-24T08:11:34.357993+0000","last_undegraded":"2023-11-24T08:11:34.357993+0000","last_fullsized":"2023-11-24T08:11:34.357993+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:47.437569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360786+0000","last_change":"2023-11-24T08:11:31.598780+0000","last_active":"2023-11-24T08:11:34.360786+0000","last_peered":"2023-11-24T08:11:34.360786+0000","last_clean":"2023-11-24T08:11:34.360786+0000","last_became_active":"2023-11-24T08:11:31.598453+0000","last_became_peered":"2023-11-24T08:11:31.598453+0000","last_unstale":"2023-11-24T08:11:34.360786+0000","last_undegraded":"2023-11-24T08:11:34.360786+0000","last_fullsized":"2023-11-24T08:11:34.360786+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:07.580886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356979+0000","last_change":"2023-11-24T08:11:32.600437+0000","last_active":"2023-11-24T08:11:34.356979+0000","last_peered":"2023-11-24T08:11:34.356979+0000","last_clean":"2023-11-24T08:11:34.356979+0000","last_became_active":"2023-11-24T08:11:32.600134+0000","last_became_peered":"2023-11-24T08:11:32.600134+0000","last_unstale":"2023-11-24T08:11:34.356979+0000","last_undegraded":"2023-11-24T08:11:34.356979+0000","last_fullsized":"2023-11-24T08:11:34.356979+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:42:02.553991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358062+0000","last_change":"2023-11-24T08:11:31.602434+0000","last_active":"2023-11-24T08:11:34.358062+0000","last_peered":"2023-11-24T08:11:34.358062+0000","last_clean":"2023-11-24T08:11:34.358062+0000","last_became_active":"2023-11-24T08:11:31.602133+0000","last_became_peered":"2023-11-24T08:11:31.602133+0000","last_unstale":"2023-11-24T08:11:34.358062+0000","last_undegraded":"2023-11-24T08:11:34.358062+0000","last_fullsized":"2023-11-24T08:11:34.358062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:28.934093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.360439+0000","last_change":"2023-11-24T08:11:32.587592+0000","last_active":"2023-11-24T08:11:34.360439+0000","last_peered":"2023-11-24T08:11:34.360439+0000","last_clean":"2023-11-24T08:11:34.360439+0000","last_became_active":"2023-11-24T08:11:32.582660+0000","last_became_peered":"2023-11-24T08:11:32.582660+0000","last_unstale":"2023-11-24T08:11:34.360439+0000","last_undegraded":"2023-11-24T08:11:34.360439+0000","last_fullsized":"2023-11-24T08:11:34.360439+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:42:08.811685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"61'1","reported_seq":13,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356458+0000","last_change":"2023-11-24T08:11:31.611241+0000","last_active":"2023-11-24T08:11:34.356458+0000","last_peered":"2023-11-24T08:11:34.356458+0000","last_clean":"2023-11-24T08:11:34.356458+0000","last_became_active":"2023-11-24T08:11:31.610771+0000","last_became_peered":"2023-11-24T08:11:31.610771+0000","last_unstale":"2023-11-24T08:11:34.356458+0000","last_undegraded":"2023-11-24T08:11:34.356458+0000","last_fullsized":"2023-11-24T08:11:34.356458+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:33:52.179482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358135+0000","last_change":"2023-11-24T08:11:32.618402+0000","last_active":"2023-11-24T08:11:34.358135+0000","last_peered":"2023-11-24T08:11:34.358135+0000","last_clean":"2023-11-24T08:11:34.358135+0000","last_became_active":"2023-11-24T08:11:32.617328+0000","last_became_peered":"2023-11-24T08:11:32.617328+0000","last_unstale":"2023-11-24T08:11:34.358135+0000","last_undegraded":"2023-11-24T08:11:34.358135+0000","last_fullsized":"2023-11-24T08:11:34.358135+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:29.949539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356948+0000","last_change":"2023-11-24T08:11:31.602230+0000","last_active":"2023-11-24T08:11:34.356948+0000","last_peered":"2023-11-24T08:11:34.356948+0000","last_clean":"2023-11-24T08:11:34.356948+0000","last_became_active":"2023-11-24T08:11:31.601927+0000","last_became_peered":"2023-11-24T08:11:31.601927+0000","last_unstale":"2023-11-24T08:11:34.356948+0000","last_undegraded":"2023-11-24T08:11:34.356948+0000","last_fullsized":"2023-11-24T08:11:34.356948+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:24:46.422122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356139+0000","last_change":"2023-11-24T08:11:32.598215+0000","last_active":"2023-11-24T08:11:34.356139+0000","last_peered":"2023-11-24T08:11:34.356139+0000","last_clean":"2023-11-24T08:11:34.356139+0000","last_became_active":"2023-11-24T08:11:32.597673+0000","last_became_peered":"2023-11-24T08:11:32.597673+0000","last_unstale":"2023-11-24T08:11:34.356139+0000","last_undegraded":"2023-11-24T08:11:34.356139+0000","last_fullsized":"2023-11-24T08:11:34.356139+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:25.796185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359926+0000","last_change":"2023-11-24T08:11:31.597223+0000","last_active":"2023-11-24T08:11:34.359926+0000","last_peered":"2023-11-24T08:11:34.359926+0000","last_clean":"2023-11-24T08:11:34.359926+0000","last_became_active":"2023-11-24T08:11:31.596938+0000","last_became_peered":"2023-11-24T08:11:31.596938+0000","last_unstale":"2023-11-24T08:11:34.359926+0000","last_undegraded":"2023-11-24T08:11:34.359926+0000","last_fullsized":"2023-11-24T08:11:34.359926+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:46:28.425327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356664+0000","last_change":"2023-11-24T08:11:32.610157+0000","last_active":"2023-11-24T08:11:34.356664+0000","last_peered":"2023-11-24T08:11:34.356664+0000","last_clean":"2023-11-24T08:11:34.356664+0000","last_became_active":"2023-11-24T08:11:32.609891+0000","last_became_peered":"2023-11-24T08:11:32.609891+0000","last_unstale":"2023-11-24T08:11:34.356664+0000","last_undegraded":"2023-11-24T08:11:34.356664+0000","last_fullsized":"2023-11-24T08:11:34.356664+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:07.670199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357312+0000","last_change":"2023-11-24T08:11:31.604110+0000","last_active":"2023-11-24T08:11:34.357312+0000","last_peered":"2023-11-24T08:11:34.357312+0000","last_clean":"2023-11-24T08:11:34.357312+0000","last_became_active":"2023-11-24T08:11:31.602553+0000","last_became_peered":"2023-11-24T08:11:31.602553+0000","last_unstale":"2023-11-24T08:11:34.357312+0000","last_undegraded":"2023-11-24T08:11:34.357312+0000","last_fullsized":"2023-11-24T08:11:34.357312+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:18.676968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356390+0000","last_change":"2023-11-24T08:11:32.616025+0000","last_active":"2023-11-24T08:11:34.356390+0000","last_peered":"2023-11-24T08:11:34.356390+0000","last_clean":"2023-11-24T08:11:34.356390+0000","last_became_active":"2023-11-24T08:11:32.615762+0000","last_became_peered":"2023-11-24T08:11:32.615762+0000","last_unstale":"2023-11-24T08:11:34.356390+0000","last_undegraded":"2023-11-24T08:11:34.356390+0000","last_fullsized":"2023-11-24T08:11:34.356390+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:48:08.596333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.355517+0000","last_change":"2023-11-24T08:11:31.607415+0000","last_active":"2023-11-24T08:11:34.355517+0000","last_peered":"2023-11-24T08:11:34.355517+0000","last_clean":"2023-11-24T08:11:34.355517+0000","last_became_active":"2023-11-24T08:11:31.606741+0000","last_became_peered":"2023-11-24T08:11:31.606741+0000","last_unstale":"2023-11-24T08:11:34.355517+0000","last_undegraded":"2023-11-24T08:11:34.355517+0000","last_fullsized":"2023-11-24T08:11:34.355517+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:50:30.440723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356756+0000","last_change":"2023-11-24T08:11:32.597379+0000","last_active":"2023-11-24T08:11:34.356756+0000","last_peered":"2023-11-24T08:11:34.356756+0000","last_clean":"2023-11-24T08:11:34.356756+0000","last_became_active":"2023-11-24T08:11:32.595600+0000","last_became_peered":"2023-11-24T08:11:32.595600+0000","last_unstale":"2023-11-24T08:11:34.356756+0000","last_undegraded":"2023-11-24T08:11:34.356756+0000","last_fullsized":"2023-11-24T08:11:34.356756+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:07:57.231725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"61'2","reported_seq":14,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.361154+0000","last_change":"2023-11-24T08:11:31.587099+0000","last_active":"2023-11-24T08:11:34.361154+0000","last_peered":"2023-11-24T08:11:34.361154+0000","last_clean":"2023-11-24T08:11:34.361154+0000","last_became_active":"2023-11-24T08:11:31.586796+0000","last_became_peered":"2023-11-24T08:11:31.586796+0000","last_unstale":"2023-11-24T08:11:34.361154+0000","last_undegraded":"2023-11-24T08:11:34.361154+0000","last_fullsized":"2023-11-24T08:11:34.361154+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:10:06.468969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357910+0000","last_change":"2023-11-24T08:11:32.605435+0000","last_active":"2023-11-24T08:11:34.357910+0000","last_peered":"2023-11-24T08:11:34.357910+0000","last_clean":"2023-11-24T08:11:34.357910+0000","last_became_active":"2023-11-24T08:11:32.605139+0000","last_became_peered":"2023-11-24T08:11:32.605139+0000","last_unstale":"2023-11-24T08:11:34.357910+0000","last_undegraded":"2023-11-24T08:11:34.357910+0000","last_fullsized":"2023-11-24T08:11:34.357910+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:52:10.328843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358084+0000","last_change":"2023-11-24T08:11:31.603608+0000","last_active":"2023-11-24T08:11:34.358084+0000","last_peered":"2023-11-24T08:11:34.358084+0000","last_clean":"2023-11-24T08:11:34.358084+0000","last_became_active":"2023-11-24T08:11:31.603315+0000","last_became_peered":"2023-11-24T08:11:31.603315+0000","last_unstale":"2023-11-24T08:11:34.358084+0000","last_undegraded":"2023-11-24T08:11:34.358084+0000","last_fullsized":"2023-11-24T08:11:34.358084+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:06:32.215576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359241+0000","last_change":"2023-11-24T08:11:32.593945+0000","last_active":"2023-11-24T08:11:34.359241+0000","last_peered":"2023-11-24T08:11:34.359241+0000","last_clean":"2023-11-24T08:11:34.359241+0000","last_became_active":"2023-11-24T08:11:32.593432+0000","last_became_peered":"2023-11-24T08:11:32.593432+0000","last_unstale":"2023-11-24T08:11:34.359241+0000","last_undegraded":"2023-11-24T08:11:34.359241+0000","last_fullsized":"2023-11-24T08:11:34.359241+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:01:21.679802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.593100+0000","last_change":"2023-11-24T08:11:31.599746+0000","last_active":"2023-11-24T08:11:33.593100+0000","last_peered":"2023-11-24T08:11:33.593100+0000","last_clean":"2023-11-24T08:11:33.593100+0000","last_became_active":"2023-11-24T08:11:31.599459+0000","last_became_peered":"2023-11-24T08:11:31.599459+0000","last_unstale":"2023-11-24T08:11:33.593100+0000","last_undegraded":"2023-11-24T08:11:33.593100+0000","last_fullsized":"2023-11-24T08:11:33.593100+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:46:12.977558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357537+0000","last_change":"2023-11-24T08:11:32.599306+0000","last_active":"2023-11-24T08:11:34.357537+0000","last_peered":"2023-11-24T08:11:34.357537+0000","last_clean":"2023-11-24T08:11:34.357537+0000","last_became_active":"2023-11-24T08:11:32.599046+0000","last_became_peered":"2023-11-24T08:11:32.599046+0000","last_unstale":"2023-11-24T08:11:34.357537+0000","last_undegraded":"2023-11-24T08:11:34.357537+0000","last_fullsized":"2023-11-24T08:11:34.357537+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:33.685432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.356902+0000","last_change":"2023-11-24T08:11:31.611485+0000","last_active":"2023-11-24T08:11:34.356902+0000","last_peered":"2023-11-24T08:11:34.356902+0000","last_clean":"2023-11-24T08:11:34.356902+0000","last_became_active":"2023-11-24T08:11:31.611181+0000","last_became_peered":"2023-11-24T08:11:31.611181+0000","last_unstale":"2023-11-24T08:11:34.356902+0000","last_undegraded":"2023-11-24T08:11:34.356902+0000","last_fullsized":"2023-11-24T08:11:34.356902+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:08:04.385355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358631+0000","last_change":"2023-11-24T08:11:32.597050+0000","last_active":"2023-11-24T08:11:34.358631+0000","last_peered":"2023-11-24T08:11:34.358631+0000","last_clean":"2023-11-24T08:11:34.358631+0000","last_became_active":"2023-11-24T08:11:32.596757+0000","last_became_peered":"2023-11-24T08:11:32.596757+0000","last_unstale":"2023-11-24T08:11:34.358631+0000","last_undegraded":"2023-11-24T08:11:34.358631+0000","last_fullsized":"2023-11-24T08:11:34.358631+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:19:31.627907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.355816+0000","last_change":"2023-11-24T08:11:31.605941+0000","last_active":"2023-11-24T08:11:34.355816+0000","last_peered":"2023-11-24T08:11:34.355816+0000","last_clean":"2023-11-24T08:11:34.355816+0000","last_became_active":"2023-11-24T08:11:31.605664+0000","last_became_peered":"2023-11-24T08:11:31.605664+0000","last_unstale":"2023-11-24T08:11:34.355816+0000","last_undegraded":"2023-11-24T08:11:34.355816+0000","last_fullsized":"2023-11-24T08:11:34.355816+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T20:07:26.797319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.358467+0000","last_change":"2023-11-24T08:11:32.615959+0000","last_active":"2023-11-24T08:11:34.358467+0000","last_peered":"2023-11-24T08:11:34.358467+0000","last_clean":"2023-11-24T08:11:34.358467+0000","last_became_active":"2023-11-24T08:11:32.613079+0000","last_became_peered":"2023-11-24T08:11:32.613079+0000","last_unstale":"2023-11-24T08:11:34.358467+0000","last_undegraded":"2023-11-24T08:11:34.358467+0000","last_fullsized":"2023-11-24T08:11:34.358467+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:54.670881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.593113+0000","last_change":"2023-11-24T08:11:31.601306+0000","last_active":"2023-11-24T08:11:33.593113+0000","last_peered":"2023-11-24T08:11:33.593113+0000","last_clean":"2023-11-24T08:11:33.593113+0000","last_became_active":"2023-11-24T08:11:31.600873+0000","last_became_peered":"2023-11-24T08:11:31.600873+0000","last_unstale":"2023-11-24T08:11:33.593113+0000","last_undegraded":"2023-11-24T08:11:33.593113+0000","last_fullsized":"2023-11-24T08:11:33.593113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:17:35.352637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.355692+0000","last_change":"2023-11-24T08:11:32.594025+0000","last_active":"2023-11-24T08:11:34.355692+0000","last_peered":"2023-11-24T08:11:34.355692+0000","last_clean":"2023-11-24T08:11:34.355692+0000","last_became_active":"2023-11-24T08:11:32.593754+0000","last_became_peered":"2023-11-24T08:11:32.593754+0000","last_unstale":"2023-11-24T08:11:34.355692+0000","last_undegraded":"2023-11-24T08:11:34.355692+0000","last_fullsized":"2023-11-24T08:11:34.355692+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:33:19.005228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.357062+0000","last_change":"2023-11-24T08:11:31.596200+0000","last_active":"2023-11-24T08:11:34.357062+0000","last_peered":"2023-11-24T08:11:34.357062+0000","last_clean":"2023-11-24T08:11:34.357062+0000","last_became_active":"2023-11-24T08:11:31.595247+0000","last_became_peered":"2023-11-24T08:11:31.595247+0000","last_unstale":"2023-11-24T08:11:34.357062+0000","last_undegraded":"2023-11-24T08:11:34.357062+0000","last_fullsized":"2023-11-24T08:11:34.357062+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:17:40.656331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359076+0000","last_change":"2023-11-24T08:11:32.592613+0000","last_active":"2023-11-24T08:11:34.359076+0000","last_peered":"2023-11-24T08:11:34.359076+0000","last_clean":"2023-11-24T08:11:34.359076+0000","last_became_active":"2023-11-24T08:11:32.592345+0000","last_became_peered":"2023-11-24T08:11:32.592345+0000","last_unstale":"2023-11-24T08:11:34.359076+0000","last_undegraded":"2023-11-24T08:11:34.359076+0000","last_fullsized":"2023-11-24T08:11:34.359076+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:47:45.743337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359597+0000","last_change":"2023-11-24T08:11:31.608775+0000","last_active":"2023-11-24T08:11:34.359597+0000","last_peered":"2023-11-24T08:11:34.359597+0000","last_clean":"2023-11-24T08:11:34.359597+0000","last_became_active":"2023-11-24T08:11:31.608484+0000","last_became_peered":"2023-11-24T08:11:31.608484+0000","last_unstale":"2023-11-24T08:11:34.359597+0000","last_undegraded":"2023-11-24T08:11:34.359597+0000","last_fullsized":"2023-11-24T08:11:34.359597+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:32:28.934093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":11,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:34.359360+0000","last_change":"2023-11-24T08:11:32.607248+0000","last_active":"2023-11-24T08:11:34.359360+0000","last_peered":"2023-11-24T08:11:34.359360+0000","last_clean":"2023-11-24T08:11:34.359360+0000","last_became_active":"2023-11-24T08:11:32.606675+0000","last_became_peered":"2023-11-24T08:11:32.606675+0000","last_unstale":"2023-11-24T08:11:34.359360+0000","last_undegraded":"2023-11-24T08:11:34.359360+0000","last_fullsized":"2023-11-24T08:11:34.359360+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:31.556545+0000","last_clean_scrub_stamp":"2023-11-24T08:11:31.556545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:51:50.800975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-24T08:11:33.592222+0000","last_change":"2023-11-24T08:11:31.610400+0000","last_active":"2023-11-24T08:11:33.592222+0000","last_peered":"2023-11-24T08:11:33.592222+0000","last_clean":"2023-11-24T08:11:33.592222+0000","last_became_active":"2023-11-24T08:11:31.610107+0000","last_became_peered":"2023-11-24T08:11:31.610107+0000","last_unstale":"2023-11-24T08:11:33.592222+0000","last_undegraded":"2023-11-24T08:11:33.592222+0000","last_fullsized":"2023-11-24T08:11:33.592222+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:11:30.541099+0000","last_clean_scrub_stamp":"2023-11-24T08:11:30.541099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:19:57.965885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":197,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-11-24T08:11:32.592553+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:11:32.592553+0000","last_peered":"2023-11-24T08:11:32.592553+0000","last_clean":"2023-11-24T08:11:32.592553+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:11:32.592553+0000","last_undegraded":"2023-11-24T08:11:32.592553+0000","last_fullsized":"2023-11-24T08:11:32.592553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542183,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32492,"kb_used_data":1312,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339348,"statfs":{"total":96636764160,"available":96603492352,"internally_reserved":0,"allocated":1343488,"data_stored":900032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.018,"15min":1.0129999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":2.5329999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.95699999999999996},{"interface":"front","average":{"1min":1.1399999999999999,"5min":1.014,"15min":0.99299999999999999},"min":{"1min":0.65300000000000002,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.7989999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.012}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.141,"15min":1.129},"min":{"1min":0.76500000000000001,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.179,"15min":1.167},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":2.2250000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.3480000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.093,"15min":1.0589999999999999},"min":{"1min":0.72199999999999998,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":3.1829999999999998,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.385},{"interface":"front","average":{"1min":1.27,"5min":1.2150000000000001,"15min":1.206},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.0699999999999998,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.298}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.26,"15min":1.2410000000000001},"min":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"max":{"1min":3.387,"5min":3.387,"15min":3.387},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.25,"15min":1.222},"min":{"1min":0.97999999999999998,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.4020000000000001,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.583}]},{"osd":4,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.2330000000000001,"15min":1.2110000000000001},"min":{"1min":0.95599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.6699999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.5069999999999999,"5min":1.3959999999999999,"15min":1.3779999999999999},"min":{"1min":1.0920000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":3.2869999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.5289999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.4399999999999999,"15min":1.4390000000000001},"min":{"1min":1.0449999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.931,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.371,"15min":1.355},"min":{"1min":1.0149999999999999,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.9750000000000001,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.222}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.494,"15min":1.4790000000000001},"min":{"1min":1.1890000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.4630000000000001,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.611,"5min":1.5,"15min":1.482},"min":{"1min":1.016,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":3.8450000000000002,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.728}]}]},{"osd":6,"up_from":14,"seq":60129542182,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31904,"kb_used_data":724,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339936,"statfs":{"total":96636764160,"available":96604094464,"internally_reserved":0,"allocated":741376,"data_stored":309504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.091,"15min":1.099},"min":{"1min":0.67800000000000005,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6220000000000001,"5min":2.085,"15min":2.085},"last":0.58699999999999997},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.2390000000000001,"15min":1.2450000000000001},"min":{"1min":0.627,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.766,"5min":1.794,"15min":1.794},"last":0.66700000000000004}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.121,"15min":1.127},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.837,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1659999999999999,"15min":1.1699999999999999},"min":{"1min":0.77400000000000002,"5min":0.749,"15min":0.749},"max":{"1min":1.615,"5min":1.98,"15min":1.98},"last":1.0449999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.2190000000000001,"15min":1.2190000000000001},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.7729999999999999,"5min":1.778,"15min":1.778},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.105,"5min":1.1859999999999999,"15min":1.2},"min":{"1min":0.70299999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7649999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.095}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.212,"5min":1.3340000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7010000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.1799999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4399999999999999,"15min":1.448},"min":{"1min":0.83299999999999996,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.948,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.381}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.359,"15min":1.3740000000000001},"min":{"1min":0.92200000000000004,"5min":0.622,"15min":0.622},"max":{"1min":1.9750000000000001,"5min":2.343,"15min":2.343},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.444,"5min":1.571,"15min":1.5920000000000001},"min":{"1min":0.79800000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.9099999999999999,"5min":2.661,"15min":2.661},"last":1.5329999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.431,"5min":1.6859999999999999,"15min":1.728},"min":{"1min":0.88400000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.012,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.635,"15min":1.6659999999999999},"min":{"1min":0.70699999999999996,"5min":0.505,"15min":0.505},"max":{"1min":2.2610000000000001,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.494}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.6259999999999999,"5min":1.7709999999999999,"15min":1.7949999999999999},"min":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.3140000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.573,"5min":1.7669999999999999,"15min":1.7989999999999999},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.109,"5min":2.657,"15min":2.657},"last":1.5720000000000001}]}]},{"osd":5,"up_from":14,"seq":60129542181,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31912,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339928,"statfs":{"total":96636764160,"available":96604086272,"internally_reserved":0,"allocated":749568,"data_stored":310156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.042,"15min":1.022},"min":{"1min":0.63600000000000001,"5min":0.627,"15min":0.627},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1639999999999999,"15min":1.1679999999999999},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.071}]},{"osd":1,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.109,"15min":1.1120000000000001},"min":{"1min":0.65800000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6140000000000001,"5min":1.6140000000000001,"15min":1.6140000000000001},"last":1.274},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1850000000000001,"15min":1.1970000000000001},"min":{"1min":0.70199999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.5469999999999999,"5min":1.722,"15min":1.722},"last":0.999}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.3140000000000001,"15min":1.3300000000000001},"min":{"1min":0.76500000000000001,"5min":0.754,"15min":0.754},"max":{"1min":1.9179999999999999,"5min":2.085,"15min":2.085},"last":1.25},{"interface":"front","average":{"1min":1.304,"5min":1.3620000000000001,"15min":1.3720000000000001},"min":{"1min":0.748,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.75,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.3600000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.4330000000000001,"15min":1.4450000000000001},"min":{"1min":0.85599999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.7130000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.512,"15min":1.532},"min":{"1min":0.874,"5min":0.874,"15min":0.874},"max":{"1min":1.9510000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.1619999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":0.91800000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7849999999999999,"5min":3.577,"15min":3.577},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.694,"15min":1.7190000000000001},"min":{"1min":1.133,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.9750000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.383}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.5680000000000001,"5min":1.6100000000000001,"15min":1.617},"min":{"1min":0.98299999999999998,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":2.121,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.48},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.546,"15min":1.54},"min":{"1min":1.1279999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.996,"5min":2.512,"15min":2.512},"last":1.204}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.6359999999999999,"5min":1.7130000000000001,"15min":1.726},"min":{"1min":1.2070000000000001,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.897,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.4570000000000001},{"interface":"front","average":{"1min":1.6950000000000001,"5min":1.74,"15min":1.748},"min":{"1min":1.3220000000000001,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.1989999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.4239999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542182,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31916,"kb_used_data":736,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339924,"statfs":{"total":96636764160,"available":96604082176,"internally_reserved":0,"allocated":753664,"data_stored":311116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":0.99099999999999999,"15min":0.98099999999999998},"min":{"1min":0.747,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.524,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.077,"5min":1.0880000000000001,"15min":1.0900000000000001},"min":{"1min":0.69599999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":1.202}]},{"osd":1,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":1.135,"15min":1.1479999999999999},"min":{"1min":0.60099999999999998,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.4790000000000001,"5min":1.859,"15min":1.859},"last":1.246},{"interface":"front","average":{"1min":1.254,"5min":1.171,"15min":1.157},"min":{"1min":0.83399999999999996,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.2749999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.764}]},{"osd":2,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.262,"15min":1.2729999999999999},"min":{"1min":0.871,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8759999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.677},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0980000000000001,"15min":1.105},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.0600000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.4299999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.2889999999999999,"15min":1.2929999999999999},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.004,"5min":2.25,"15min":2.25},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.4139999999999999,"15min":1.4199999999999999},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.0899999999999999,"5min":2.746,"15min":2.746},"last":1.8360000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.319,"15min":1.325},"min":{"1min":0.82399999999999995,"5min":0.627,"15min":0.627},"max":{"1min":1.7350000000000001,"5min":1.923,"15min":1.923},"last":0.82499999999999996},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4530000000000001,"15min":1.458},"min":{"1min":0.93100000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.145,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.718}]},{"osd":6,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.444,"15min":1.452},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.946,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.296},{"interface":"front","average":{"1min":1.5369999999999999,"5min":1.5609999999999999,"15min":1.5649999999999999},"min":{"1min":0.84799999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.9239999999999999,"5min":2.641,"15min":2.641},"last":1.4770000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:10:42 2023","interfaces":[{"interface":"back","average":{"1min":1.54,"5min":1.5169999999999999,"15min":1.5129999999999999},"min":{"1min":0.872,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.3769999999999998,"5min":2.548,"15min":2.548},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.5189999999999999,"5min":1.5489999999999999,"15min":1.554},"min":{"1min":0.77100000000000002,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":2.1989999999999998,"5min":2.6869999999999998,"15min":2.6869999999999998},"last":1.538}]}]},{"osd":3,"up_from":14,"seq":60129542182,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":749568,"data_stored":309560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.1439999999999999,"15min":1.177},"min":{"1min":0.51800000000000002,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5389999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.89800000000000002,"5min":1.127,"15min":1.165},"min":{"1min":0.61299999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.333,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.98999999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.98699999999999999,"15min":1.0049999999999999},"min":{"1min":0.63100000000000001,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.2529999999999999,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.131,"15min":1.1359999999999999},"min":{"1min":0.61399999999999999,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.466,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.78100000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.222,"15min":1.242},"min":{"1min":0.67200000000000004,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4930000000000001,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.224},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.159,"15min":1.1599999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.766,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.075}]},{"osd":4,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.339,"5min":1.4630000000000001,"15min":1.484},"min":{"1min":0.71899999999999997,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.9179999999999999,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.28},{"interface":"front","average":{"1min":1.369,"5min":1.387,"15min":1.3899999999999999},"min":{"1min":1.087,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.9950000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.4570000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.5649999999999999,"15min":1.587},"min":{"1min":0.98499999999999999,"5min":0.84999999999999998,"15min":0.84999999999999998},"max":{"1min":1.853,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.363},{"interface":"front","average":{"1min":1.494,"5min":1.548,"15min":1.5569999999999999},"min":{"1min":1.0129999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.0339999999999998,"5min":3.339,"15min":3.339},"last":1.333}]},{"osd":6,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.657,"15min":1.698},"min":{"1min":1.0580000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.732,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.111},{"interface":"front","average":{"1min":1.504,"5min":1.5609999999999999,"15min":1.571},"min":{"1min":1.081,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.966,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.538}]},{"osd":7,"last update":"Fri Nov 24 08:10:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5940000000000001,"5min":1.6859999999999999,"15min":1.702},"min":{"1min":1.248,"5min":1.048,"15min":1.048},"max":{"1min":2.028,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.484,"5min":1.9830000000000001,"15min":2.0659999999999998},"min":{"1min":0.86099999999999999,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.8420000000000001,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.0449999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542182,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27760,"kb_used_data":732,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344080,"statfs":{"total":96636764160,"available":96608337920,"internally_reserved":0,"allocated":749568,"data_stored":309560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.77400000000000002,"15min":0.76500000000000001},"min":{"1min":0.371,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.91800000000000004,"5min":0.78900000000000003,"15min":0.76700000000000002},"min":{"1min":0.45300000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.92800000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.85999999999999999,"15min":0.874},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.212,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.016},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.93100000000000005,"15min":0.92100000000000004},"min":{"1min":0.499,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.514}]},{"osd":3,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":0.97199999999999998,"15min":0.96999999999999997},"min":{"1min":0.48199999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.466,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.3080000000000001},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.999,"15min":1.0049999999999999},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.7769999999999999,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.153}]},{"osd":4,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.2130000000000001,"15min":1.2150000000000001},"min":{"1min":0.752,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.5960000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.2150000000000001,"15min":1.2190000000000001},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.458}]},{"osd":5,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.4179999999999999,"15min":1.45},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":1.895,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.27},{"interface":"front","average":{"1min":1.28,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.73399999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8340000000000001,"5min":1.8340000000000001,"15min":1.8340000000000001},"last":1.407}]},{"osd":6,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.4039999999999999,"15min":1.423},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.8440000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.333,"15min":1.339},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.958,"5min":1.958,"15min":1.958},"last":1.359}]},{"osd":7,"last update":"Fri Nov 24 08:10:44 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.4650000000000001,"15min":1.4870000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.9219999999999999,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.222},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4350000000000001,"15min":1.446},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.9910000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.603}]}]},{"osd":1,"up_from":14,"seq":60129542182,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27748,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344092,"statfs":{"total":96636764160,"available":96608350208,"internally_reserved":0,"allocated":745472,"data_stored":309594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.175,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":0.88900000000000001},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":1.125,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.145}]},{"osd":2,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.85899999999999999,"15min":0.85499999999999998},"min":{"1min":0.46700000000000003,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.52,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0209999999999999,"15min":1.01},"min":{"1min":0.51000000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.4219999999999999,"5min":1.621,"15min":1.621},"last":1.242}]},{"osd":3,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.125,"5min":1.157,"15min":1.163},"min":{"1min":0.56699999999999995,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.5509999999999999,"5min":1.907,"15min":1.907},"last":1.3879999999999999},{"interface":"front","average":{"1min":0.999,"5min":1.048,"15min":1.056},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":2.069,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.64200000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.1659999999999999},"min":{"1min":0.91700000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.7270000000000001,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.294},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3380000000000001,"15min":1.335},"min":{"1min":0.86299999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.8280000000000001,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.552}]},{"osd":5,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.2929999999999999,"15min":1.3109999999999999},"min":{"1min":0.83199999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.625,"5min":2.661,"15min":2.661},"last":1.335},{"interface":"front","average":{"1min":1.2609999999999999,"5min":1.3300000000000001,"15min":1.3420000000000001},"min":{"1min":0.93500000000000005,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.6279999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.367}]},{"osd":6,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3580000000000001,"15min":1.361},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7310000000000001,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.337,"15min":1.333},"min":{"1min":0.98299999999999998,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.704,"5min":2.105,"15min":2.105},"last":1.419}]},{"osd":7,"last update":"Fri Nov 24 08:10:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4139999999999999,"15min":1.4059999999999999},"min":{"1min":1.0569999999999999,"5min":0.97799999999999998,"15min":0.97799999999999998},"max":{"1min":2.0339999999999998,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3580000000000001,"15min":1.365},"min":{"1min":0.82499999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.7709999999999999,"5min":2.008,"15min":2.008},"last":1.4550000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542182,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32504,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339336,"statfs":{"total":96636764160,"available":96603480064,"internally_reserved":0,"allocated":1351680,"data_stored":902066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80600000000000005,"15min":0.80500000000000005},"min":{"1min":0.374,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.2909999999999999,"5min":1.403,"15min":1.403},"last":1.079},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.95899999999999996,"15min":0.95799999999999996},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.494,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.96599999999999997}]},{"osd":2,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.82399999999999995,"15min":0.82799999999999996},"min":{"1min":0.44,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.872},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.92600000000000005,"15min":0.93400000000000005},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.335,"5min":1.986,"15min":1.986},"last":1.1719999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":1.0649999999999999,"15min":1.077},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.704,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.95499999999999996,"15min":0.97399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.585,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.0109999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.163,"15min":1.1699999999999999},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.744,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.383},{"interface":"front","average":{"1min":1.288,"5min":1.2909999999999999,"15min":1.2909999999999999},"min":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.7849999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.5720000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.224,"15min":1.228},"min":{"1min":0.747,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.7270000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.2549999999999999,"15min":1.2669999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.5860000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.468}]},{"osd":6,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.288,"15min":1.2789999999999999},"min":{"1min":0.755,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9370000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2869999999999999,"15min":1.2869999999999999},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.9139999999999999,"5min":2.149,"15min":2.149},"last":1.5349999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:10:41 2023","interfaces":[{"interface":"back","average":{"1min":1.2949999999999999,"5min":1.419,"15min":1.4390000000000001},"min":{"1min":0.85399999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7969999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.423,"15min":1.425},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":2.0600000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:11:37.524 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:11:37.524 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:11:37.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.785+0000 7f7e371ce640 1 Processor -- start 2023-11-24T08:11:37.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.786+0000 7f7e371ce640 1 -- start start 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.787+0000 7f7e371ce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7e3015b440 0x7f7e300cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.787+0000 7f7e371ce640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e300cbf20 0x7f7e300c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.787+0000 7f7e371ce640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3015eea0 con 0x7f7e300c57e0 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.787+0000 7f7e371ce640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e300cc3d0 con 0x7f7e3015b440 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.787+0000 7f7e371ce640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015b820 con 0x7f7e300cbf20 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7e3015b440 0x7f7e300cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7e3015b440 0x7f7e300cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43918/0 (socket says 172.21.15.106:43918) 2023-11-24T08:11:37.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 -- 172.21.15.106:0/250947195 learned_addr learned my addr 172.21.15.106:0/250947195 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e2ffff640 1 --2- 172.21.15.106:0/250947195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e300cbf20 0x7f7e300c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 -- 172.21.15.106:0/250947195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e300cbf20 msgr2=0x7f7e300c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 --2- 172.21.15.106:0/250947195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e300cbf20 0x7f7e300c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 -- 172.21.15.106:0/250947195 >> v1:172.21.15.106:6789/0 conn(0x7f7e300c57e0 legacy=0x7f7e300c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e34f6a640 1 -- 172.21.15.106:0/250947195 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3006d600 con 0x7f7e3015b440 2023-11-24T08:11:37.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.788+0000 7f7e2f7fe640 1 -- 172.21.15.106:0/250947195 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033666174 0 0) 0x7f7e3015eea0 con 0x7f7e300c57e0 2023-11-24T08:11:37.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.789+0000 7f7e34f6a640 1 --2- 172.21.15.106:0/250947195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e3015b440 0x7f7e300cb9e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e2402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797ad38e77c4d78b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:37.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e2f7fe640 1 -- 172.21.15.106:0/250947195 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24031640 con 0x7f7e3015b440 2023-11-24T08:11:37.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e2f7fe640 1 -- 172.21.15.106:0/250947195 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e240317a0 con 0x7f7e3015b440 2023-11-24T08:11:37.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e2f7fe640 1 -- 172.21.15.106:0/250947195 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24031b30 con 0x7f7e3015b440 2023-11-24T08:11:37.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e371ce640 1 -- 172.21.15.106:0/250947195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e3015b440 msgr2=0x7f7e300cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e371ce640 1 --2- 172.21.15.106:0/250947195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e3015b440 0x7f7e300cb9e0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e2402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e371ce640 1 -- 172.21.15.106:0/250947195 shutdown_connections 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e371ce640 1 --2- 172.21.15.106:0/250947195 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e300cbf20 0x7f7e300c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e371ce640 1 --2- 172.21.15.106:0/250947195 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e3015b440 0x7f7e300cb9e0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.790+0000 7f7e371ce640 1 -- 172.21.15.106:0/250947195 >> 172.21.15.106:0/250947195 conn(0x7f7e300b0c50 msgr2=0x7f7e300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.791+0000 7f7e371ce640 1 -- 172.21.15.106:0/250947195 shutdown_connections 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.791+0000 7f7e371ce640 1 -- 172.21.15.106:0/250947195 wait complete. 2023-11-24T08:11:37.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.792+0000 7f7e371ce640 1 Processor -- start 2023-11-24T08:11:37.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.792+0000 7f7e371ce640 1 -- start start 2023-11-24T08:11:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.793+0000 7f7e371ce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 0x7f7e3015b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.793+0000 7f7e371ce640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e300cbf20 0x7f7e3015d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.793+0000 7f7e371ce640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e3015b440 0x7f7e301f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.793+0000 7f7e371ce640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015eee0 con 0x7f7e300c57e0 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.793+0000 7f7e371ce640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015c2d0 con 0x7f7e300cbf20 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.793+0000 7f7e371ce640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3015c5b0 con 0x7f7e3015b440 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 0x7f7e3015b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 0x7f7e3015b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43922/0 (socket says 172.21.15.106:43922) 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 -- 172.21.15.106:0/3432286403 learned_addr learned my addr 172.21.15.106:0/3432286403 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e2ffff640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e300cbf20 0x7f7e3015d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e3576b640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e3015b440 0x7f7e301f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 -- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e3015b440 msgr2=0x7f7e301f0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e3015b440 0x7f7e301f0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 -- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e300cbf20 msgr2=0x7f7e3015d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:37.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e300cbf20 0x7f7e3015d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e34f6a640 1 -- 172.21.15.106:0/3432286403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e240094d0 con 0x7f7e300c57e0 2023-11-24T08:11:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e2ffff640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e300cbf20 0x7f7e3015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:37.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.794+0000 7f7e3576b640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e3015b440 0x7f7e301f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.795+0000 7f7e34f6a640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 0x7f7e3015b960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.795+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24004230 con 0x7f7e300c57e0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.795+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e240043b0 con 0x7f7e300c57e0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.795+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3015c890 con 0x7f7e300c57e0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.795+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e24004520 con 0x7f7e300c57e0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.795+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3015ce50 con 0x7f7e300c57e0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.796+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3004e8b0 con 0x7f7e300c57e0 2023-11-24T08:11:37.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.797+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f7e24042070 con 0x7f7e300c57e0 2023-11-24T08:11:37.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.798+0000 7f7e2d7fa640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7e1406b080 0x7f7e1406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:37.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.798+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7e240ab7e0 con 0x7f7e300c57e0 2023-11-24T08:11:37.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.798+0000 7f7e2ffff640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7e1406b080 0x7f7e1406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:37.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.799+0000 7f7e2ffff640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7e1406b080 0x7f7e1406d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e300ca850 tx=0x7f7e1c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:37.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:37.803+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7e24076410 con 0x7f7e300c57e0 2023-11-24T08:11:38.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.063+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f7e3006dc80 con 0x7f7e300c57e0 2023-11-24T08:11:38.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.067+0000 7f7e2d7fa640 1 -- 172.21.15.106:0/3432286403 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f7e240350b0 con 0x7f7e300c57e0 2023-11-24T08:11:38.069 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2023-11-24T08:11:38.069 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:11:38.070 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2023-11-24T08:11:38.070 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2023-11-24T08:11:38.070 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:11:38.070 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2023-11-24T08:11:38.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7e1406b080 msgr2=0x7f7e1406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7e1406b080 0x7f7e1406d540 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7e300ca850 tx=0x7f7e1c008040 comp rx=0 tx=0).stop 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 msgr2=0x7f7e3015b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 0x7f7e3015b960 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f7e24009870 tx=0x7f7e240046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 shutdown_connections 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7e1406b080 0x7f7e1406d540 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7e3015b440 0x7f7e301f0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e300cbf20 0x7f7e3015d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 --2- 172.21.15.106:0/3432286403 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7e300c57e0 0x7f7e3015b960 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.070+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 >> 172.21.15.106:0/3432286403 conn(0x7f7e300b0c50 msgr2=0x7f7e300b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.071+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 shutdown_connections 2023-11-24T08:11:38.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.071+0000 7f7e371ce640 1 -- 172.21.15.106:0/3432286403 wait complete. 2023-11-24T08:11:38.091 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:38.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.301+0000 7fb92d06e640 1 Processor -- start 2023-11-24T08:11:38.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.301+0000 7fb92d06e640 1 -- start start 2023-11-24T08:11:38.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb92d06e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb92806beb0 0x7fb92806c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb92d06e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806c7d0 0x7fb9280cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb92d06e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9280ce9f0 con 0x7fb9280b0520 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb92d06e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9280aff70 con 0x7fb92806beb0 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb92d06e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92806b840 con 0x7fb92806c7d0 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb927fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb92806beb0 0x7fb92806c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb927fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb92806beb0 0x7fb92806c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43926/0 (socket says 172.21.15.106:43926) 2023-11-24T08:11:38.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.303+0000 7fb927fff640 1 -- 172.21.15.106:0/931370774 learned_addr learned my addr 172.21.15.106:0/931370774 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.304+0000 7fb92c86d640 1 --2- 172.21.15.106:0/931370774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806c7d0 0x7fb9280cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.304+0000 7fb927fff640 1 -- 172.21.15.106:0/931370774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806c7d0 msgr2=0x7fb9280cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.304+0000 7fb927fff640 1 --2- 172.21.15.106:0/931370774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806c7d0 0x7fb9280cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.304+0000 7fb927fff640 1 -- 172.21.15.106:0/931370774 >> v1:172.21.15.156:6789/0 conn(0x7fb9280b0520 legacy=0x7fb9280cf340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.304+0000 7fb927fff640 1 -- 172.21.15.106:0/931370774 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92806bcd0 con 0x7fb92806beb0 2023-11-24T08:11:38.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.305+0000 7fb927fff640 1 --2- 172.21.15.106:0/931370774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806beb0 0x7fb92806c290 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb918009870 tx=0x7fb91802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd82143257f27dad server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.305+0000 7fb9277fe640 1 -- 172.21.15.106:0/931370774 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9180317c0 con 0x7fb92806beb0 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.305+0000 7fb9277fe640 1 -- 172.21.15.106:0/931370774 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb918031920 con 0x7fb92806beb0 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.305+0000 7fb9277fe640 1 -- 172.21.15.106:0/931370774 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb918031cb0 con 0x7fb92806beb0 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 -- 172.21.15.106:0/931370774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806beb0 msgr2=0x7fb92806c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 --2- 172.21.15.106:0/931370774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806beb0 0x7fb92806c290 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fb918009870 tx=0x7fb91802b860 comp rx=0 tx=0).stop 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 -- 172.21.15.106:0/931370774 shutdown_connections 2023-11-24T08:11:38.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 --2- 172.21.15.106:0/931370774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806c7d0 0x7fb9280cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 --2- 172.21.15.106:0/931370774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806beb0 0x7fb92806c290 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 -- 172.21.15.106:0/931370774 >> 172.21.15.106:0/931370774 conn(0x7fb9280cda50 msgr2=0x7fb9280cde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 -- 172.21.15.106:0/931370774 shutdown_connections 2023-11-24T08:11:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.306+0000 7fb92d06e640 1 -- 172.21.15.106:0/931370774 wait complete. 2023-11-24T08:11:38.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.308+0000 7fb92d06e640 1 Processor -- start 2023-11-24T08:11:38.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.308+0000 7fb92d06e640 1 -- start start 2023-11-24T08:11:38.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92d06e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806beb0 0x7fb9281493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92d06e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 0x7fb9280c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92d06e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9280b0520 0x7fb9280b61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92d06e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280b5d20 con 0x7fb92806c7d0 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92d06e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb928149930 con 0x7fb9280b0520 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92d06e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9280b5110 con 0x7fb92806beb0 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 0x7fb9280c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 0x7fb9280c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43934/0 (socket says 172.21.15.106:43934) 2023-11-24T08:11:38.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 -- 172.21.15.106:0/1035237925 learned_addr learned my addr 172.21.15.106:0/1035237925 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb927fff640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806beb0 0x7fb9281493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb92c86d640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9280b0520 0x7fb9280b61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 -- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806beb0 msgr2=0x7fb9281493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806beb0 0x7fb9281493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 -- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9280b0520 msgr2=0x7fb9280b61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9280b0520 0x7fb9280b61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.309+0000 7fb91ffff640 1 -- 172.21.15.106:0/1035237925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9180094d0 con 0x7fb92806c7d0 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.310+0000 7fb91ffff640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 0x7fb9280c80a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b3f0 tx=0x7fb91000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.310+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910011830 con 0x7fb92806c7d0 2023-11-24T08:11:38.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.310+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb910011990 con 0x7fb92806c7d0 2023-11-24T08:11:38.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.310+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9280b53f0 con 0x7fb92806c7d0 2023-11-24T08:11:38.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.310+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb910011d40 con 0x7fb92806c7d0 2023-11-24T08:11:38.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.310+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9280b59b0 con 0x7fb92806c7d0 2023-11-24T08:11:38.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.312+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9280b5b70 con 0x7fb92806c7d0 2023-11-24T08:11:38.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.314+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fb910004020 con 0x7fb92806c7d0 2023-11-24T08:11:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.314+0000 7fb9257fa640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb90806b080 0x7fb90806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.315+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb910089d20 con 0x7fb92806c7d0 2023-11-24T08:11:38.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.315+0000 7fb927fff640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb90806b080 0x7fb90806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:38.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.315+0000 7fb927fff640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb90806b080 0x7fb90806d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb918036000 tx=0x7fb91803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:38.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.317+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb910054a50 con 0x7fb92806c7d0 2023-11-24T08:11:38.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.543+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb92804e840 con 0x7fb92806c7d0 2023-11-24T08:11:38.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.544+0000 7fb9257fa640 1 -- 172.21.15.106:0/1035237925 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fb910015090 con 0x7fb92806c7d0 2023-11-24T08:11:38.545 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:38.545 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:11:32.053291+0000","modified":"2023-11-24T08:11:35.081758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/180703350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":180703350},{"type":"v1","addr":"172.21.15.106:6837","nonce":180703350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:11:38.546 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 33 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb90806b080 msgr2=0x7fb90806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb90806b080 0x7fb90806d540 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fb918036000 tx=0x7fb91803d040 comp rx=0 tx=0).stop 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 msgr2=0x7fb9280c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 0x7fb9280c80a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fb91000b3f0 tx=0x7fb91000b8b0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 shutdown_connections 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9280b0520 0x7fb9280b61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb92806c7d0 0x7fb9280c80a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb90806b080 0x7fb90806d540 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.546+0000 7fb92d06e640 1 --2- 172.21.15.106:0/1035237925 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb92806beb0 0x7fb9281493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.547+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 >> 172.21.15.106:0/1035237925 conn(0x7fb9280cda50 msgr2=0x7fb9280aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.547+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 shutdown_connections 2023-11-24T08:11:38.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:38.547+0000 7fb92d06e640 1 -- 172.21.15.106:0/1035237925 wait complete. 2023-11-24T08:11:38.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:11:32.053291+0000', 'modified': '2023-11-24T08:11:35.081758+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6075}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6075': {'gid': 6075, 'name': 'c', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/180703350', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6836', 'nonce': 180703350}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 180703350}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:11:38.969 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:11:38.969 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T08:11:39.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.195+0000 7fac67fff640 1 Processor -- start 2023-11-24T08:11:39.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.195+0000 7fac67fff640 1 -- start start 2023-11-24T08:11:39.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac67fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac68160890 0x7fac680c9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac67fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680cb690 0x7fac680b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac67fff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6806c530 con 0x7fac680b1080 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac67fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6806d170 con 0x7fac68160890 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac67fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac680b1510 con 0x7fac680cb690 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac66ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac68160890 0x7fac680c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac66ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fac68160890 0x7fac680c9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43950/0 (socket says 172.21.15.106:43950) 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.197+0000 7fac66ffd640 1 -- 172.21.15.106:0/2849447713 learned_addr learned my addr 172.21.15.106:0/2849447713 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.198+0000 7fac66ffd640 1 -- 172.21.15.106:0/2849447713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680cb690 msgr2=0x7fac680b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.198+0000 7fac66ffd640 1 --2- 172.21.15.106:0/2849447713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680cb690 0x7fac680b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.198+0000 7fac66ffd640 1 -- 172.21.15.106:0/2849447713 >> v1:172.21.15.156:6789/0 conn(0x7fac680b1080 legacy=0x7fac68160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.198+0000 7fac66ffd640 1 -- 172.21.15.106:0/2849447713 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6806c7d0 con 0x7fac68160890 2023-11-24T08:11:39.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.199+0000 7fac66ffd640 1 --2- 172.21.15.106:0/2849447713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac68160890 0x7fac680c9710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fac50009870 tx=0x7fac5002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dc3b4cea01a4c26 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.199+0000 7fac65ffb640 1 -- 172.21.15.106:0/2849447713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac500317c0 con 0x7fac68160890 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.199+0000 7fac65ffb640 1 -- 172.21.15.106:0/2849447713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac50031920 con 0x7fac68160890 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.199+0000 7fac65ffb640 1 -- 172.21.15.106:0/2849447713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac50031cb0 con 0x7fac68160890 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 -- 172.21.15.106:0/2849447713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac68160890 msgr2=0x7fac680c9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 --2- 172.21.15.106:0/2849447713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac68160890 0x7fac680c9710 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fac50009870 tx=0x7fac5002b860 comp rx=0 tx=0).stop 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 -- 172.21.15.106:0/2849447713 shutdown_connections 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 --2- 172.21.15.106:0/2849447713 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680cb690 0x7fac680b0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 --2- 172.21.15.106:0/2849447713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac68160890 0x7fac680c9710 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 -- 172.21.15.106:0/2849447713 >> 172.21.15.106:0/2849447713 conn(0x7fac680cbb40 msgr2=0x7fac680cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 -- 172.21.15.106:0/2849447713 shutdown_connections 2023-11-24T08:11:39.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.200+0000 7fac67fff640 1 -- 172.21.15.106:0/2849447713 wait complete. 2023-11-24T08:11:39.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.202+0000 7fac67fff640 1 Processor -- start 2023-11-24T08:11:39.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.202+0000 7fac67fff640 1 -- start start 2023-11-24T08:11:39.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac67fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680b1080 0x7fac68158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac67fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 0x7fac6809ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac67fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac68160890 0x7fac6809bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac67fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac681e8700 con 0x7fac680cb690 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac67fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac681e7850 con 0x7fac68160890 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac67fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac681e7af0 con 0x7fac680b1080 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 0x7fac6809ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 0x7fac6809ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43960/0 (socket says 172.21.15.106:43960) 2023-11-24T08:11:39.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.203+0000 7fac667fc640 1 -- 172.21.15.106:0/2083119955 learned_addr learned my addr 172.21.15.106:0/2083119955 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac66ffd640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680b1080 0x7fac68158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac677fe640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac68160890 0x7fac6809bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac667fc640 1 -- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680b1080 msgr2=0x7fac68158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac667fc640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680b1080 0x7fac68158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac667fc640 1 -- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac68160890 msgr2=0x7fac6809bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac667fc640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac68160890 0x7fac6809bf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac667fc640 1 -- 172.21.15.106:0/2083119955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac500094d0 con 0x7fac680cb690 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac667fc640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 0x7fac6809ba10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac5800ba50 tx=0x7fac5800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac5ffff640 1 -- 172.21.15.106:0/2083119955 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5800cbe0 con 0x7fac680cb690 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac5ffff640 1 -- 172.21.15.106:0/2083119955 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac5800cd60 con 0x7fac680cb690 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac5ffff640 1 -- 172.21.15.106:0/2083119955 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58014680 con 0x7fac680cb690 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.204+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac681e7dd0 con 0x7fac680cb690 2023-11-24T08:11:39.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.205+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac681e8390 con 0x7fac680cb690 2023-11-24T08:11:39.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.207+0000 7fac5ffff640 1 -- 172.21.15.106:0/2083119955 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fac5801d3b0 con 0x7fac680cb690 2023-11-24T08:11:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.207+0000 7fac5ffff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fac4806b080 0x7fac4806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.208+0000 7fac5ffff640 1 -- 172.21.15.106:0/2083119955 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac58089b20 con 0x7fac680cb690 2023-11-24T08:11:39.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.208+0000 7fac66ffd640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fac4806b080 0x7fac4806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.208+0000 7fac705cd740 1 Processor -- start 2023-11-24T08:11:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.208+0000 7fac705cd740 1 -- start start 2023-11-24T08:11:39.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac66ffd640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fac4806b080 0x7fac4806d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac50036000 tx=0x7fac5003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e69c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e69b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac705cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6df730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac705cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e5fa880 con 0x559c8e69b7b0 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e609ae0 con 0x559c8e69c070 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.209+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c8e6071d0 con 0x559c8e696d70 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6df730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6df730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43970/0 (socket says 172.21.15.106:43970) 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 -- 172.21.15.106:0/120609284 learned_addr learned my addr 172.21.15.106:0/120609284 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac66ffd640 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e69c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 -- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 msgr2=0x559c8e69b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e69b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 -- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 msgr2=0x559c8e69c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e69c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 -- 172.21.15.106:0/120609284 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c8e5ca860 con 0x559c8e69b7b0 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.210+0000 7fac677fe640 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6df730 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac6000b750 tx=0x7fac6000bc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2e1ee48b2634d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac5d7fa640 1 -- 172.21.15.106:0/120609284 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000be60 con 0x559c8e69b7b0 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac5d7fa640 1 -- 172.21.15.106:0/120609284 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac6000d950 con 0x559c8e69b7b0 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac5d7fa640 1 -- 172.21.15.106:0/120609284 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac6000dd00 con 0x559c8e69b7b0 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 -- 172.21.15.106:0/120609284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 msgr2=0x559c8e6df730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6df730 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fac6000b750 tx=0x7fac6000bc10 comp rx=0 tx=0).stop 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 -- 172.21.15.106:0/120609284 shutdown_connections 2023-11-24T08:11:39.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6df730 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e69b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 --2- 172.21.15.106:0/120609284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e69c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 -- 172.21.15.106:0/120609284 >> 172.21.15.106:0/120609284 conn(0x559c8e65dc70 msgr2=0x559c8e6e1b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 -- 172.21.15.106:0/120609284 shutdown_connections 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.211+0000 7fac705cd740 1 -- 172.21.15.106:0/120609284 wait complete. 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.212+0000 7fac705cd740 1 -- start start 2023-11-24T08:11:39.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.212+0000 7fac705cd740 1 Processor -- start 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e696d70 0x559c8e6c2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e69b7b0 0x559c8e6e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 0x559c8e6c4470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e602ac0 con 0x559c8e69c070 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e59f2f0 con 0x559c8e696d70 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c8e5bcc70 con 0x559c8e69b7b0 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.213+0000 7fac705cd740 10 client.0 _ll_register_callbacks cb 0x559c8e5c13b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 0x559c8e6c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 0x559c8e6c4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43978/0 (socket says 172.21.15.106:43978) 2023-11-24T08:11:39.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 -- 172.21.15.106:0/38782617 learned_addr learned my addr 172.21.15.106:0/38782617 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac667fc640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e69b7b0 0x559c8e6e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac66ffd640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e696d70 0x559c8e6c2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 -- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e69b7b0 msgr2=0x559c8e6e7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e69b7b0 0x559c8e6e7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 -- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e696d70 msgr2=0x559c8e6c2280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e696d70 0x559c8e6c2280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 -- 172.21.15.106:0/38782617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6000e040 con 0x559c8e69c070 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac677fe640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 0x559c8e6c4470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fac60016040 tx=0x7fac60004640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e22dc3facc8db7a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.214+0000 7fac467fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac467fc640 1 -- 172.21.15.106:0/38782617 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60014630 con 0x559c8e69c070 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac467fc640 1 -- 172.21.15.106:0/38782617 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac6000de50 con 0x559c8e69c070 2023-11-24T08:11:39.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c8e5fbaa0 con 0x559c8e69c070 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac467fc640 1 -- 172.21.15.106:0/38782617 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac60014790 con 0x559c8e69c070 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac705cd740 10 client.6207 fetch_fsmap learned FSMap version 33 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x559c8e601140 con 0x559c8e69c070 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac467fc640 1 -- 172.21.15.106:0/38782617 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c8e5fbaa0 con 0x559c8e69c070 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac467fc640 1 -- 172.21.15.106:0/38782617 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac60025030 con 0x559c8e69c070 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac705cd740 10 client.6207 fetch_fsmap finished waiting for FSMap version 33 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac705cd740 10 client.6207 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.215+0000 7fac705cd740 20 client.6207 populate_metadata read hostname 'smithi106' 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.216+0000 7fac705cd740 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e6e97a0 0x559c8e6c0790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.216+0000 7fac705cd740 4 client.6207 mds_command: new command op to 6075 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.216+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 --> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c8e5fb7c0 con 0x559c8e6e97a0 2023-11-24T08:11:39.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.216+0000 7fac66ffd640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e6e97a0 0x559c8e6c0790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.216+0000 7fac467fc640 10 client.6207 ms_handle_connect on v2:172.21.15.106:6836/180703350 2023-11-24T08:11:39.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.216+0000 7fac66ffd640 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e6e97a0 0x559c8e6c0790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d59d92d561b31dd2 server_cookie=282d6ef3dce95965 in_seq=0 out_seq=0 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac467fc640 1 -- 172.21.15.106:0/38782617 <== mds.0 v2:172.21.15.106:6836/180703350 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x559c8e5fb7c0 con 0x559c8e6e97a0 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac467fc640 10 client.6207 handle_command_reply: tid=0 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 client.6207 shutdown 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 msgr2=0x559c8e6c4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 0x559c8e6c4470 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fac60016040 tx=0x7fac60004640 comp rx=0 tx=0).stop 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 shutdown_connections 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e6e97a0 0x559c8e6c0790 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69c070 0x559c8e6c4470 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e69b7b0 0x559c8e6e7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 --2- 172.21.15.106:0/38782617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e696d70 0x559c8e6c2280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac467fc640 0 client.6207 ms_handle_reset on v2:172.21.15.106:6836/180703350 2023-11-24T08:11:39.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.217+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 >> 172.21.15.106:0/38782617 conn(0x559c8e65dc70 msgr2=0x559c8e6df4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.218+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 shutdown_connections 2023-11-24T08:11:39.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.218+0000 7fac705cd740 1 -- 172.21.15.106:0/38782617 wait complete. 2023-11-24T08:11:39.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.218+0000 7fac705cd740 20 client.6207 trim_cache size 0 max 16384 2023-11-24T08:11:39.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.234+0000 7fac705cd740 1 Processor -- start 2023-11-24T08:11:39.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.234+0000 7fac705cd740 1 -- start start 2023-11-24T08:11:39.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6e7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac705cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6e1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e695dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac705cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e365f20 con 0x559c8e69b7b0 2023-11-24T08:11:39.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e6e8ad0 con 0x559c8e69c070 2023-11-24T08:11:39.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c8e6ca900 con 0x559c8e696d70 2023-11-24T08:11:39.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6e1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6e1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43982/0 (socket says 172.21.15.106:43982) 2023-11-24T08:11:39.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.235+0000 7fac667fc640 1 -- 172.21.15.106:0/2546780300 learned_addr learned my addr 172.21.15.106:0/2546780300 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac667fc640 1 -- 172.21.15.106:0/2546780300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 msgr2=0x559c8e6e7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac667fc640 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6e7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac667fc640 1 -- 172.21.15.106:0/2546780300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 msgr2=0x559c8e695dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac667fc640 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e695dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac667fc640 1 -- 172.21.15.106:0/2546780300 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c8e59eb80 con 0x559c8e69b7b0 2023-11-24T08:11:39.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac667fc640 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6e1d50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fac58004af0 tx=0x7fac58058b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63b0ec505a12eda0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac5dffb640 1 -- 172.21.15.106:0/2546780300 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac58060790 con 0x559c8e69b7b0 2023-11-24T08:11:39.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac5dffb640 1 -- 172.21.15.106:0/2546780300 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac580567a0 con 0x559c8e69b7b0 2023-11-24T08:11:39.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.236+0000 7fac5dffb640 1 -- 172.21.15.106:0/2546780300 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5805c170 con 0x559c8e69b7b0 2023-11-24T08:11:39.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 -- 172.21.15.106:0/2546780300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 msgr2=0x559c8e6e1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6e1d50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fac58004af0 tx=0x7fac58058b00 comp rx=0 tx=0).stop 2023-11-24T08:11:39.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 -- 172.21.15.106:0/2546780300 shutdown_connections 2023-11-24T08:11:39.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e695dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6e1d50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 --2- 172.21.15.106:0/2546780300 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6e7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 -- 172.21.15.106:0/2546780300 >> 172.21.15.106:0/2546780300 conn(0x559c8e65dc70 msgr2=0x559c8e69a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 -- 172.21.15.106:0/2546780300 shutdown_connections 2023-11-24T08:11:39.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 -- 172.21.15.106:0/2546780300 wait complete. 2023-11-24T08:11:39.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.237+0000 7fac705cd740 1 -- start start 2023-11-24T08:11:39.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.238+0000 7fac705cd740 1 Processor -- start 2023-11-24T08:11:39.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6befc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e6e55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e74fea0 con 0x559c8e69b7b0 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c8e6e4ff0 con 0x559c8e69c070 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c8e6c5ed0 con 0x559c8e696d70 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac705cd740 10 client.0 _ll_register_callbacks cb 0x7fac6000b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:11:39.250 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6befc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6befc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43996/0 (socket says 172.21.15.106:43996) 2023-11-24T08:11:39.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 -- 172.21.15.106:0/1353457459 learned_addr learned my addr 172.21.15.106:0/1353457459 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac66ffd640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 -- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 msgr2=0x559c8e6c5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6c5990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac677fe640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e6e55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 -- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 msgr2=0x559c8e6e55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e6e55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.239+0000 7fac667fc640 1 -- 172.21.15.106:0/1353457459 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac681e8390 con 0x559c8e69b7b0 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac667fc640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6befc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fac580569f0 tx=0x7fac5800c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4253216fb625035c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac467fc640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac467fc640 1 -- 172.21.15.106:0/1353457459 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac5805c370 con 0x559c8e69b7b0 2023-11-24T08:11:39.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac467fc640 1 -- 172.21.15.106:0/1353457459 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac580578f0 con 0x559c8e69b7b0 2023-11-24T08:11:39.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c8e6e6fe0 con 0x559c8e69b7b0 2023-11-24T08:11:39.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac467fc640 1 -- 172.21.15.106:0/1353457459 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7fac58057ab0 con 0x559c8e69b7b0 2023-11-24T08:11:39.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac705cd740 10 client.6213 fetch_fsmap learned FSMap version 33 2023-11-24T08:11:39.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x559c8e5c4170 con 0x559c8e69b7b0 2023-11-24T08:11:39.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac467fc640 1 -- 172.21.15.106:0/1353457459 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c8e6e6fe0 con 0x559c8e69b7b0 2023-11-24T08:11:39.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac467fc640 1 -- 172.21.15.106:0/1353457459 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 33) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fac58060300 con 0x559c8e69b7b0 2023-11-24T08:11:39.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.240+0000 7fac705cd740 10 client.6213 fetch_fsmap finished waiting for FSMap version 33 2023-11-24T08:11:39.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac705cd740 10 client.6213 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:11:39.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac705cd740 20 client.6213 populate_metadata read hostname 'smithi106' 2023-11-24T08:11:39.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac705cd740 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e74bd40 0x559c8e74c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac705cd740 4 client.6213 mds_command: new command op to 6075 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:11:39.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 --> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7fac6000d950 con 0x559c8e74bd40 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac66ffd640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e74bd40 0x559c8e74c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac66ffd640 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e74bd40 0x559c8e74c120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cd67f15d90ac6a8 server_cookie=f4328da6751cf240 in_seq=0 out_seq=0 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac467fc640 10 client.6213 ms_handle_connect on v2:172.21.15.106:6836/180703350 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac467fc640 1 -- 172.21.15.106:0/1353457459 <== mds.0 v2:172.21.15.106:6836/180703350 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x7fac6000d950 con 0x559c8e74bd40 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.241+0000 7fac467fc640 10 client.6213 handle_command_reply: tid=0 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 client.6213 shutdown 2023-11-24T08:11:39.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 msgr2=0x559c8e6befc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6befc0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fac580569f0 tx=0x7fac5800c650 comp rx=0 tx=0).stop 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 shutdown_connections 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:6836/180703350,v1:172.21.15.106:6837/180703350] conn(0x559c8e74bd40 0x559c8e74c120 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c8e69c070 0x559c8e6e55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x559c8e69b7b0 0x559c8e6befc0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 --2- 172.21.15.106:0/1353457459 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x559c8e696d70 0x559c8e6c5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 >> 172.21.15.106:0/1353457459 conn(0x559c8e65dc70 msgr2=0x559c8e69a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac467fc640 0 client.6213 ms_handle_reset on v2:172.21.15.106:6836/180703350 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 shutdown_connections 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 1 -- 172.21.15.106:0/1353457459 wait complete. 2023-11-24T08:11:39.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.242+0000 7fac705cd740 20 client.6213 trim_cache size 0 max 16384 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d1a897c2-15ce-4272-ad66-4b3723970cb1", 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 6075, 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2023-11-24T08:11:39.257 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 6.6597736440000004, 2023-11-24T08:11:39.258 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 33, 2023-11-24T08:11:39.258 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 62, 2023-11-24T08:11:39.258 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 62, 2023-11-24T08:11:39.258 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 13.072195653 2023-11-24T08:11:39.258 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fac4806b080 msgr2=0x7fac4806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fac4806b080 0x7fac4806d540 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fac50036000 tx=0x7fac5003d040 comp rx=0 tx=0).stop 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 msgr2=0x7fac6809ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 0x7fac6809ba10 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fac5800ba50 tx=0x7fac5800bf10 comp rx=0 tx=0).stop 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 shutdown_connections 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fac68160890 0x7fac6809bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fac680cb690 0x7fac6809ba10 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fac4806b080 0x7fac4806d540 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 --2- 172.21.15.106:0/2083119955 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fac680b1080 0x7fac68158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 >> 172.21.15.106:0/2083119955 conn(0x7fac680cbb40 msgr2=0x7fac68070cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 shutdown_connections 2023-11-24T08:11:39.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.243+0000 7fac67fff640 1 -- 172.21.15.106:0/2083119955 wait complete. 2023-11-24T08:11:39.261 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:11:39.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.464+0000 7f2ce2a31640 1 Processor -- start 2023-11-24T08:11:39.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.465+0000 7f2ce2a31640 1 -- start start 2023-11-24T08:11:39.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.466+0000 7f2ce2a31640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2cdc15c7f0 0x7f2cdc06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.466+0000 7f2ce2a31640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc0cece0 con 0x7f2cdc0ce6c0 2023-11-24T08:11:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.466+0000 7f2ce2a31640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cdc06d040 con 0x7f2cdc0cf450 2023-11-24T08:11:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.466+0000 7f2ce2a31640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cdc0ceaf0 con 0x7f2cdc15c7f0 2023-11-24T08:11:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.466+0000 7f2ce0fce640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2cdc0ce6c0 0x7f2cdc0714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37610/0 (socket says 172.21.15.106:37610) 2023-11-24T08:11:39.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.466+0000 7f2ce0fce640 1 -- 172.21.15.106:0/1385454477 learned_addr learned my addr 172.21.15.106:0/1385454477 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdbfff640 1 --2- 172.21.15.106:0/1385454477 >> v2:172.21.15.156:3300/0 conn(0x7f2cdc15c7f0 0x7f2cdc06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228239638 0 0) 0x7f2cdc0cece0 con 0x7f2cdc0ce6c0 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cc8003520 con 0x7f2cdc0ce6c0 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 341735467 0 0) 0x7f2cc8003520 con 0x7f2cdc0ce6c0 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 >> v2:172.21.15.156:3300/0 conn(0x7f2cdc15c7f0 msgr2=0x7f2cdc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 --2- 172.21.15.106:0/1385454477 >> v2:172.21.15.156:3300/0 conn(0x7f2cdc15c7f0 0x7f2cdc06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 >> v1:172.21.15.156:6789/0 conn(0x7f2cdc0cf450 legacy=0x7f2cdc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc06d320 con 0x7f2cdc0ce6c0 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2cd4001670 con 0x7f2cdc0ce6c0 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 >> v1:172.21.15.106:6789/0 conn(0x7f2cdc0ce6c0 legacy=0x7f2cdc0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.467+0000 7f2cdaffd640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc80039c0 0x7f2cc8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdaffd640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc80047c0 0x7f2cc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdaffd640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cc8005180 0x7f2cc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc800c050 con 0x7f2cc8005180 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc800c1c0 con 0x7f2cc80039c0 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc800c380 con 0x7f2cc80047c0 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2ce0fce640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cc8005180 0x7f2cc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdbfff640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc80039c0 0x7f2cc8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2cdb7fe640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc80047c0 0x7f2cc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2ce0fce640 1 -- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc80047c0 msgr2=0x7f2cc8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2ce0fce640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc80047c0 0x7f2cc8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2ce0fce640 1 -- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc80039c0 msgr2=0x7f2cc8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2ce0fce640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cc80039c0 0x7f2cc8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.468+0000 7f2ce0fce640 1 -- 172.21.15.106:0/1385454477 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc8003520 con 0x7f2cc8005180 2023-11-24T08:11:39.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.470+0000 7f2ce0fce640 1 -- 172.21.15.106:0/1385454477 reap_dead start 2023-11-24T08:11:39.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.470+0000 7f2ce0fce640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cc8005180 0x7f2cc800b980 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c0e0 tx=0x7f2cc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=732b89fd92345a28 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.470+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd4005ad0 con 0x7f2cc8005180 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.470+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cd4005c70 con 0x7f2cc8005180 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.470+0000 7f2cdaffd640 1 -- 172.21.15.106:0/1385454477 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cd402c810 con 0x7f2cc8005180 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1385454477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cc8005180 msgr2=0x7f2cc800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cc8005180 0x7f2cc800b980 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f2cd400c0e0 tx=0x7f2cc80057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1385454477 shutdown_connections 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1385454477 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cc8005180 0x7f2cc800b980 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1385454477 >> 172.21.15.106:0/1385454477 conn(0x7f2cdc06a920 msgr2=0x7f2cdc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1385454477 shutdown_connections 2023-11-24T08:11:39.474 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.471+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1385454477 wait complete. 2023-11-24T08:11:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.473+0000 7f2ce2a31640 1 Processor -- start 2023-11-24T08:11:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.473+0000 7f2ce2a31640 1 -- start start 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce2a31640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc8005180 0x7f2cdc099ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce2a31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 0x7f2cdc0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce2a31640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cdc153ba0 0x7f2cdc154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce2a31640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc09a0e0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce2a31640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc154560 con 0x7f2cdc153ba0 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce2a31640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc1f1b10 con 0x7f2cc8005180 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 0x7f2cdc0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 0x7f2cdc0ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44014/0 (socket says 172.21.15.106:44014) 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 -- 172.21.15.106:0/1445922283 learned_addr learned my addr 172.21.15.106:0/1445922283 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2ce0fce640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cdc153ba0 0x7f2cdc154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdbfff640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc8005180 0x7f2cdc099ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 -- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc8005180 msgr2=0x7f2cdc099ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc8005180 0x7f2cdc099ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 -- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cdc153ba0 msgr2=0x7f2cdc154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cdc153ba0 0x7f2cdc154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdb7fe640 1 -- 172.21.15.106:0/1445922283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cdc1f1df0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.475+0000 7f2cdbfff640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc8005180 0x7f2cdc099ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.476+0000 7f2cdb7fe640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 0x7f2cdc0ad280 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e480 tx=0x7f2cc400e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.476+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40052e0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.476+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cc4005440 con 0x7f2cdc1f1360 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.476+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4003830 con 0x7f2cdc1f1360 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.476+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1f20d0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.476+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc1f2690 con 0x7f2cdc1f1360 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.477+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc1f2820 con 0x7f2cdc1f1360 2023-11-24T08:11:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.478+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2cc4003990 con 0x7f2cdc1f1360 2023-11-24T08:11:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.479+0000 7f2cd8ff9640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cbc06b080 0x7f2cbc06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.479+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cc40896f0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.479+0000 7f2cdbfff640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cbc06b080 0x7f2cbc06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.480+0000 7f2cdbfff640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cbc06b080 0x7f2cbc06d540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc007330 tx=0x7f2ccc00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:39.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.483+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cc40544d0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.719+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2cdc067470 con 0x7f2cdc1f1360 2023-11-24T08:11:39.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.720+0000 7f2cd8ff9640 1 -- 172.21.15.106:0/1445922283 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2cc4057af0 con 0x7f2cdc1f1360 2023-11-24T08:11:39.721 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:11:39.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.722+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cbc06b080 msgr2=0x7f2cbc06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.722+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cbc06b080 0x7f2cbc06d540 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc007330 tx=0x7f2ccc00b040 comp rx=0 tx=0).stop 2023-11-24T08:11:39.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.722+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 msgr2=0x7f2cdc0ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:39.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.722+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 0x7f2cdc0ad280 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400e480 tx=0x7f2cc400e940 comp rx=0 tx=0).stop 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 shutdown_connections 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cdc153ba0 0x7f2cdc154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cdc1f1360 0x7f2cdc0ad280 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cbc06b080 0x7f2cbc06d540 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 --2- 172.21.15.106:0/1445922283 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cc8005180 0x7f2cdc099ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 >> 172.21.15.106:0/1445922283 conn(0x7f2cdc06a920 msgr2=0x7f2cdc15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 shutdown_connections 2023-11-24T08:11:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:39.723+0000 7f2ce2a31640 1 -- 172.21.15.106:0/1445922283 wait complete. 2023-11-24T08:11:39.743 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:11:39.743 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:11:39.743 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:11:39.744 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:11:39.744 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:11:39.744 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:11:39.744 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:11:39.744 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:11:39.744 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:11:39.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:39.744 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:11:39.798 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:11:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:11:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:11:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.798 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:39.799 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42216sec preferred_lft 42216sec 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a407:f7ff:fe84:63d0/64 scope link 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:11:39.800 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9079:d8ff:febb:c08c/64 scope link 2023-11-24T08:11:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.801 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:11:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:11:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::20e3:f2ff:fe93:487e/64 scope link 2023-11-24T08:11:39.801 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:39.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:39.802 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:11:39.888 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:11:39.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:39.892 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:11:39.945 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:11:39.945 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:11:39.945 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:11:39.945 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:11:39.945 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:11:39.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:39.946 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:11:39.946 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:11:39.946 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:11:39.946 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:11:39.946 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:11:39.946 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:11:40.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:40.125 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:11:40.173 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:11:40.173 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:11:40.174 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:11:40.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:40.175 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2023-11-24T08:11:40.271 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:11:40.271 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:11:40.328 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:11:40.328 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:11:40.328 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:11:40.381 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:11:40.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:40.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:11:40.515 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:11:40.515 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:11:40.547 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:11:40.548 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:11:40.548 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:11:40.548 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:11:40.548 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:11:40.549 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-24T08:11:40.549 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:11:40.549 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:11:40.550 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:11:40.550 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:11:40.553 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:11:40.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:40.553 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:11:40.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:40.646 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:11:40.706 INFO:teuthology.orchestra.run.smithi106.stdout:client6222 2023-11-24T08:11:40.707 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:40.707 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:40.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:11:41.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.014+0000 7fa1dac64640 1 Processor -- start 2023-11-24T08:11:41.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.015+0000 7fa1dac64640 1 -- start start 2023-11-24T08:11:41.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.016+0000 7fa1dac64640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa1d415df80 0x7fa1d40ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:41.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.016+0000 7fa1dac64640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d40cd470 0x7fa1d415b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:41.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.016+0000 7fa1dac64640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d415e3f0 con 0x7fa1d40b2250 2023-11-24T08:11:41.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.016+0000 7fa1dac64640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d415a990 con 0x7fa1d415df80 2023-11-24T08:11:41.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.016+0000 7fa1dac64640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d415ab80 con 0x7fa1d40cd470 2023-11-24T08:11:41.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d9201640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d40cd470 0x7fa1d415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:41.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa1d415df80 0x7fa1d40ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:41.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa1d415df80 0x7fa1d40ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:35174/0 (socket says 172.21.15.106:35174) 2023-11-24T08:11:41.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2962200557 learned_addr learned my addr 172.21.15.106:0/2962200557 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:41.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d40cd470 msgr2=0x7fa1d415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:41.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 --2- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d40cd470 0x7fa1d415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2962200557 >> v1:172.21.15.156:6789/0 conn(0x7fa1d40b2250 legacy=0x7fa1d40b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:41.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.017+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2962200557 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40b2d40 con 0x7fa1d415df80 2023-11-24T08:11:41.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.019+0000 7fa1d3fff640 1 --2- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d415df80 0x7fa1d40ccf30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009870 tx=0x7fa1c002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dbf026957730f29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:41.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.019+0000 7fa1d37fe640 1 -- 172.21.15.106:0/2962200557 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0030840 con 0x7fa1d415df80 2023-11-24T08:11:41.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.019+0000 7fa1d37fe640 1 -- 172.21.15.106:0/2962200557 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c00309a0 con 0x7fa1d415df80 2023-11-24T08:11:41.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.019+0000 7fa1d37fe640 1 -- 172.21.15.106:0/2962200557 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0030d30 con 0x7fa1d415df80 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 -- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d415df80 msgr2=0x7fa1d40ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d415df80 0x7fa1d40ccf30 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0009870 tx=0x7fa1c002b980 comp rx=0 tx=0).stop 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 -- 172.21.15.106:0/2962200557 shutdown_connections 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d40cd470 0x7fa1d415b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2962200557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d415df80 0x7fa1d40ccf30 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 -- 172.21.15.106:0/2962200557 >> 172.21.15.106:0/2962200557 conn(0x7fa1d406a920 msgr2=0x7fa1d406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 -- 172.21.15.106:0/2962200557 shutdown_connections 2023-11-24T08:11:41.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.020+0000 7fa1dac64640 1 -- 172.21.15.106:0/2962200557 wait complete. 2023-11-24T08:11:41.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.022+0000 7fa1dac64640 1 Processor -- start 2023-11-24T08:11:41.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.022+0000 7fa1dac64640 1 -- start start 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1dac64640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d40b2250 0x7fa1d40c9530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1dac64640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 0x7fa1d414d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1dac64640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d415df80 0x7fa1d414d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1dac64640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4152180 con 0x7fa1d40cd470 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1dac64640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4152460 con 0x7fa1d40b2250 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1dac64640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d4152740 con 0x7fa1d415df80 2023-11-24T08:11:41.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 0x7fa1d414d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1d3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 0x7fa1d414d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44026/0 (socket says 172.21.15.106:44026) 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.023+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2075829305 learned_addr learned my addr 172.21.15.106:0/2075829305 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d9201640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d415df80 0x7fa1d414d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d8a00640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d40b2250 0x7fa1d40c9530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d415df80 msgr2=0x7fa1d414d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d3fff640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d415df80 0x7fa1d414d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d40b2250 msgr2=0x7fa1d40c9530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d3fff640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d40b2250 0x7fa1d40c9530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d3fff640 1 -- 172.21.15.106:0/2075829305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c00094d0 con 0x7fa1d40cd470 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d9201640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d415df80 0x7fa1d414d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d3fff640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 0x7fa1d414d110 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0000c00 tx=0x7fa1c0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:41.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0039a90 con 0x7fa1d40cd470 2023-11-24T08:11:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.024+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c0039bf0 con 0x7fa1d40cd470 2023-11-24T08:11:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.025+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1d4152a20 con 0x7fa1d40cd470 2023-11-24T08:11:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.025+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c0043410 con 0x7fa1d40cd470 2023-11-24T08:11:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.025+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d40c9a70 con 0x7fa1d40cd470 2023-11-24T08:11:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.026+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d40af9f0 con 0x7fa1d40cd470 2023-11-24T08:11:41.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.027+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa1c004c3b0 con 0x7fa1d40cd470 2023-11-24T08:11:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.028+0000 7fa1d17fa640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa1b806b0b0 0x7fa1b806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.028+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1c00abdf0 con 0x7fa1d40cd470 2023-11-24T08:11:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.028+0000 7fa1d8a00640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa1b806b0b0 0x7fa1b806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:41.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.029+0000 7fa1d8a00640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa1b806b0b0 0x7fa1b806d570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80015a0 tx=0x7fa1c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:41.034 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.032+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa1d40af9f0 con 0x7fa1d40cd470 2023-11-24T08:11:41.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.266+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1d40af9f0 con 0x7fa1d40cd470 2023-11-24T08:11:41.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.267+0000 7fa1d17fa640 1 -- 172.21.15.106:0/2075829305 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa1d40af9f0 con 0x7fa1d40cd470 2023-11-24T08:11:41.269 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa1b806b0b0 msgr2=0x7fa1b806d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa1b806b0b0 0x7fa1b806d570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fa1c80015a0 tx=0x7fa1c8009040 comp rx=0 tx=0).stop 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 msgr2=0x7fa1d414d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 0x7fa1d414d110 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fa1c0000c00 tx=0x7fa1c0004730 comp rx=0 tx=0).stop 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 shutdown_connections 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa1b806b0b0 0x7fa1b806d570 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa1d415df80 0x7fa1d414d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa1d40cd470 0x7fa1d414d110 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 --2- 172.21.15.106:0/2075829305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa1d40b2250 0x7fa1d40c9530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:41.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 >> 172.21.15.106:0/2075829305 conn(0x7fa1d406a920 msgr2=0x7fa1d40ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:41.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 shutdown_connections 2023-11-24T08:11:41.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:41.270+0000 7fa1dac64640 1 -- 172.21.15.106:0/2075829305 wait complete. 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:11:41.290 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:11:41.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.291 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.353 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42231sec preferred_lft 42231sec 2023-11-24T08:11:41.354 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::1831:a6ff:fe65:552d/64 scope link 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::683b:9fff:fe67:7152/64 scope link 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.355 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:11:41.356 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether a2:a3:a7:89:e5:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:11:41.356 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a0a3:a7ff:fe89:e507/64 scope link 2023-11-24T08:11:41.356 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:11:41.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.357 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:11:41.448 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:11:41.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.452 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:11:41.507 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:11:41.507 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:11:41.507 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:11:41.507 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:11:41.507 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:11:41.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.508 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:11:41.508 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:11:41.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:11:41.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:11:41.508 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:11:41.508 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:11:41.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.695 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:11:41.746 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:11:41.746 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:11:41.746 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:11:41.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.746 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-24T08:11:41.834 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:11:41.834 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:11:41.886 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:11:41.887 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:11:41.887 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:11:41.937 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:11:41.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:41.937 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:11:42.077 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:11:42.077 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:11:42.111 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:11:42.112 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:11:42.112 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:11:42.112 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 1@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:11:42.112 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:11:42.113 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-24T08:11:42.113 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:11:42.113 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:11:42.113 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:11:42.114 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:11:42.117 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:11:42.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:42.117 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:11:42.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:42.207 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:11:42.263 INFO:teuthology.orchestra.run.smithi156.stdout:client5125 2023-11-24T08:11:42.263 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:42.263 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:42.315 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:11:42.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.545+0000 7f105fa31640 1 Processor -- start 2023-11-24T08:11:42.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.546+0000 7f105fa31640 1 -- start start 2023-11-24T08:11:42.548 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.547+0000 7f105fa31640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f105806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:42.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.547+0000 7f105fa31640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10580ce6c0 0x7f10580cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:42.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.547+0000 7f105fa31640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10580cece0 con 0x7f10580cf450 2023-11-24T08:11:42.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.547+0000 7f105fa31640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1058156010 con 0x7f10580ce6c0 2023-11-24T08:11:42.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.547+0000 7f105fa31640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105806ce30 con 0x7f105815c7f0 2023-11-24T08:11:42.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.548+0000 7f105d7cd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f105806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.548+0000 7f105dfce640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10580ce6c0 0x7f10580cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.548+0000 7f105dfce640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f10580ce6c0 0x7f10580cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:35194/0 (socket says 172.21.15.106:35194) 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.548+0000 7f105d7cd640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f105806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:36696/0 (socket says 172.21.15.106:36696) 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.548+0000 7f105dfce640 1 -- 172.21.15.106:0/4059816045 learned_addr learned my addr 172.21.15.106:0/4059816045 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.548+0000 7f105cfcc640 1 --1- 172.21.15.106:0/4059816045 >> v1:172.21.15.156:6789/0 conn(0x7f10580cf450 0x7f10580cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:38786/0 (socket says 172.21.15.106:38786) 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.549+0000 7f105dfce640 1 -- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 msgr2=0x7f105806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:42.550 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.549+0000 7f105dfce640 1 --2- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f105806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.549+0000 7f105dfce640 1 -- 172.21.15.106:0/4059816045 >> v1:172.21.15.156:6789/0 conn(0x7f10580cf450 legacy=0x7f10580cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:42.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.549+0000 7f105dfce640 1 -- 172.21.15.106:0/4059816045 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105806d040 con 0x7f10580ce6c0 2023-11-24T08:11:42.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.549+0000 7f105d7cd640 1 --2- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f105806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:11:42.551 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.550+0000 7f105dfce640 1 --2- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580ce6c0 0x7f10580cf8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1050009870 tx=0x7f105002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=233b08fd64aab791 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:42.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f1057fff640 1 -- 172.21.15.106:0/4059816045 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050030840 con 0x7f10580ce6c0 2023-11-24T08:11:42.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f1057fff640 1 -- 172.21.15.106:0/4059816045 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10500309a0 con 0x7f10580ce6c0 2023-11-24T08:11:42.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f1057fff640 1 -- 172.21.15.106:0/4059816045 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050030d30 con 0x7f10580ce6c0 2023-11-24T08:11:42.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 -- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580ce6c0 msgr2=0x7f10580cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:42.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 --2- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580ce6c0 0x7f10580cf8d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f1050009870 tx=0x7f105002b980 comp rx=0 tx=0).stop 2023-11-24T08:11:42.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 -- 172.21.15.106:0/4059816045 shutdown_connections 2023-11-24T08:11:42.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 --2- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580ce6c0 0x7f10580cf8d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 --2- 172.21.15.106:0/4059816045 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f105806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 -- 172.21.15.106:0/4059816045 >> 172.21.15.106:0/4059816045 conn(0x7f105806a920 msgr2=0x7f105806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:42.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.551+0000 7f105fa31640 1 -- 172.21.15.106:0/4059816045 shutdown_connections 2023-11-24T08:11:42.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.552+0000 7f105fa31640 1 -- 172.21.15.106:0/4059816045 wait complete. 2023-11-24T08:11:42.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.553+0000 7f105fa31640 1 Processor -- start 2023-11-24T08:11:42.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.553+0000 7f105fa31640 1 -- start start 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.554+0000 7f105fa31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 0x7f1058158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.554+0000 7f105fa31640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580cf450 0x7f10580add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.554+0000 7f105fa31640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f10580b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.554+0000 7f105fa31640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058158880 con 0x7f10580ce6c0 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.554+0000 7f105fa31640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058157c70 con 0x7f10580cf450 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.554+0000 7f105fa31640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1058157f50 con 0x7f105815c7f0 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 0x7f1058158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:42.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 0x7f1058158d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44040/0 (socket says 172.21.15.106:44040) 2023-11-24T08:11:42.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 -- 172.21.15.106:0/1529801274 learned_addr learned my addr 172.21.15.106:0/1529801274 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105cfcc640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580cf450 0x7f10580add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105dfce640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f10580b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 -- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 msgr2=0x7f10580b20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f10580b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 -- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580cf450 msgr2=0x7f10580add10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580cf450 0x7f10580add10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 -- 172.21.15.106:0/1529801274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c009a70 con 0x7f10580ce6c0 2023-11-24T08:11:42.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105cfcc640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580cf450 0x7f10580add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.555+0000 7f105d7cd640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 0x7f1058158d10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f104c00efc0 tx=0x7f104c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.556+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c014680 con 0x7f10580ce6c0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.556+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c00f040 con 0x7f10580ce6c0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.556+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c014a00 con 0x7f10580ce6c0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.556+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10500094d0 con 0x7f10580ce6c0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.556+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1058158510 con 0x7f10580ce6c0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.557+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10580b1d30 con 0x7f10580ce6c0 2023-11-24T08:11:42.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.558+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f104c014b60 con 0x7f10580ce6c0 2023-11-24T08:11:42.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.559+0000 7f1055ffb640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f103c06b080 0x7f103c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:42.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.559+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f104c08ab30 con 0x7f10580ce6c0 2023-11-24T08:11:42.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.559+0000 7f105cfcc640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f103c06b080 0x7f103c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:42.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.560+0000 7f105cfcc640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f103c06b080 0x7f103c06d540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f105815ffa0 tx=0x7f1044009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:42.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.564+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f104c0556d0 con 0x7f10580ce6c0 2023-11-24T08:11:42.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.793+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1058067c80 con 0x7f10580ce6c0 2023-11-24T08:11:42.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.848+0000 7f1055ffb640 1 -- 172.21.15.106:0/1529801274 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1058067c80 con 0x7f10580ce6c0 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f103c06b080 msgr2=0x7f103c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f103c06b080 0x7f103c06d540 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f105815ffa0 tx=0x7f1044009040 comp rx=0 tx=0).stop 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 msgr2=0x7f1058158d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 0x7f1058158d10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f104c00efc0 tx=0x7f104c00c490 comp rx=0 tx=0).stop 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 shutdown_connections 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f103c06b080 0x7f103c06d540 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f105815c7f0 0x7f10580b20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10580cf450 0x7f10580add10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 --2- 172.21.15.106:0/1529801274 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10580ce6c0 0x7f1058158d10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:42.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.851+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 >> 172.21.15.106:0/1529801274 conn(0x7f105806a920 msgr2=0x7f10580af590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:42.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.852+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 shutdown_connections 2023-11-24T08:11:42.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:42.852+0000 7f105fa31640 1 -- 172.21.15.106:0/1529801274 wait complete. 2023-11-24T08:11:42.874 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:11:43.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.083+0000 7f48d293c640 1 Processor -- start 2023-11-24T08:11:43.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.083+0000 7f48d293c640 1 -- start start 2023-11-24T08:11:43.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.084+0000 7f48d293c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f48cc15a1c0 0x7f48cc070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d293c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc0b15e0 0x7f48cc15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d293c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48cc06d640 0x7f48cc06da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d293c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0cd920 con 0x7f48cc06d640 2023-11-24T08:11:43.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d293c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48cc0b1a20 con 0x7f48cc15a1c0 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d293c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc06dfb0 con 0x7f48cc0b15e0 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48cc06d640 0x7f48cc06da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f48cc06d640 0x7f48cc06da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44050/0 (socket says 172.21.15.106:44050) 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 -- 172.21.15.106:0/1272623590 learned_addr learned my addr 172.21.15.106:0/1272623590 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d1139640 1 --2- 172.21.15.106:0/1272623590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc0b15e0 0x7f48cc15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d193a640 1 --2- 172.21.15.106:0/1272623590 >> v2:172.21.15.156:3300/0 conn(0x7f48cc15a1c0 0x7f48cc070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.088 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 -- 172.21.15.106:0/1272623590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc0b15e0 msgr2=0x7f48cc15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:43.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 --2- 172.21.15.106:0/1272623590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc0b15e0 0x7f48cc15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 -- 172.21.15.106:0/1272623590 >> v2:172.21.15.156:3300/0 conn(0x7f48cc15a1c0 msgr2=0x7f48cc070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:43.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.085+0000 7f48d213b640 1 --2- 172.21.15.106:0/1272623590 >> v2:172.21.15.156:3300/0 conn(0x7f48cc15a1c0 0x7f48cc070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.086+0000 7f48d213b640 1 -- 172.21.15.106:0/1272623590 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48cc15e2e0 con 0x7f48cc06d640 2023-11-24T08:11:43.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.087+0000 7f48d213b640 1 --2- 172.21.15.106:0/1272623590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc06da70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0ccc20 tx=0x7f48c402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fe9fb3a953242db server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:43.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.087+0000 7f48d0938640 1 -- 172.21.15.106:0/1272623590 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c402cb00 con 0x7f48cc06d640 2023-11-24T08:11:43.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.087+0000 7f48d0938640 1 -- 172.21.15.106:0/1272623590 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c402cc60 con 0x7f48cc06d640 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.087+0000 7f48d0938640 1 -- 172.21.15.106:0/1272623590 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c40343f0 con 0x7f48cc06d640 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 -- 172.21.15.106:0/1272623590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 msgr2=0x7f48cc06da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 --2- 172.21.15.106:0/1272623590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc06da70 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0ccc20 tx=0x7f48c402bf60 comp rx=0 tx=0).stop 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 -- 172.21.15.106:0/1272623590 shutdown_connections 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 --2- 172.21.15.106:0/1272623590 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc06da70 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 --2- 172.21.15.106:0/1272623590 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc0b15e0 0x7f48cc15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 --2- 172.21.15.106:0/1272623590 >> v2:172.21.15.156:3300/0 conn(0x7f48cc15a1c0 0x7f48cc070ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 -- 172.21.15.106:0/1272623590 >> 172.21.15.106:0/1272623590 conn(0x7f48cc0cbb40 msgr2=0x7f48cc0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 -- 172.21.15.106:0/1272623590 shutdown_connections 2023-11-24T08:11:43.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.088+0000 7f48d293c640 1 -- 172.21.15.106:0/1272623590 wait complete. 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.090+0000 7f48d293c640 1 Processor -- start 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.090+0000 7f48d293c640 1 -- start start 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d293c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc0c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d293c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0b15e0 0x7f48cc0c5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d293c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15a1c0 0x7f48cc1558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d293c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0c41e0 con 0x7f48cc06d640 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d293c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0c44c0 con 0x7f48cc0b15e0 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d293c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48cc0c47a0 con 0x7f48cc15a1c0 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc0c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc0c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44060/0 (socket says 172.21.15.106:44060) 2023-11-24T08:11:43.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 -- 172.21.15.106:0/243210428 learned_addr learned my addr 172.21.15.106:0/243210428 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:43.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 -- 172.21.15.106:0/243210428 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15a1c0 msgr2=0x7f48cc1558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15a1c0 0x7f48cc1558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 -- 172.21.15.106:0/243210428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0b15e0 msgr2=0x7f48cc0c5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0b15e0 0x7f48cc0c5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.091+0000 7f48d193a640 1 -- 172.21.15.106:0/243210428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c4009b30 con 0x7f48cc06d640 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.092+0000 7f48d193a640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc0c7850 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f48c800d800 tx=0x7f48c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.092+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800b6c0 con 0x7f48cc06d640 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.092+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48c800b820 con 0x7f48cc06d640 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.092+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48cc0c4a80 con 0x7f48cc06d640 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.092+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48c800bbb0 con 0x7f48cc06d640 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.092+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48cc155e20 con 0x7f48cc06d640 2023-11-24T08:11:43.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.093+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48cc04e840 con 0x7f48cc06d640 2023-11-24T08:11:43.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.094+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f48c800bd10 con 0x7f48cc06d640 2023-11-24T08:11:43.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.096+0000 7f48c27fc640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006b080 0x7f48b006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.096+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f48c808a080 con 0x7f48cc06d640 2023-11-24T08:11:43.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.096+0000 7f48d1139640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006b080 0x7f48b006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.096+0000 7f48d1139640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006b080 0x7f48b006d540 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0c9420 tx=0x7f48bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:43.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.100+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f48c80bce90 con 0x7f48cc06d640 2023-11-24T08:11:43.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.313+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f48cc067de0 con 0x7f48b006b080 2023-11-24T08:11:43.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.317+0000 7f48c27fc640 1 -- 172.21.15.106:0/243210428 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f48cc067de0 con 0x7f48b006b080 2023-11-24T08:11:43.318 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:11:43.318 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:11:43.318 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2023-11-24T08:11:43.318 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:11:43.319 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:11:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006b080 msgr2=0x7f48b006d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006b080 0x7f48b006d540 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f48cc0c9420 tx=0x7f48bc008040 comp rx=0 tx=0).stop 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 msgr2=0x7f48cc0c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc0c7850 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f48c800d800 tx=0x7f48c800dcc0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 shutdown_connections 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f48b006b080 0x7f48b006d540 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f48cc15a1c0 0x7f48cc1558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48cc0b15e0 0x7f48cc0c5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.319+0000 7f48d293c640 1 --2- 172.21.15.106:0/243210428 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f48cc06d640 0x7f48cc0c7850 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.320+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 >> 172.21.15.106:0/243210428 conn(0x7f48cc0cbb40 msgr2=0x7f48cc071d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.320+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 shutdown_connections 2023-11-24T08:11:43.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.320+0000 7f48d293c640 1 -- 172.21.15.106:0/243210428 wait complete. 2023-11-24T08:11:43.341 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:11:43.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.555+0000 7faa9e7ac640 1 Processor -- start 2023-11-24T08:11:43.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.556+0000 7faa9e7ac640 1 -- start start 2023-11-24T08:11:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.557+0000 7faa9e7ac640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa98143c80 0x7faa9806fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.557+0000 7faa9e7ac640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7faa980d0c00 0x7faa9806a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.557+0000 7faa9e7ac640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faa9806aa40 0x7faa9806ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.557+0000 7faa9e7ac640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9806aef0 con 0x7faa9806aa40 2023-11-24T08:11:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.557+0000 7faa9e7ac640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa98144280 con 0x7faa980d0c00 2023-11-24T08:11:43.559 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.557+0000 7faa9e7ac640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa98072510 con 0x7faa98143c80 2023-11-24T08:11:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faa9806aa40 0x7faa9806ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7faa9806aa40 0x7faa9806ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44076/0 (socket says 172.21.15.106:44076) 2023-11-24T08:11:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 -- 172.21.15.106:0/2756357943 learned_addr learned my addr 172.21.15.106:0/2756357943 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:43.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9cfa9640 1 --2- 172.21.15.106:0/2756357943 >> v2:172.21.15.156:3300/0 conn(0x7faa980d0c00 0x7faa9806a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9d7aa640 1 --2- 172.21.15.106:0/2756357943 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa98143c80 0x7faa9806fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 -- 172.21.15.106:0/2756357943 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa98143c80 msgr2=0x7faa9806fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 --2- 172.21.15.106:0/2756357943 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa98143c80 0x7faa9806fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 -- 172.21.15.106:0/2756357943 >> v2:172.21.15.156:3300/0 conn(0x7faa980d0c00 msgr2=0x7faa9806a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 --2- 172.21.15.106:0/2756357943 >> v2:172.21.15.156:3300/0 conn(0x7faa980d0c00 0x7faa9806a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9dfab640 1 -- 172.21.15.106:0/2756357943 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa98071f90 con 0x7faa9806aa40 2023-11-24T08:11:43.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.558+0000 7faa9cfa9640 1 --2- 172.21.15.106:0/2756357943 >> v2:172.21.15.156:3300/0 conn(0x7faa980d0c00 0x7faa9806a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:11:43.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.560+0000 7faa9dfab640 1 --2- 172.21.15.106:0/2756357943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa9806ead0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faa9806c500 tx=0x7faa9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fcfe25152e8eb68 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:43.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.560+0000 7faa8ffff640 1 -- 172.21.15.106:0/2756357943 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9002ca80 con 0x7faa9806aa40 2023-11-24T08:11:43.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.560+0000 7faa8ffff640 1 -- 172.21.15.106:0/2756357943 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9002cbe0 con 0x7faa9806aa40 2023-11-24T08:11:43.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.560+0000 7faa8ffff640 1 -- 172.21.15.106:0/2756357943 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa900343f0 con 0x7faa9806aa40 2023-11-24T08:11:43.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.560+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756357943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 msgr2=0x7faa9806ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.560+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756357943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa9806ead0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7faa9806c500 tx=0x7faa9002bee0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756357943 shutdown_connections 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756357943 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa9806ead0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756357943 >> v2:172.21.15.156:3300/0 conn(0x7faa980d0c00 0x7faa9806a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756357943 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa98143c80 0x7faa9806fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756357943 >> 172.21.15.106:0/2756357943 conn(0x7faa980d0000 msgr2=0x7faa980d0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756357943 shutdown_connections 2023-11-24T08:11:43.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.561+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756357943 wait complete. 2023-11-24T08:11:43.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.562+0000 7faa9e7ac640 1 Processor -- start 2023-11-24T08:11:43.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 -- start start 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa981f6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa980d0c00 0x7faa9806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa98143c80 0x7faa981331e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa980c8690 con 0x7faa9806aa40 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa98137540 con 0x7faa98143c80 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.563+0000 7faa9e7ac640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa981f6200 con 0x7faa980d0c00 2023-11-24T08:11:43.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa981f6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa981f6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:44080/0 (socket says 172.21.15.106:44080) 2023-11-24T08:11:43.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 -- 172.21.15.106:0/2756353293 learned_addr learned my addr 172.21.15.106:0/2756353293 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 -- 172.21.15.106:0/2756353293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa980d0c00 msgr2=0x7faa9806f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa980d0c00 0x7faa9806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 -- 172.21.15.106:0/2756353293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa98143c80 msgr2=0x7faa981331e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa98143c80 0x7faa981331e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 -- 172.21.15.106:0/2756353293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa90009b30 con 0x7faa9806aa40 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.564+0000 7faa9d7aa640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa981f6c70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7faa9400e870 tx=0x7faa9400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.565+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa940094f0 con 0x7faa9806aa40 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.565+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9400c8c0 con 0x7faa9806aa40 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.565+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9400cc70 con 0x7faa9806aa40 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.565+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa98133d80 con 0x7faa9806aa40 2023-11-24T08:11:43.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.565+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa981f6750 con 0x7faa9806aa40 2023-11-24T08:11:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.566+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9806ead0 con 0x7faa9806aa40 2023-11-24T08:11:43.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.567+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7faa94013470 con 0x7faa9806aa40 2023-11-24T08:11:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.568+0000 7faa8dffb640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faa7c06b080 0x7faa7c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:43.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.568+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa94089b60 con 0x7faa9806aa40 2023-11-24T08:11:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.568+0000 7faa9cfa9640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faa7c06b080 0x7faa7c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:43.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.569+0000 7faa9cfa9640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faa7c06b080 0x7faa7c06d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faa980b35b0 tx=0x7faa88008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:43.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.573+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7faa94054810 con 0x7faa9806aa40 2023-11-24T08:11:43.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.767+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faa9804e840 con 0x7faa9806aa40 2023-11-24T08:11:43.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.767+0000 7faa8dffb640 1 -- 172.21.15.106:0/2756353293 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7faa940170c0 con 0x7faa9806aa40 2023-11-24T08:11:43.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faa7c06b080 msgr2=0x7faa7c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faa7c06b080 0x7faa7c06d540 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7faa980b35b0 tx=0x7faa88008040 comp rx=0 tx=0).stop 2023-11-24T08:11:43.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 msgr2=0x7faa981f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:43.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa981f6c70 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7faa9400e870 tx=0x7faa9400ed30 comp rx=0 tx=0).stop 2023-11-24T08:11:43.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 shutdown_connections 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faa98143c80 0x7faa981331e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7faa7c06b080 0x7faa7c06d540 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7faa980d0c00 0x7faa9806f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 --2- 172.21.15.106:0/2756353293 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7faa9806aa40 0x7faa981f6c70 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 >> 172.21.15.106:0/2756353293 conn(0x7faa980d0000 msgr2=0x7faa980cf2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 shutdown_connections 2023-11-24T08:11:43.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.770+0000 7faa9e7ac640 1 -- 172.21.15.106:0/2756353293 wait complete. 2023-11-24T08:11:43.791 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol6902 2023-11-24T08:11:44.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.999+0000 7fe219490640 1 Processor -- start 2023-11-24T08:11:44.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:43.999+0000 7fe219490640 1 -- start start 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe219490640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe214072160 0x7fe21415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe219490640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe2140c49f0 0x7fe21406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe219490640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21415ec90 con 0x7fe21406dc40 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe219490640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21406e090 con 0x7fe214072160 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe219490640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2140cd530 con 0x7fe2140c49f0 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe212ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe214072160 0x7fe21415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe2137fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe21406dc40 0x7fe2140cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40674/0 (socket says 172.21.15.106:40674) 2023-11-24T08:11:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.001+0000 7fe2137fe640 1 -- 172.21.15.106:0/4283245011 learned_addr learned my addr 172.21.15.106:0/4283245011 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.002+0000 7fe212ffd640 1 -- 172.21.15.106:0/4283245011 >> v2:172.21.15.156:3300/0 conn(0x7fe2140c49f0 msgr2=0x7fe21406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.002+0000 7fe212ffd640 1 --2- 172.21.15.106:0/4283245011 >> v2:172.21.15.156:3300/0 conn(0x7fe2140c49f0 0x7fe21406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.002+0000 7fe212ffd640 1 -- 172.21.15.106:0/4283245011 >> v1:172.21.15.106:6789/0 conn(0x7fe21406dc40 legacy=0x7fe2140cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.002+0000 7fe212ffd640 1 -- 172.21.15.106:0/4283245011 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2140c4ed0 con 0x7fe214072160 2023-11-24T08:11:44.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.003+0000 7fe212ffd640 1 --2- 172.21.15.106:0/4283245011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe21415e750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe2140cef60 tx=0x7fe20402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6c1717650fd3868 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:44.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.003+0000 7fe2127fc640 1 -- 172.21.15.106:0/4283245011 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204031830 con 0x7fe214072160 2023-11-24T08:11:44.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.003+0000 7fe2127fc640 1 -- 172.21.15.106:0/4283245011 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe204031990 con 0x7fe214072160 2023-11-24T08:11:44.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.003+0000 7fe2127fc640 1 -- 172.21.15.106:0/4283245011 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe204031d20 con 0x7fe214072160 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 -- 172.21.15.106:0/4283245011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 msgr2=0x7fe21415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 --2- 172.21.15.106:0/4283245011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe21415e750 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fe2140cef60 tx=0x7fe20402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 -- 172.21.15.106:0/4283245011 shutdown_connections 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 --2- 172.21.15.106:0/4283245011 >> v2:172.21.15.156:3300/0 conn(0x7fe2140c49f0 0x7fe21406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 --2- 172.21.15.106:0/4283245011 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe21415e750 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 -- 172.21.15.106:0/4283245011 >> 172.21.15.106:0/4283245011 conn(0x7fe2140b0c50 msgr2=0x7fe2140b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 -- 172.21.15.106:0/4283245011 shutdown_connections 2023-11-24T08:11:44.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.004+0000 7fe219490640 1 -- 172.21.15.106:0/4283245011 wait complete. 2023-11-24T08:11:44.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.006+0000 7fe219490640 1 Processor -- start 2023-11-24T08:11:44.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.006+0000 7fe219490640 1 -- start start 2023-11-24T08:11:44.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe219490640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21406dc40 0x7fe214143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe219490640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe214143e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe219490640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2140c49f0 0x7fe2140cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe219490640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2140caae0 con 0x7fe214072160 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe219490640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415d3d0 con 0x7fe21406dc40 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe219490640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21415ce10 con 0x7fe2140c49f0 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe20a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe214143e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe20a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe214143e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33308/0 (socket says 172.21.15.106:33308) 2023-11-24T08:11:44.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.007+0000 7fe20a7fc640 1 -- 172.21.15.106:0/569652464 learned_addr learned my addr 172.21.15.106:0/569652464 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe212ffd640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21406dc40 0x7fe214143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe2137fe640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2140c49f0 0x7fe2140cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20a7fc640 1 -- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2140c49f0 msgr2=0x7fe2140cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20a7fc640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2140c49f0 0x7fe2140cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20a7fc640 1 -- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21406dc40 msgr2=0x7fe214143540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20a7fc640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21406dc40 0x7fe214143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20a7fc640 1 -- 172.21.15.106:0/569652464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204009400 con 0x7fe214072160 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20a7fc640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe214143e70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00b4a0 tx=0x7fe1fc00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc010610 con 0x7fe214072160 2023-11-24T08:11:44.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1fc010770 con 0x7fe214072160 2023-11-24T08:11:44.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1fc010b20 con 0x7fe214072160 2023-11-24T08:11:44.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.008+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21415d0f0 con 0x7fe214072160 2023-11-24T08:11:44.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.009+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21409c740 con 0x7fe214072160 2023-11-24T08:11:44.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.010+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2140caf70 con 0x7fe214072160 2023-11-24T08:11:44.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.010+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fe1fc010c80 con 0x7fe214072160 2023-11-24T08:11:44.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.011+0000 7fe20bfff640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1f406b0b0 0x7fe1f406d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.011+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1fc0896b0 con 0x7fe214072160 2023-11-24T08:11:44.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.012+0000 7fe212ffd640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1f406b0b0 0x7fe1f406d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.012+0000 7fe212ffd640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1f406b0b0 0x7fe1f406d570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe204036000 tx=0x7fe204008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:44.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.016+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe2140caf70 con 0x7fe214072160 2023-11-24T08:11:44.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.233+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol6902", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2140c5770 con 0x7fe1f406b0b0 2023-11-24T08:11:44.248 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:44.247+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:44.248 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:44.247+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:44.249 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:44.247+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:44.249 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:44.247+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:44.249 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:44.247+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:44.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.300+0000 7fe20bfff640 1 -- 172.21.15.106:0/569652464 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe204031630 con 0x7fe1f406b0b0 2023-11-24T08:11:44.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.303+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1f406b0b0 msgr2=0x7fe1f406d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.303+0000 7fe219490640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1f406b0b0 0x7fe1f406d570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe204036000 tx=0x7fe204008030 comp rx=0 tx=0).stop 2023-11-24T08:11:44.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.303+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 msgr2=0x7fe214143e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.303+0000 7fe219490640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe214143e70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe1fc00b4a0 tx=0x7fe1fc00b960 comp rx=0 tx=0).stop 2023-11-24T08:11:44.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 shutdown_connections 2023-11-24T08:11:44.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1f406b0b0 0x7fe1f406d570 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe2140c49f0 0x7fe2140cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe214072160 0x7fe214143e70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 --2- 172.21.15.106:0/569652464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe21406dc40 0x7fe214143540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 >> 172.21.15.106:0/569652464 conn(0x7fe2140b0c50 msgr2=0x7fe2140704c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:44.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 shutdown_connections 2023-11-24T08:11:44.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.304+0000 7fe219490640 1 -- 172.21.15.106:0/569652464 wait complete. 2023-11-24T08:11:44.327 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol6902 subvol_snap5006 2023-11-24T08:11:44.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.530+0000 7fa31c8bb640 1 Processor -- start 2023-11-24T08:11:44.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.531+0000 7fa31c8bb640 1 -- start start 2023-11-24T08:11:44.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.532+0000 7fa31c8bb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3180b38c0 0x7fa3180cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.532+0000 7fa31c8bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180999d0 0x7fa318099e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.532+0000 7fa31c8bb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3180cf8a0 con 0x7fa31815b6c0 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.532+0000 7fa31c8bb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa31809a340 con 0x7fa3180b38c0 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.532+0000 7fa31c8bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180cd210 con 0x7fa3180999d0 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3180b38c0 0x7fa3180cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa3180b38c0 0x7fa3180cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33338/0 (socket says 172.21.15.106:33338) 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 -- 172.21.15.106:0/3056772391 learned_addr learned my addr 172.21.15.106:0/3056772391 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa316d9d640 1 --2- 172.21.15.106:0/3056772391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180999d0 0x7fa318099e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 -- 172.21.15.106:0/3056772391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180999d0 msgr2=0x7fa318099e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 --2- 172.21.15.106:0/3056772391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180999d0 0x7fa318099e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 -- 172.21.15.106:0/3056772391 >> v1:172.21.15.156:6789/0 conn(0x7fa31815b6c0 legacy=0x7fa3180cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:44.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.533+0000 7fa315d9b640 1 -- 172.21.15.106:0/3056772391 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3180cefb0 con 0x7fa3180b38c0 2023-11-24T08:11:44.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.534+0000 7fa315d9b640 1 --2- 172.21.15.106:0/3056772391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3180b38c0 0x7fa3180cbe30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa30002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9199fcf9e2f96f29 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:44.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.534+0000 7fa31559a640 1 -- 172.21.15.106:0/3056772391 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3000317c0 con 0x7fa3180b38c0 2023-11-24T08:11:44.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31559a640 1 -- 172.21.15.106:0/3056772391 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa300031920 con 0x7fa3180b38c0 2023-11-24T08:11:44.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31559a640 1 -- 172.21.15.106:0/3056772391 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa300031cb0 con 0x7fa3180b38c0 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3056772391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3180b38c0 msgr2=0x7fa3180cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3056772391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3180b38c0 0x7fa3180cbe30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fa300009870 tx=0x7fa30002b860 comp rx=0 tx=0).stop 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3056772391 shutdown_connections 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3056772391 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180999d0 0x7fa318099e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3056772391 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa3180b38c0 0x7fa3180cbe30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3056772391 >> 172.21.15.106:0/3056772391 conn(0x7fa3180b2da0 msgr2=0x7fa3180b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:44.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3056772391 shutdown_connections 2023-11-24T08:11:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.535+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3056772391 wait complete. 2023-11-24T08:11:44.538 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.537+0000 7fa31c8bb640 1 Processor -- start 2023-11-24T08:11:44.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.537+0000 7fa31c8bb640 1 -- start start 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.538+0000 7fa31c8bb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3180999d0 0x7fa31815a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.538+0000 7fa31c8bb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180b38c0 0x7fa3180cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.538+0000 7fa31c8bb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 0x7fa31809bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.538+0000 7fa31c8bb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31815ad50 con 0x7fa31815b6c0 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.538+0000 7fa31c8bb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180b3f50 con 0x7fa3180999d0 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.538+0000 7fa31c8bb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3180b41f0 con 0x7fa3180b38c0 2023-11-24T08:11:44.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 0x7fa31809bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 0x7fa31809bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33342/0 (socket says 172.21.15.106:33342) 2023-11-24T08:11:44.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 -- 172.21.15.106:0/3865390207 learned_addr learned my addr 172.21.15.106:0/3865390207 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa315d9b640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180b38c0 0x7fa3180cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa31659c640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3180999d0 0x7fa31815a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 -- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180b38c0 msgr2=0x7fa3180cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180b38c0 0x7fa3180cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 -- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3180999d0 msgr2=0x7fa31815a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3180999d0 0x7fa31815a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 -- 172.21.15.106:0/3865390207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000094d0 con 0x7fa31815b6c0 2023-11-24T08:11:44.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa31659c640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3180999d0 0x7fa31815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.539+0000 7fa316d9d640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 0x7fa31809bdd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c550 tx=0x7fa31000ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.540+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310004220 con 0x7fa31815b6c0 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.540+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa310004380 con 0x7fa31815b6c0 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.540+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa310003830 con 0x7fa31815b6c0 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.540+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3180b44d0 con 0x7fa31815b6c0 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.540+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3180c9a90 con 0x7fa31815b6c0 2023-11-24T08:11:44.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.541+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3180b49f0 con 0x7fa31815b6c0 2023-11-24T08:11:44.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.542+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7fa310003990 con 0x7fa31815b6c0 2023-11-24T08:11:44.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.543+0000 7fa30effd640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa2f806b080 0x7fa2f806d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:44.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.544+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3100897b0 con 0x7fa31815b6c0 2023-11-24T08:11:44.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.544+0000 7fa31659c640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa2f806b080 0x7fa2f806d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:44.546 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.545+0000 7fa31659c640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa2f806b080 0x7fa2f806d540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa308007e90 tx=0x7fa3080073a0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:44.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.548+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa3180b49f0 con 0x7fa31815b6c0 2023-11-24T08:11:44.772 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.770+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol6902", "snap_name": "subvol_snap5006", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3180cf530 con 0x7fa2f806b080 2023-11-24T08:11:44.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.801+0000 7fa30effd640 1 -- 172.21.15.106:0/3865390207 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa30800cdb0 con 0x7fa2f806b080 2023-11-24T08:11:44.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa2f806b080 msgr2=0x7fa2f806d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa2f806b080 0x7fa2f806d540 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa308007e90 tx=0x7fa3080073a0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 msgr2=0x7fa31809bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 0x7fa31809bdd0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c550 tx=0x7fa31000ca10 comp rx=0 tx=0).stop 2023-11-24T08:11:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 shutdown_connections 2023-11-24T08:11:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa31815b6c0 0x7fa31809bdd0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa2f806b080 0x7fa2f806d540 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa3180b38c0 0x7fa3180cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 --2- 172.21.15.106:0/3865390207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3180999d0 0x7fa31815a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:44.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.804+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 >> 172.21.15.106:0/3865390207 conn(0x7fa3180b2da0 msgr2=0x7fa3180ce0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:44.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.805+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 shutdown_connections 2023-11-24T08:11:44.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:44.805+0000 7fa31c8bb640 1 -- 172.21.15.106:0/3865390207 wait complete. 2023-11-24T08:11:44.826 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6902 subvol_snap5006 2023-11-24T08:11:45.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.044+0000 7f2ba0fa9640 1 Processor -- start 2023-11-24T08:11:45.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.044+0000 7f2ba0fa9640 1 -- start start 2023-11-24T08:11:45.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.045+0000 7f2ba0fa9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2b9c15b6c0 0x7f2b9c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2ba0fa9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0cf8a0 con 0x7f2b9c0999d0 2023-11-24T08:11:45.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2ba0fa9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9c0cd210 con 0x7f2b9c0b38c0 2023-11-24T08:11:45.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2ba0fa9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9c0cefb0 con 0x7f2b9c15b6c0 2023-11-24T08:11:45.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2b9ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2b9c0999d0 0x7f2b9c099de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40678/0 (socket says 172.21.15.106:40678) 2023-11-24T08:11:45.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2b9ad9d640 1 -- 172.21.15.106:0/1279387818 learned_addr learned my addr 172.21.15.106:0/1279387818 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:45.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2b9a59c640 1 --2- 172.21.15.106:0/1279387818 >> v2:172.21.15.156:3300/0 conn(0x7f2b9c15b6c0 0x7f2b9c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304806706 0 0) 0x7f2b9c0cf8a0 con 0x7f2b9c0999d0 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.046+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b88003520 con 0x7f2b9c0999d0 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3200995866 0 0) 0x7f2b88003520 con 0x7f2b9c0999d0 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 >> v2:172.21.15.156:3300/0 conn(0x7f2b9c15b6c0 msgr2=0x7f2b9c0cbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 --2- 172.21.15.106:0/1279387818 >> v2:172.21.15.156:3300/0 conn(0x7f2b9c15b6c0 0x7f2b9c0cbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 >> v1:172.21.15.156:6789/0 conn(0x7f2b9c0b38c0 legacy=0x7f2b9c0cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0cf250 con 0x7f2b9c0999d0 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2b94001670 con 0x7f2b9c0999d0 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 >> v1:172.21.15.106:6789/0 conn(0x7f2b9c0999d0 legacy=0x7f2b9c099de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b88003970 0x7f2b88003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b880047c0 0x7f2b88004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880052a0 0x7f2b8800fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0cefb0 con 0x7f2b880047c0 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b88010560 con 0x7f2b880052a0 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b88010720 con 0x7f2b88003970 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b880047c0 0x7f2b88004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.047+0000 7f2b9ad9d640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880052a0 0x7f2b8800fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b9a59c640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b88003970 0x7f2b88003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b88003970 msgr2=0x7f2b88003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b88003970 0x7f2b88003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880052a0 msgr2=0x7f2b8800fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880052a0 0x7f2b8800fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1279387818 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b88003520 con 0x7f2b880047c0 2023-11-24T08:11:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.048+0000 7f2b9ad9d640 1 -- 172.21.15.106:0/1279387818 reap_dead start 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.049+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b880047c0 0x7f2b88004c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2b84001620 tx=0x7f2b880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec7c35b5eb91bd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.049+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84005070 con 0x7f2b880047c0 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.049+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b8402e070 con 0x7f2b880047c0 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.049+0000 7f2b9959a640 1 -- 172.21.15.106:0/1279387818 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84005430 con 0x7f2b880047c0 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.049+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1279387818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b880047c0 msgr2=0x7f2b88004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.049+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b880047c0 0x7f2b88004c40 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f2b84001620 tx=0x7f2b880057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.050+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1279387818 shutdown_connections 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.050+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1279387818 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b880047c0 0x7f2b88004c40 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.050+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1279387818 >> 172.21.15.106:0/1279387818 conn(0x7f2b9c0b2da0 msgr2=0x7f2b9c0b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.050+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1279387818 shutdown_connections 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.050+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1279387818 wait complete. 2023-11-24T08:11:45.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.051+0000 7f2ba0fa9640 1 Processor -- start 2023-11-24T08:11:45.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.052+0000 7f2ba0fa9640 1 -- start start 2023-11-24T08:11:45.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.052+0000 7f2ba0fa9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880047c0 0x7f2b9c155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2ba0fa9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 0x7f2b9c0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2ba0fa9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b9c1ef740 0x7f2b9c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2ba0fa9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c155650 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2ba0fa9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c155b70 con 0x7f2b880047c0 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2ba0fa9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b9c0b58c0 con 0x7f2b9c1ef740 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 0x7f2b9c0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 0x7f2b9c0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33362/0 (socket says 172.21.15.106:33362) 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1308722381 learned_addr learned my addr 172.21.15.106:0/1308722381 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b9a59c640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880047c0 0x7f2b9c155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b9ad9d640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b9c1ef740 0x7f2b9c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b9c1ef740 msgr2=0x7f2b9c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b9c1ef740 0x7f2b9c15d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880047c0 msgr2=0x7f2b9c155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880047c0 0x7f2b9c155110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b99d9b640 1 -- 172.21.15.106:0/1308722381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9c0cb2a0 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.053+0000 7f2b9a59c640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880047c0 0x7f2b9c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.054+0000 7f2b99d9b640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 0x7f2b9c0c9b40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2b84001740 tx=0x7f2b84000fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.054+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b84007240 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.054+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b840073a0 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.054+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b9c0ca720 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.054+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840054a0 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.054+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b9c0cace0 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.055+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b9c067b80 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.056+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f2b84030030 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.057+0000 7f2b92ffd640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b7c06b080 0x7f2b7c06d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.057+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2b840aa600 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.057+0000 7f2b9a59c640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b7c06b080 0x7f2b7c06d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.058+0000 7f2b9a59c640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b7c06b080 0x7f2b7c06d540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0681f0 tx=0x7f2b8c00b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:45.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.062+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2b84075300 con 0x7f2b9c1ef2c0 2023-11-24T08:11:45.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.284+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6902", "snap_name": "subvol_snap5006", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b9c09a360 con 0x7f2b7c06b080 2023-11-24T08:11:45.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.334+0000 7f2b92ffd640 1 -- 172.21.15.106:0/1308722381 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2b9c09a360 con 0x7f2b7c06b080 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b7c06b080 msgr2=0x7f2b7c06d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b7c06b080 0x7f2b7c06d540 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c0681f0 tx=0x7f2b8c00b040 comp rx=0 tx=0).stop 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 msgr2=0x7f2b9c0c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 0x7f2b9c0c9b40 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f2b84001740 tx=0x7f2b84000fb0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 shutdown_connections 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2b7c06b080 0x7f2b7c06d540 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2b9c1ef740 0x7f2b9c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2b9c1ef2c0 0x7f2b9c0c9b40 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 --2- 172.21.15.106:0/1308722381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b880047c0 0x7f2b9c155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 >> 172.21.15.106:0/1308722381 conn(0x7f2b9c0b2da0 msgr2=0x7f2b9c06cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:45.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 shutdown_connections 2023-11-24T08:11:45.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.338+0000 7f2ba0fa9640 1 -- 172.21.15.106:0/1308722381 wait complete. 2023-11-24T08:11:45.359 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol6902 subvol_snap5006 2023-11-24T08:11:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.574+0000 7f6f43c23640 1 Processor -- start 2023-11-24T08:11:45.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.575+0000 7f6f43c23640 1 -- start start 2023-11-24T08:11:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.576+0000 7f6f43c23640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 0x7f6f3c06ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.576+0000 7f6f43c23640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c0c65b0 con 0x7f6f3c0d0d30 2023-11-24T08:11:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.576+0000 7f6f43c23640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f3c06ef50 con 0x7f6f3c1618a0 2023-11-24T08:11:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.576+0000 7f6f43c23640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0cf5b0 con 0x7f6f3c06e5e0 2023-11-24T08:11:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.577+0000 7f6f411be640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6f3c0d0d30 0x7f6f3c0ceea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40692/0 (socket says 172.21.15.106:40692) 2023-11-24T08:11:45.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.577+0000 7f6f411be640 1 -- 172.21.15.106:0/1522708730 learned_addr learned my addr 172.21.15.106:0/1522708730 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.577+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 99326964 0 0) 0x7f6f3c0c65b0 con 0x7f6f3c0d0d30 2023-11-24T08:11:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.577+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f24003520 con 0x7f6f3c0d0d30 2023-11-24T08:11:45.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.577+0000 7f6f421c0640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 0x7f6f3c06ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1907837074 0 0) 0x7f6f24003520 con 0x7f6f3c0d0d30 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 msgr2=0x7f6f3c06ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 0x7f6f3c06ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 >> v1:172.21.15.156:6789/0 conn(0x7f6f3c1618a0 legacy=0x7f6f3c165510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c165d20 con 0x7f6f3c0d0d30 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f6f30001670 con 0x7f6f3c0d0d30 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 >> v1:172.21.15.106:6789/0 conn(0x7f6f3c0d0d30 legacy=0x7f6f3c0ceea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 0x7f6f240039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f24004020 0x7f6f240047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004d00 0x7f6f24005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c0cf5b0 con 0x7f6f24004020 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2400bbc0 con 0x7f6f24004d00 2023-11-24T08:11:45.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f2400bd80 con 0x7f6f3c06e5e0 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f411be640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f24004020 0x7f6f240047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.578+0000 7f6f419bf640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 0x7f6f240039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.579+0000 7f6f421c0640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004d00 0x7f6f24005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.579+0000 7f6f411be640 1 -- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 msgr2=0x7f6f240039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.579+0000 7f6f411be640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c06e5e0 0x7f6f240039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.579+0000 7f6f411be640 1 -- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004d00 msgr2=0x7f6f24005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.579+0000 7f6f411be640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004d00 0x7f6f24005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.579+0000 7f6f411be640 1 -- 172.21.15.106:0/1522708730 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24003520 con 0x7f6f24004020 2023-11-24T08:11:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.580+0000 7f6f411be640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f24004020 0x7f6f240047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c0e0 tx=0x7f6f240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac6d2e7fb68195e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.580+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30005ad0 con 0x7f6f24004020 2023-11-24T08:11:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.580+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f30005c30 con 0x7f6f24004020 2023-11-24T08:11:45.584 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.580+0000 7f6f409bd640 1 -- 172.21.15.106:0/1522708730 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3002ca50 con 0x7f6f24004020 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.580+0000 7f6f43c23640 1 -- 172.21.15.106:0/1522708730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f24004020 msgr2=0x7f6f240047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.581+0000 7f6f43c23640 1 --2- 172.21.15.106:0/1522708730 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f24004020 0x7f6f240047c0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c0e0 tx=0x7f6f240057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.581+0000 7f6f421c0640 1 -- 172.21.15.106:0/1522708730 reap_dead start 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.581+0000 7f6f43c23640 1 -- 172.21.15.106:0/1522708730 shutdown_connections 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.581+0000 7f6f43c23640 1 -- 172.21.15.106:0/1522708730 >> 172.21.15.106:0/1522708730 conn(0x7f6f3c0d12d0 msgr2=0x7f6f3c0d16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.581+0000 7f6f43c23640 1 -- 172.21.15.106:0/1522708730 shutdown_connections 2023-11-24T08:11:45.585 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.581+0000 7f6f43c23640 1 -- 172.21.15.106:0/1522708730 wait complete. 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.582+0000 7f6f43c23640 1 Processor -- start 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.583+0000 7f6f43c23640 1 -- start start 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.583+0000 7f6f43c23640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004020 0x7f6f3c164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.583+0000 7f6f43c23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f3c0b3f80 0x7f6f3c1fb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.583+0000 7f6f43c23640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c0b6280 0x7f6f3c162de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f43c23640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c1fbab0 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f43c23640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c164e10 con 0x7f6f24004020 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f43c23640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3c162020 con 0x7f6f3c0b6280 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f3c0b3f80 0x7f6f3c1fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f3c0b3f80 0x7f6f3c1fb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33376/0 (socket says 172.21.15.106:33376) 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 -- 172.21.15.106:0/3165356799 learned_addr learned my addr 172.21.15.106:0/3165356799 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:45.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f421c0640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c0b6280 0x7f6f3c162de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f419bf640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004020 0x7f6f3c164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 -- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c0b6280 msgr2=0x7f6f3c162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6f3c0b6280 0x7f6f3c162de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 -- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004020 msgr2=0x7f6f3c164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004020 0x7f6f3c164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f411be640 1 -- 172.21.15.106:0/3165356799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3c162300 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.584+0000 7f6f419bf640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6f24004020 0x7f6f3c164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:45.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.585+0000 7f6f411be640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6f3c0b3f80 0x7f6f3c1fb570 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6f3000c0e0 tx=0x7f6f30001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.585+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f3003c040 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.585+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f3002dcb0 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.585+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30007270 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.585+0000 7f6f43c23640 1 -- 172.21.15.106:0/3165356799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3c1625e0 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.585+0000 7f6f43c23640 1 -- 172.21.15.106:0/3165356799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3c0b6700 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.586+0000 7f6f43c23640 1 -- 172.21.15.106:0/3165356799 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3c067b80 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.587+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f6f30032070 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.588+0000 7f6f3a7fc640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6f2806b0b0 0x7f6f2806d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:45.590 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.588+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f300ab4e0 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.588+0000 7f6f419bf640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6f2806b0b0 0x7f6f2806d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:45.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.589+0000 7f6f419bf640 1 --2- 172.21.15.106:0/3165356799 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6f2806b0b0 0x7f6f2806d570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c007fd0 tx=0x7f6f2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:45.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.592+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6f30076220 con 0x7f6f3c0b3f80 2023-11-24T08:11:45.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.815+0000 7f6f43c23640 1 -- 172.21.15.106:0/3165356799 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol6902", "snap_name": "subvol_snap5006", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f3c0cf040 con 0x7f6f2806b0b0 2023-11-24T08:11:45.839 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:45.836+0000 7f67aa1bf640 -1 mgr.server reply reply (2) No such file or directory snapshot 'b'subvol_snap5006'' does not exist 2023-11-24T08:11:45.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:45.836+0000 7f6f3a7fc640 1 -- 172.21.15.106:0/3165356799 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: -2 snapshot 'b'subvol_snap5006'' does not exist) v1 ==== 52+0+0 (secure 0 0 0) 0x7f6f3c0cf040 con 0x7f6f2806b0b0 2023-11-24T08:11:45.839 INFO:teuthology.orchestra.run.smithi106.stderr:Error ENOENT: snapshot 'b'subvol_snap5006'' does not exist 2023-11-24T08:11:45.854 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-24T08:11:45.855 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol6902 2023-11-24T08:11:46.023 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:11:46.032 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:11:46.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.068+0000 7f98767fa640 1 Processor -- start 2023-11-24T08:11:46.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.069+0000 7f98767fa640 1 -- start start 2023-11-24T08:11:46.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f98767fa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d7d0 0x7f98700cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f98767fa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98700d15e0 con 0x7f98700d1f40 2023-11-24T08:11:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f98767fa640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98700d0280 con 0x7f987015d3f0 2023-11-24T08:11:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f98767fa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98700b0eb0 con 0x7f987015d7d0 2023-11-24T08:11:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f9875ff9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f98700d1f40 0x7f98700ce280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40706/0 (socket says 172.21.15.106:40706) 2023-11-24T08:11:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f9875ff9640 1 -- 172.21.15.106:0/1802777487 learned_addr learned my addr 172.21.15.106:0/1802777487 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:46.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.070+0000 7f9874ff7640 1 --2- 172.21.15.106:0/1802777487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d7d0 0x7f98700cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170908973 0 0) 0x7f98700d15e0 con 0x7f98700d1f40 2023-11-24T08:11:46.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f985c003520 con 0x7f98700d1f40 2023-11-24T08:11:46.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256396495 0 0) 0x7f98700d0280 con 0x7f987015d3f0 2023-11-24T08:11:46.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 --> v1:172.21.15.156:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98700d15e0 con 0x7f987015d3f0 2023-11-24T08:11:46.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9874ff7640 1 -- 172.21.15.106:0/1802777487 >> v1:172.21.15.156:6789/0 conn(0x7f987015d3f0 legacy=0x7f98700d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9874ff7640 1 -- 172.21.15.106:0/1802777487 >> v1:172.21.15.106:6789/0 conn(0x7f98700d1f40 legacy=0x7f98700ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.071+0000 7f9874ff7640 1 -- 172.21.15.106:0/1802777487 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98700d1300 con 0x7f987015d7d0 2023-11-24T08:11:46.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.073+0000 7f9874ff7640 1 --2- 172.21.15.106:0/1802777487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d7d0 0x7f98700cdd40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98600097d0 tx=0x7f986002b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7f5967538ff1ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:46.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.073+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860031890 con 0x7f987015d7d0 2023-11-24T08:11:46.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.073+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98600319f0 con 0x7f987015d7d0 2023-11-24T08:11:46.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.073+0000 7f9867fff640 1 -- 172.21.15.106:0/1802777487 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9860031d80 con 0x7f987015d7d0 2023-11-24T08:11:46.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.073+0000 7f98767fa640 1 -- 172.21.15.106:0/1802777487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d7d0 msgr2=0x7f98700cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.074+0000 7f98767fa640 1 --2- 172.21.15.106:0/1802777487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d7d0 0x7f98700cdd40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98600097d0 tx=0x7f986002b930 comp rx=0 tx=0).stop 2023-11-24T08:11:46.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.074+0000 7f98767fa640 1 -- 172.21.15.106:0/1802777487 shutdown_connections 2023-11-24T08:11:46.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.074+0000 7f98767fa640 1 --2- 172.21.15.106:0/1802777487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d7d0 0x7f98700cdd40 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.074+0000 7f98767fa640 1 -- 172.21.15.106:0/1802777487 >> 172.21.15.106:0/1802777487 conn(0x7f987009c5b0 msgr2=0x7f98700d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:46.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.074+0000 7f98767fa640 1 -- 172.21.15.106:0/1802777487 shutdown_connections 2023-11-24T08:11:46.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.074+0000 7f98767fa640 1 -- 172.21.15.106:0/1802777487 wait complete. 2023-11-24T08:11:46.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.075+0000 7f98767fa640 1 Processor -- start 2023-11-24T08:11:46.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 -- start start 2023-11-24T08:11:46.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98700d1f40 0x7f98700c92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d3f0 0x7f98700b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 0x7f98701f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98700b8d30 con 0x7f987015d7d0 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98700c8210 con 0x7f98700d1f40 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.076+0000 7f98767fa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98700c84f0 con 0x7f987015d3f0 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 0x7f98701f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 0x7f98701f39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33390/0 (socket says 172.21.15.106:33390) 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 -- 172.21.15.106:0/1286189773 learned_addr learned my addr 172.21.15.106:0/1286189773 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f98757f8640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98700d1f40 0x7f98700c92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9874ff7640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d3f0 0x7f98700b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 -- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d3f0 msgr2=0x7f98700b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d3f0 0x7f98700b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 -- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98700d1f40 msgr2=0x7f98700c92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98700d1f40 0x7f98700c92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 -- 172.21.15.106:0/1286189773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860009400 con 0x7f987015d7d0 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f98757f8640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98700d1f40 0x7f98700c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.077+0000 7f9875ff9640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 0x7f98701f39d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f986800cac0 tx=0x7f986800cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.078+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986800ac30 con 0x7f987015d7d0 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.078+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986800ad90 con 0x7f987015d7d0 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.078+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98700c87d0 con 0x7f987015d7d0 2023-11-24T08:11:46.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.078+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9868012620 con 0x7f987015d7d0 2023-11-24T08:11:46.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.078+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98700c8d90 con 0x7f987015d7d0 2023-11-24T08:11:46.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.079+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98700c8f80 con 0x7f987015d7d0 2023-11-24T08:11:46.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.080+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 8) v1 ==== 89435+0+0 (secure 0 0 0) 0x7f9868007070 con 0x7f987015d7d0 2023-11-24T08:11:46.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.081+0000 7f9865ffb640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f985406b080 0x7f985406d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.081+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98680899d0 con 0x7f987015d7d0 2023-11-24T08:11:46.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.081+0000 7f98757f8640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f985406b080 0x7f985406d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.082+0000 7f98757f8640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f985406b080 0x7f985406d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f986c0011e0 tx=0x7f986c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:46.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.085+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9868054710 con 0x7f987015d7d0 2023-11-24T08:11:46.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.305+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol6902", "target": ["mon-mgr", ""]}) v1 -- 0x7f98700c8660 con 0x7f985406b080 2023-11-24T08:11:46.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.319+0000 7f9865ffb640 1 -- 172.21.15.106:0/1286189773 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f98700c8660 con 0x7f985406b080 2023-11-24T08:11:46.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f985406b080 msgr2=0x7f985406d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f985406b080 0x7f985406d540 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f986c0011e0 tx=0x7f986c009040 comp rx=0 tx=0).stop 2023-11-24T08:11:46.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 msgr2=0x7f98701f39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 0x7f98701f39d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f986800cac0 tx=0x7f986800cf80 comp rx=0 tx=0).stop 2023-11-24T08:11:46.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 shutdown_connections 2023-11-24T08:11:46.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f987015d7d0 0x7f98701f39d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f985406b080 0x7f985406d540 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f987015d3f0 0x7f98700b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 --2- 172.21.15.106:0/1286189773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98700d1f40 0x7f98700c92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.322+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 >> 172.21.15.106:0/1286189773 conn(0x7f987009c5b0 msgr2=0x7f98700c7740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:46.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.323+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 shutdown_connections 2023-11-24T08:11:46.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.323+0000 7f98767fa640 1 -- 172.21.15.106:0/1286189773 wait complete. 2023-11-24T08:11:46.333 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.331+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.333 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.331+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.334 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.331+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.334 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.331+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.334 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.331+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:46.349 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T08:11:46.381 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.378+0000 7f67ac9c4640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.382 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.378+0000 7f67ac9c4640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.382 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.378+0000 7f67ac9c4640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.382 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.378+0000 7f67ac9c4640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.382 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:11:46.378+0000 7f67ac9c4640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:11:46.404 INFO:teuthology.orchestra.run.smithi106.stdout:2 2023-11-24T08:11:46.406 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:11:46.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:11:46.550 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:11:46.669 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:11:46.670 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:46.670 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:46.728 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:46.729 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:46.780 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:46.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.983+0000 7f45e7933640 1 Processor -- start 2023-11-24T08:11:46.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.983+0000 7f45e7933640 1 -- start start 2023-11-24T08:11:46.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e7933640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45e00cf450 0x7f45e00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e7933640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e00cece0 con 0x7f45e00cdfb0 2023-11-24T08:11:46.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e7933640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45e006d040 con 0x7f45e015c7f0 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e7933640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45e00cfe40 con 0x7f45e00cf450 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e4ece640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45e00cf450 0x7f45e00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e4ece640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45e00cf450 0x7f45e00cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33434/0 (socket says 172.21.15.106:33434) 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e4ece640 1 -- 172.21.15.106:0/3320446202 learned_addr learned my addr 172.21.15.106:0/3320446202 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.985+0000 7f45e4ece640 1 -- 172.21.15.106:0/3320446202 >> v1:172.21.15.156:6789/0 conn(0x7f45e015c7f0 legacy=0x7f45e006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.986+0000 7f45e4ece640 1 -- 172.21.15.106:0/3320446202 >> v1:172.21.15.106:6789/0 conn(0x7f45e00cdfb0 legacy=0x7f45e00ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.986+0000 7f45e4ece640 1 -- 172.21.15.106:0/3320446202 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45e006d320 con 0x7f45e00cf450 2023-11-24T08:11:46.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.987+0000 7f45e4ece640 1 --2- 172.21.15.106:0/3320446202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e00cf450 0x7f45e00cf900 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009870 tx=0x7f45d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d55cf42a3dd3bf22 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:46.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.987+0000 7f45d7fff640 1 -- 172.21.15.106:0/3320446202 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0030860 con 0x7f45e00cf450 2023-11-24T08:11:46.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.987+0000 7f45d7fff640 1 -- 172.21.15.106:0/3320446202 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d00309c0 con 0x7f45e00cf450 2023-11-24T08:11:46.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.987+0000 7f45d7fff640 1 -- 172.21.15.106:0/3320446202 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d0030d50 con 0x7f45e00cf450 2023-11-24T08:11:46.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.987+0000 7f45e7933640 1 -- 172.21.15.106:0/3320446202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e00cf450 msgr2=0x7f45e00cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.987+0000 7f45e7933640 1 --2- 172.21.15.106:0/3320446202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e00cf450 0x7f45e00cf900 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f45d0009870 tx=0x7f45d002b9a0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.988+0000 7f45e7933640 1 -- 172.21.15.106:0/3320446202 shutdown_connections 2023-11-24T08:11:46.990 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.988+0000 7f45e7933640 1 --2- 172.21.15.106:0/3320446202 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e00cf450 0x7f45e00cf900 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.988+0000 7f45e7933640 1 -- 172.21.15.106:0/3320446202 >> 172.21.15.106:0/3320446202 conn(0x7f45e006a920 msgr2=0x7f45e006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:46.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.988+0000 7f45e7933640 1 -- 172.21.15.106:0/3320446202 shutdown_connections 2023-11-24T08:11:46.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.988+0000 7f45e7933640 1 -- 172.21.15.106:0/3320446202 wait complete. 2023-11-24T08:11:46.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.989+0000 7f45e7933640 1 Processor -- start 2023-11-24T08:11:46.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 -- start start 2023-11-24T08:11:46.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45e00cdfb0 0x7f45e01e63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e00cf450 0x7f45e0098890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 0x7f45e015b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01e6930 con 0x7f45e015c7f0 2023-11-24T08:11:46.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01e5350 con 0x7f45e00cf450 2023-11-24T08:11:46.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.990+0000 7f45e7933640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45e01e5630 con 0x7f45e00cdfb0 2023-11-24T08:11:46.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 0x7f45e015b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 0x7f45e015b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33442/0 (socket says 172.21.15.106:33442) 2023-11-24T08:11:46.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 -- 172.21.15.106:0/2156604297 learned_addr learned my addr 172.21.15.106:0/2156604297 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 -- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45e00cdfb0 msgr2=0x7f45e01e63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e56cf640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45e00cdfb0 0x7f45e01e63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e4ece640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e00cf450 0x7f45e0098890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45e00cdfb0 0x7f45e01e63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 -- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e00cf450 msgr2=0x7f45e0098890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e00cf450 0x7f45e0098890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.991+0000 7f45e5ed0640 1 -- 172.21.15.106:0/2156604297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d00094d0 con 0x7f45e015c7f0 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.992+0000 7f45e5ed0640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 0x7f45e015b270 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c730 tx=0x7f45d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:46.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.992+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8003b70 con 0x7f45e015c7f0 2023-11-24T08:11:46.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.992+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45d8003cf0 con 0x7f45e015c7f0 2023-11-24T08:11:46.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.992+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e01e5910 con 0x7f45e015c7f0 2023-11-24T08:11:46.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.992+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45d8003ea0 con 0x7f45e015c7f0 2023-11-24T08:11:46.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.992+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e01e5ed0 con 0x7f45e015c7f0 2023-11-24T08:11:46.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.994+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e01e5770 con 0x7f45e015c7f0 2023-11-24T08:11:46.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.994+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f45d8005520 con 0x7f45e015c7f0 2023-11-24T08:11:46.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.995+0000 7f45d5ffb640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f45c406b110 0x7f45c406d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:46.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.996+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f45d8089a50 con 0x7f45e015c7f0 2023-11-24T08:11:46.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.996+0000 7f45e56cf640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f45c406b110 0x7f45c406d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:46.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:46.996+0000 7f45e56cf640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f45c406b110 0x7f45c406d5d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00a750 tx=0x7f45dc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.000+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45e01e5770 con 0x7f45e015c7f0 2023-11-24T08:11:47.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.196+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f45e01e5770 con 0x7f45e015c7f0 2023-11-24T08:11:47.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.199+0000 7f45d5ffb640 1 -- 172.21.15.106:0/2156604297 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16721 (secure 0 0 0) 0x7f45e01e5770 con 0x7f45e015c7f0 2023-11-24T08:11:47.202 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:47.202 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":63,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:46.321969+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.201+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f45c406b110 msgr2=0x7f45c406d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.201+0000 7f45e7933640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f45c406b110 0x7f45c406d5d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f45dc00a750 tx=0x7f45dc008040 comp rx=0 tx=0).stop 2023-11-24T08:11:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.201+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 msgr2=0x7f45e015b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.201+0000 7f45e7933640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 0x7f45e015b270 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f45d800c730 tx=0x7f45d800cbf0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 shutdown_connections 2023-11-24T08:11:47.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45e015c7f0 0x7f45e015b270 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45e00cf450 0x7f45e0098890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f45c406b110 0x7f45c406d5d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 --2- 172.21.15.106:0/2156604297 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f45e00cdfb0 0x7f45e01e63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 >> 172.21.15.106:0/2156604297 conn(0x7f45e006a920 msgr2=0x7f45e006db20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:47.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 shutdown_connections 2023-11-24T08:11:47.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.202+0000 7f45e7933640 1 -- 172.21.15.106:0/2156604297 wait complete. 2023-11-24T08:11:47.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:11:29.872463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:11:31.045762+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 63, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:47.225 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:47.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.431+0000 7f2cff87c640 1 Processor -- start 2023-11-24T08:11:47.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.431+0000 7f2cff87c640 1 -- start start 2023-11-24T08:11:47.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.433+0000 7f2cff87c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2cf80cc0f0 0x7f2cf80c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.433+0000 7f2cff87c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 0x7f2cf806c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.433+0000 7f2cff87c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cf815eea0 con 0x7f2cf815b440 2023-11-24T08:11:47.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.433+0000 7f2cff87c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cf815b820 con 0x7f2cf80cc0f0 2023-11-24T08:11:47.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.433+0000 7f2cff87c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80abf30 con 0x7f2cf80c57e0 2023-11-24T08:11:47.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.434+0000 7f2cfce17640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2cf80cc0f0 0x7f2cf80c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.434+0000 7f2cfde19640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 0x7f2cf806c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.434+0000 7f2cfce17640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2cf80cc0f0 0x7f2cf80c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:56518/0 (socket says 172.21.15.106:56518) 2023-11-24T08:11:47.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.434+0000 7f2cfde19640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 0x7f2cf806c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:41636/0 (socket says 172.21.15.106:41636) 2023-11-24T08:11:47.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.434+0000 7f2cfce17640 1 -- 172.21.15.106:0/2432759209 learned_addr learned my addr 172.21.15.106:0/2432759209 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:47.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.435+0000 7f2cfde19640 1 -- 172.21.15.106:0/2432759209 >> v2:172.21.15.156:3300/0 conn(0x7f2cf80cc0f0 msgr2=0x7f2cf80c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:47.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.435+0000 7f2cfde19640 1 --2- 172.21.15.106:0/2432759209 >> v2:172.21.15.156:3300/0 conn(0x7f2cf80cc0f0 0x7f2cf80c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.435+0000 7f2cfde19640 1 -- 172.21.15.106:0/2432759209 >> v1:172.21.15.156:6789/0 conn(0x7f2cf815b440 legacy=0x7f2cf80cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.435+0000 7f2cfde19640 1 -- 172.21.15.106:0/2432759209 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf815f0f0 con 0x7f2cf80c57e0 2023-11-24T08:11:47.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.436+0000 7f2cfde19640 1 --2- 172.21.15.106:0/2432759209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 0x7f2cf806c5f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009d90 tx=0x7f2cf002be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=14460cedb4dea24 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.436+0000 7f2ceffff640 1 -- 172.21.15.106:0/2432759209 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf002cbc0 con 0x7f2cf80c57e0 2023-11-24T08:11:47.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.436+0000 7f2ceffff640 1 -- 172.21.15.106:0/2432759209 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf002cd20 con 0x7f2cf80c57e0 2023-11-24T08:11:47.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.436+0000 7f2ceffff640 1 -- 172.21.15.106:0/2432759209 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0034660 con 0x7f2cf80c57e0 2023-11-24T08:11:47.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 -- 172.21.15.106:0/2432759209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 msgr2=0x7f2cf806c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 --2- 172.21.15.106:0/2432759209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 0x7f2cf806c5f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cf0009d90 tx=0x7f2cf002be10 comp rx=0 tx=0).stop 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 -- 172.21.15.106:0/2432759209 shutdown_connections 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 --2- 172.21.15.106:0/2432759209 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80c57e0 0x7f2cf806c5f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 --2- 172.21.15.106:0/2432759209 >> v2:172.21.15.156:3300/0 conn(0x7f2cf80cc0f0 0x7f2cf80c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 -- 172.21.15.106:0/2432759209 >> 172.21.15.106:0/2432759209 conn(0x7f2cf80b0c50 msgr2=0x7f2cf80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 -- 172.21.15.106:0/2432759209 shutdown_connections 2023-11-24T08:11:47.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.437+0000 7f2cff87c640 1 -- 172.21.15.106:0/2432759209 wait complete. 2023-11-24T08:11:47.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.439+0000 7f2cff87c640 1 Processor -- start 2023-11-24T08:11:47.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.440+0000 7f2cff87c640 1 -- start start 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cff87c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 0x7f2cf80719e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cff87c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80cc0f0 0x7f2cf80b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cff87c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf815b440 0x7f2cf80b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cff87c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf815c040 con 0x7f2cf80c57e0 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cff87c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf80b4bd0 con 0x7f2cf815b440 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cff87c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cf8150e70 con 0x7f2cf80cc0f0 2023-11-24T08:11:47.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 0x7f2cf80719e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 0x7f2cf80719e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33454/0 (socket says 172.21.15.106:33454) 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 -- 172.21.15.106:0/1299870450 learned_addr learned my addr 172.21.15.106:0/1299870450 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 -- 172.21.15.106:0/1299870450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80cc0f0 msgr2=0x7f2cf80b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80cc0f0 0x7f2cf80b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 -- 172.21.15.106:0/1299870450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf815b440 msgr2=0x7f2cf80b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf815b440 0x7f2cf80b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.441+0000 7f2cfd618640 1 -- 172.21.15.106:0/1299870450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf0009a70 con 0x7f2cf80c57e0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.442+0000 7f2cfd618640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 0x7f2cf80719e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c430 tx=0x7f2cf400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.442+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf40052d0 con 0x7f2cf80c57e0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.442+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf4005430 con 0x7f2cf80c57e0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.442+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4003830 con 0x7f2cf80c57e0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.442+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cf815b9a0 con 0x7f2cf80c57e0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.442+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cf80c8f00 con 0x7f2cf80c57e0 2023-11-24T08:11:47.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.444+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cf815c180 con 0x7f2cf80c57e0 2023-11-24T08:11:47.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.445+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f2cf4003990 con 0x7f2cf80c57e0 2023-11-24T08:11:47.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.446+0000 7f2cedffb640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cdc06b110 0x7f2cdc06d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.446+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2cf4089090 con 0x7f2cf80c57e0 2023-11-24T08:11:47.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.446+0000 7f2cfce17640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cdc06b110 0x7f2cdc06d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.447+0000 7f2cfce17640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cdc06b110 0x7f2cdc06d5d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80094a0 tx=0x7f2ce80093f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.451+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2cf815c180 con 0x7f2cf80c57e0 2023-11-24T08:11:47.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.650+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cf815c180 con 0x7f2cf80c57e0 2023-11-24T08:11:47.653 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.651+0000 7f2cedffb640 1 -- 172.21.15.106:0/1299870450 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2cf815c180 con 0x7f2cf80c57e0 2023-11-24T08:11:47.654 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.653+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cdc06b110 msgr2=0x7f2cdc06d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.653+0000 7f2cff87c640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cdc06b110 0x7f2cdc06d5d0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f2ce80094a0 tx=0x7f2ce80093f0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.653+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 msgr2=0x7f2cf80719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 0x7f2cf80719e0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f2cf400c430 tx=0x7f2cf400c8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 shutdown_connections 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2cf815b440 0x7f2cf80b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2cdc06b110 0x7f2cdc06d5d0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2cf80cc0f0 0x7f2cf80b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 --2- 172.21.15.106:0/1299870450 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2cf80c57e0 0x7f2cf80719e0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 >> 172.21.15.106:0/1299870450 conn(0x7f2cf80b0c50 msgr2=0x7f2cf81598c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:47.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 shutdown_connections 2023-11-24T08:11:47.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.654+0000 7f2cff87c640 1 -- 172.21.15.106:0/1299870450 wait complete. 2023-11-24T08:11:47.675 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:11:47.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.889+0000 7f321589b640 1 Processor -- start 2023-11-24T08:11:47.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.889+0000 7f321589b640 1 -- start start 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.890+0000 7f321589b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100d0a00 0x7f321006ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.890+0000 7f321589b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32100b40d0 0x7f32100cf670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.890+0000 7f321589b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3210070870 con 0x7f32100cb850 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.890+0000 7f321589b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3210070b50 con 0x7f32100b40d0 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.890+0000 7f321589b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3210070e30 con 0x7f32100d0a00 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f32100cb850 0x7f32100cbc30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40722/0 (socket says 172.21.15.106:40722) 2023-11-24T08:11:47.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320effd640 1 -- 172.21.15.106:0/3422204220 learned_addr learned my addr 172.21.15.106:0/3422204220 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:47.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320f7fe640 1 --2- 172.21.15.106:0/3422204220 >> v2:172.21.15.106:3300/0 conn(0x7f32100b40d0 0x7f32100cf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320e7fc640 1 --2- 172.21.15.106:0/3422204220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100d0a00 0x7f321006ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320dffb640 1 -- 172.21.15.106:0/3422204220 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951114480 0 0) 0x7f3210070870 con 0x7f32100cb850 2023-11-24T08:11:47.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320dffb640 1 -- 172.21.15.106:0/3422204220 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31fc003520 con 0x7f32100cb850 2023-11-24T08:11:47.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320f7fe640 1 -- 172.21.15.106:0/3422204220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100d0a00 msgr2=0x7f321006ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:47.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320f7fe640 1 --2- 172.21.15.106:0/3422204220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100d0a00 0x7f321006ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320f7fe640 1 -- 172.21.15.106:0/3422204220 >> v1:172.21.15.106:6789/0 conn(0x7f32100cb850 legacy=0x7f32100cbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.891+0000 7f320f7fe640 1 -- 172.21.15.106:0/3422204220 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32100cee70 con 0x7f32100b40d0 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.892+0000 7f320f7fe640 1 --2- 172.21.15.106:0/3422204220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100b40d0 0x7f32100cf670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f32080097a0 tx=0x7f320802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f632646af418280 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f320dffb640 1 -- 172.21.15.106:0/3422204220 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208031780 con 0x7f32100b40d0 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f320dffb640 1 -- 172.21.15.106:0/3422204220 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32080318e0 con 0x7f32100b40d0 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f320dffb640 1 -- 172.21.15.106:0/3422204220 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208031c70 con 0x7f32100b40d0 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 -- 172.21.15.106:0/3422204220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100b40d0 msgr2=0x7f32100cf670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 --2- 172.21.15.106:0/3422204220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100b40d0 0x7f32100cf670 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f32080097a0 tx=0x7f320802b820 comp rx=0 tx=0).stop 2023-11-24T08:11:47.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 -- 172.21.15.106:0/3422204220 shutdown_connections 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 --2- 172.21.15.106:0/3422204220 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100b40d0 0x7f32100cf670 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 --2- 172.21.15.106:0/3422204220 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100d0a00 0x7f321006ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 -- 172.21.15.106:0/3422204220 >> 172.21.15.106:0/3422204220 conn(0x7f321006e7d0 msgr2=0x7f321006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 -- 172.21.15.106:0/3422204220 shutdown_connections 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.893+0000 7f321589b640 1 -- 172.21.15.106:0/3422204220 wait complete. 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.895+0000 7f321589b640 1 Processor -- start 2023-11-24T08:11:47.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.895+0000 7f321589b640 1 -- start start 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.896+0000 7f321589b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32100b40d0 0x7f321006d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.896+0000 7f321589b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100cb850 0x7f32100c9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.896+0000 7f321589b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 0x7f32101546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.896+0000 7f321589b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100c9fc0 con 0x7f32100d0a00 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.896+0000 7f321589b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ca290 con 0x7f32100b40d0 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.896+0000 7f321589b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32100ca570 con 0x7f32100cb850 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 0x7f32101546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 0x7f32101546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33484/0 (socket says 172.21.15.106:33484) 2023-11-24T08:11:47.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 -- 172.21.15.106:0/2487127139 learned_addr learned my addr 172.21.15.106:0/2487127139 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320effd640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32100b40d0 0x7f321006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320e7fc640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100cb850 0x7f32100c9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 -- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100cb850 msgr2=0x7f32100c9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100cb850 0x7f32100c9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 -- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32100b40d0 msgr2=0x7f321006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32100b40d0 0x7f321006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 -- 172.21.15.106:0/2487127139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3208009400 con 0x7f32100d0a00 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320effd640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32100b40d0 0x7f321006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.897+0000 7f320f7fe640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 0x7f32101546d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3208034040 tx=0x7f32080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.898+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208032820 con 0x7f32100d0a00 2023-11-24T08:11:47.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.898+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3208032980 con 0x7f32100d0a00 2023-11-24T08:11:47.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.898+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3208032d30 con 0x7f32100d0a00 2023-11-24T08:11:47.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.898+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32100ca850 con 0x7f32100d0a00 2023-11-24T08:11:47.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.898+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32100cae10 con 0x7f32100d0a00 2023-11-24T08:11:47.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.899+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32100cb000 con 0x7f32100d0a00 2023-11-24T08:11:47.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.900+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7f320804e450 con 0x7f32100d0a00 2023-11-24T08:11:47.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.901+0000 7f3207fff640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f31f006b110 0x7f31f006d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:47.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.901+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f32080abb30 con 0x7f32100d0a00 2023-11-24T08:11:47.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.901+0000 7f320effd640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f31f006b110 0x7f31f006d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:47.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.902+0000 7f320effd640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f31f006b110 0x7f31f006d5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3200001560 tx=0x7f3200009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:47.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:47.905+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3208076790 con 0x7f32100d0a00 2023-11-24T08:11:48.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.095+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f321004e8b0 con 0x7f32100d0a00 2023-11-24T08:11:48.099 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.096+0000 7f3207fff640 1 -- 172.21.15.106:0/2487127139 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3208079db0 con 0x7f32100d0a00 2023-11-24T08:11:48.099 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:11:48.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.098+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f31f006b110 msgr2=0x7f31f006d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.098+0000 7f321589b640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f31f006b110 0x7f31f006d5d0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3200001560 tx=0x7f3200009040 comp rx=0 tx=0).stop 2023-11-24T08:11:48.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.098+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 msgr2=0x7f32101546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.101 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.098+0000 7f321589b640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 0x7f32101546d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f3208034040 tx=0x7f32080046c0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 shutdown_connections 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32100d0a00 0x7f32101546d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f31f006b110 0x7f31f006d5d0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32100cb850 0x7f32100c9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 --2- 172.21.15.106:0/2487127139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32100b40d0 0x7f321006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 >> 172.21.15.106:0/2487127139 conn(0x7f321006e7d0 msgr2=0x7f32100b2470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 shutdown_connections 2023-11-24T08:11:48.102 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.099+0000 7f321589b640 1 -- 172.21.15.106:0/2487127139 wait complete. 2023-11-24T08:11:48.120 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:11:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.338+0000 7fcd43901640 1 Processor -- start 2023-11-24T08:11:48.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.338+0000 7fcd43901640 1 -- start start 2023-11-24T08:11:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.340+0000 7fcd43901640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.340+0000 7fcd43901640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd3c0cbf20 0x7fcd3c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.340+0000 7fcd43901640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0c57e0 0x7fcd3c0c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.340+0000 7fcd43901640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c15b820 con 0x7fcd3c0cbf20 2023-11-24T08:11:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.340+0000 7fcd43901640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd3c0cc3a0 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.340+0000 7fcd43901640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c15eea0 con 0x7fcd3c15b440 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd3c0cbf20 0x7fcd3c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd3c0cbf20 0x7fcd3c0c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33500/0 (socket says 172.21.15.106:33500) 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 -- 172.21.15.106:0/3268396081 learned_addr learned my addr 172.21.15.106:0/3268396081 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd4169d640 1 --2- 172.21.15.106:0/3268396081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd41e9e640 1 --2- 172.21.15.106:0/3268396081 >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0c57e0 0x7fcd3c0c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 -- 172.21.15.106:0/3268396081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 msgr2=0x7fcd3c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/3268396081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 -- 172.21.15.106:0/3268396081 >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0c57e0 msgr2=0x7fcd3c0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/3268396081 >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0c57e0 0x7fcd3c0c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.341+0000 7fcd40e9c640 1 -- 172.21.15.106:0/3268396081 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd3c0abf30 con 0x7fcd3c0cbf20 2023-11-24T08:11:48.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.342+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/3268396081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c0c52a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0cb910 tx=0x7fcd3002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98795be92d1f75f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:48.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.346+0000 7fcd2ffff640 1 -- 172.21.15.106:0/3268396081 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd300317e0 con 0x7fcd3c0cbf20 2023-11-24T08:11:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.346+0000 7fcd2ffff640 1 -- 172.21.15.106:0/3268396081 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd30031940 con 0x7fcd3c0cbf20 2023-11-24T08:11:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.346+0000 7fcd2ffff640 1 -- 172.21.15.106:0/3268396081 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30031cd0 con 0x7fcd3c0cbf20 2023-11-24T08:11:48.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 -- 172.21.15.106:0/3268396081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0cbf20 msgr2=0x7fcd3c0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 --2- 172.21.15.106:0/3268396081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c0c52a0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c0cb910 tx=0x7fcd3002b880 comp rx=0 tx=0).stop 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 -- 172.21.15.106:0/3268396081 shutdown_connections 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 --2- 172.21.15.106:0/3268396081 >> v2:172.21.15.156:3300/0 conn(0x7fcd3c0c57e0 0x7fcd3c0c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 --2- 172.21.15.106:0/3268396081 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c0c52a0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 --2- 172.21.15.106:0/3268396081 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 -- 172.21.15.106:0/3268396081 >> 172.21.15.106:0/3268396081 conn(0x7fcd3c0b0c50 msgr2=0x7fcd3c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 -- 172.21.15.106:0/3268396081 shutdown_connections 2023-11-24T08:11:48.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.347+0000 7fcd43901640 1 -- 172.21.15.106:0/3268396081 wait complete. 2023-11-24T08:11:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.350+0000 7fcd43901640 1 Processor -- start 2023-11-24T08:11:48.352 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.350+0000 7fcd43901640 1 -- start start 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd43901640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 0x7fcd3c0b2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd43901640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c09b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd43901640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd43901640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c09bdf0 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd43901640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b2d40 con 0x7fcd3c0cbf20 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd43901640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3c0b3020 con 0x7fcd3c15b440 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd4169d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 0x7fcd3c0b2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd4169d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 0x7fcd3c0b2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33502/0 (socket says 172.21.15.106:33502) 2023-11-24T08:11:48.354 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.351+0000 7fcd4169d640 1 -- 172.21.15.106:0/4061360987 learned_addr learned my addr 172.21.15.106:0/4061360987 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:48.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd4169d640 1 -- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 msgr2=0x7fcd3c1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd41e9e640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c09b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd4169d640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd4169d640 1 -- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0cbf20 msgr2=0x7fcd3c09b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd4169d640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c09b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd4169d640 1 -- 172.21.15.106:0/4061360987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd300094d0 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c09b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.352+0000 7fcd4169d640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 0x7fcd3c0b2720 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800d5f0 tx=0x7fcd3800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.354+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd380125f0 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.354+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd38012750 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.354+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3c0b3300 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.354+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd38012b00 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.354+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd3c156800 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.355+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3c0cfe50 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.356+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 9) v1 ==== 89486+0+0 (secure 0 0 0) 0x7fcd38012c60 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.357+0000 7fcd2dffb640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd2006b160 0x7fcd2006d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.357+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcd38089e60 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.357+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd2006b160 0x7fcd2006d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.358+0000 7fcd40e9c640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd2006b160 0x7fcd2006d620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcd30036000 tx=0x7fcd3003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:48.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.362+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd38054c90 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.368+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcd38055060 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.557 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.554+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd3c04e8b0 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.555+0000 7fcd2dffb640 1 -- 172.21.15.106:0/4061360987 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd38087d60 con 0x7fcd3c0c57e0 2023-11-24T08:11:48.558 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd2006b160 msgr2=0x7fcd2006d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd2006b160 0x7fcd2006d620 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fcd30036000 tx=0x7fcd3003d040 comp rx=0 tx=0).stop 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 msgr2=0x7fcd3c0b2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 0x7fcd3c0b2720 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fcd3800d5f0 tx=0x7fcd3800dab0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 shutdown_connections 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd2006b160 0x7fcd2006d620 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd3c15b440 0x7fcd3c1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd3c0cbf20 0x7fcd3c09b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 --2- 172.21.15.106:0/4061360987 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd3c0c57e0 0x7fcd3c0b2720 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 >> 172.21.15.106:0/4061360987 conn(0x7fcd3c0b0c50 msgr2=0x7fcd3c06c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 shutdown_connections 2023-11-24T08:11:48.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.558+0000 7fcd43901640 1 -- 172.21.15.106:0/4061360987 wait complete. 2023-11-24T08:11:48.579 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:11:48.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.782+0000 7f45595c1640 1 Processor -- start 2023-11-24T08:11:48.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.782+0000 7f45595c1640 1 -- start start 2023-11-24T08:11:48.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f45595c1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45540b1250 0x7f4554166160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f45595c1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45540ca0e0 con 0x7f455415b1d0 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f45595c1640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4554072800 con 0x7f4554070550 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f45595c1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455415fde0 con 0x7f45540b1250 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4558dc0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45540b1250 0x7f4554166160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4558dc0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f45540b1250 0x7f4554166160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33514/0 (socket says 172.21.15.106:33514) 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4558dc0640 1 -- 172.21.15.106:0/2738566950 learned_addr learned my addr 172.21.15.106:0/2738566950 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4553fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f455415b1d0 0x7f4554071f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40732/0 (socket says 172.21.15.106:40732) 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4558dc0640 1 -- 172.21.15.106:0/2738566950 >> v1:172.21.15.156:6789/0 conn(0x7f4554070550 legacy=0x7f45540b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4558dc0640 1 -- 172.21.15.106:0/2738566950 >> v1:172.21.15.106:6789/0 conn(0x7f455415b1d0 legacy=0x7f4554071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.784+0000 7f4558dc0640 1 -- 172.21.15.106:0/2738566950 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45540c97b0 con 0x7f45540b1250 2023-11-24T08:11:48.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.785+0000 7f4558dc0640 1 --2- 172.21.15.106:0/2738566950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f4554166160 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f454c009e50 tx=0x7f454c02bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52cbbbf5b01450fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:48.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45537fe640 1 -- 172.21.15.106:0/2738566950 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c02cb20 con 0x7f45540b1250 2023-11-24T08:11:48.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45537fe640 1 -- 172.21.15.106:0/2738566950 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f454c02cc80 con 0x7f45540b1250 2023-11-24T08:11:48.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45537fe640 1 -- 172.21.15.106:0/2738566950 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f454c0343f0 con 0x7f45540b1250 2023-11-24T08:11:48.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 -- 172.21.15.106:0/2738566950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 msgr2=0x7f4554166160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 --2- 172.21.15.106:0/2738566950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f4554166160 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f454c009e50 tx=0x7f454c02bf80 comp rx=0 tx=0).stop 2023-11-24T08:11:48.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 -- 172.21.15.106:0/2738566950 shutdown_connections 2023-11-24T08:11:48.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 --2- 172.21.15.106:0/2738566950 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f4554166160 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 -- 172.21.15.106:0/2738566950 >> 172.21.15.106:0/2738566950 conn(0x7f45540cbb40 msgr2=0x7f45540cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:48.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 -- 172.21.15.106:0/2738566950 shutdown_connections 2023-11-24T08:11:48.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.786+0000 7f45595c1640 1 -- 172.21.15.106:0/2738566950 wait complete. 2023-11-24T08:11:48.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.788+0000 7f45595c1640 1 Processor -- start 2023-11-24T08:11:48.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.788+0000 7f45595c1640 1 -- start start 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f45595c1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554070550 0x7f45540b4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f45595c1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f455415a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f45595c1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f455415b1d0 0x7f45540b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f45595c1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540b3670 con 0x7f45540b1250 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f45595c1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540b3950 con 0x7f4554070550 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f45595c1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45540b3c30 con 0x7f455415b1d0 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f454b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f455415a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f454b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f455415a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33530/0 (socket says 172.21.15.106:33530) 2023-11-24T08:11:48.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.789+0000 7f454b7fe640 1 -- 172.21.15.106:0/3911573698 learned_addr learned my addr 172.21.15.106:0/3911573698 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:48.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f454b7fe640 1 -- 172.21.15.106:0/3911573698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f455415b1d0 msgr2=0x7f45540b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f454b7fe640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f455415b1d0 0x7f45540b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f454b7fe640 1 -- 172.21.15.106:0/3911573698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554070550 msgr2=0x7f45540b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f454b7fe640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554070550 0x7f45540b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f454b7fe640 1 -- 172.21.15.106:0/3911573698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454c009b30 con 0x7f45540b1250 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f454b7fe640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f455415a5a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c410 tx=0x7f453c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c0035f0 con 0x7f45540b1250 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c003750 con 0x7f45540b1250 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c003b40 con 0x7f45540b1250 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.790+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45540b3f10 con 0x7f45540b1250 2023-11-24T08:11:48.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.791+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455415c440 con 0x7f45540b1250 2023-11-24T08:11:48.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.792+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45540b4430 con 0x7f45540b1250 2023-11-24T08:11:48.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.793+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f453c003ca0 con 0x7f45540b1250 2023-11-24T08:11:48.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.793+0000 7f45517fa640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453406b230 0x7f453406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:48.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.794+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f453c088fb0 con 0x7f45540b1250 2023-11-24T08:11:48.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.794+0000 7f4553fff640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453406b230 0x7f453406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:48.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.795+0000 7f4553fff640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453406b230 0x7f453406d6f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4544001540 tx=0x7f45440079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:48.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.798+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f45540b4430 con 0x7f45540b1250 2023-11-24T08:11:49.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.999+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45540b4430 con 0x7f45540b1250 2023-11-24T08:11:49.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:48.999+0000 7f45517fa640 1 -- 172.21.15.106:0/3911573698 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7f45540b4430 con 0x7f45540b1250 2023-11-24T08:11:49.002 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:11:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453406b230 msgr2=0x7f453406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453406b230 0x7f453406d6f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4544001540 tx=0x7f45440079c0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 msgr2=0x7f455415a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f455415a5a0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f453c00c410 tx=0x7f453c00c8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 shutdown_connections 2023-11-24T08:11:49.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f453406b230 0x7f453406d6f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f455415b1d0 0x7f45540b2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f45540b1250 0x7f455415a5a0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 --2- 172.21.15.106:0/3911573698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4554070550 0x7f45540b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 >> 172.21.15.106:0/3911573698 conn(0x7f45540cbb40 msgr2=0x7f45540afda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:49.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 shutdown_connections 2023-11-24T08:11:49.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.002+0000 7f45595c1640 1 -- 172.21.15.106:0/3911573698 wait complete. 2023-11-24T08:11:49.023 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:11:49.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:49.023 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T08:11:49.126 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:11:49.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:49.127 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:11:49.177 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:49.178 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:49.239 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:49.239 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:49.293 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:49.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.519+0000 7f364ae75640 1 Processor -- start 2023-11-24T08:11:49.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.519+0000 7f364ae75640 1 -- start start 2023-11-24T08:11:49.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.520+0000 7f364ae75640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f364415c7f0 0x7f364406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.520+0000 7f364ae75640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f36440ce6c0 0x7f36440cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.520+0000 7f364ae75640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36440cece0 con 0x7f36440cf450 2023-11-24T08:11:49.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f364ae75640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440cfe40 con 0x7f364415c7f0 2023-11-24T08:11:49.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f364ae75640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440ceaf0 con 0x7f36440ce6c0 2023-11-24T08:11:49.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3648c11640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f364415c7f0 0x7f364406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3643fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f36440cf450 0x7f36440cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40746/0 (socket says 172.21.15.106:40746) 2023-11-24T08:11:49.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3643fff640 1 -- 172.21.15.106:0/2201116924 learned_addr learned my addr 172.21.15.106:0/2201116924 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3649412640 1 --2- 172.21.15.106:0/2201116924 >> v2:172.21.15.156:3300/0 conn(0x7f36440ce6c0 0x7f36440cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3648c11640 1 -- 172.21.15.106:0/2201116924 >> v2:172.21.15.156:3300/0 conn(0x7f36440ce6c0 msgr2=0x7f36440cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3648c11640 1 --2- 172.21.15.106:0/2201116924 >> v2:172.21.15.156:3300/0 conn(0x7f36440ce6c0 0x7f36440cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3648c11640 1 -- 172.21.15.106:0/2201116924 >> v1:172.21.15.106:6789/0 conn(0x7f36440cf450 legacy=0x7f36440cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f3648c11640 1 -- 172.21.15.106:0/2201116924 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364406d040 con 0x7f364415c7f0 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.521+0000 7f36437fe640 1 -- 172.21.15.106:0/2201116924 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800715382 0 0) 0x7f36440cece0 con 0x7f36440cf450 2023-11-24T08:11:49.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.522+0000 7f3648c11640 1 --2- 172.21.15.106:0/2201116924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364406db20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f36440d00e0 tx=0x7f363802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=779142b4e521aa36 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:49.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f36437fe640 1 -- 172.21.15.106:0/2201116924 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638031830 con 0x7f364415c7f0 2023-11-24T08:11:49.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f36437fe640 1 -- 172.21.15.106:0/2201116924 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3638031990 con 0x7f364415c7f0 2023-11-24T08:11:49.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f36437fe640 1 -- 172.21.15.106:0/2201116924 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3638031d20 con 0x7f364415c7f0 2023-11-24T08:11:49.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 -- 172.21.15.106:0/2201116924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 msgr2=0x7f364406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 --2- 172.21.15.106:0/2201116924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364406db20 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f36440d00e0 tx=0x7f363802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 -- 172.21.15.106:0/2201116924 shutdown_connections 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 --2- 172.21.15.106:0/2201116924 >> v2:172.21.15.156:3300/0 conn(0x7f36440ce6c0 0x7f36440cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 --2- 172.21.15.106:0/2201116924 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364406db20 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 -- 172.21.15.106:0/2201116924 >> 172.21.15.106:0/2201116924 conn(0x7f364406a920 msgr2=0x7f364406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 -- 172.21.15.106:0/2201116924 shutdown_connections 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.523+0000 7f364ae75640 1 -- 172.21.15.106:0/2201116924 wait complete. 2023-11-24T08:11:49.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.525+0000 7f364ae75640 1 Processor -- start 2023-11-24T08:11:49.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.525+0000 7f364ae75640 1 -- start start 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f364ae75640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36440ce6c0 0x7f3644150aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f364ae75640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36440cf450 0x7f364415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f364ae75640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364415f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f364ae75640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441f7600 con 0x7f364415c7f0 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f364ae75640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36441f6a80 con 0x7f36440ce6c0 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f364ae75640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36441f6d60 con 0x7f36440cf450 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.526+0000 7f3649412640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364415f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33558/0 (socket says 172.21.15.106:33558) 2023-11-24T08:11:49.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 -- 172.21.15.106:0/2703087515 learned_addr learned my addr 172.21.15.106:0/2703087515 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:49.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3648c11640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36440ce6c0 0x7f3644150aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 -- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36440cf450 msgr2=0x7f364415eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36440cf450 0x7f364415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 -- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36440ce6c0 msgr2=0x7f3644150aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36440ce6c0 0x7f3644150aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 -- 172.21.15.106:0/2703087515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009400 con 0x7f364415c7f0 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f3649412640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364415f400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f363c007d90 tx=0x7f363c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.527+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c014420 con 0x7f364415c7f0 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.528+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c0145a0 con 0x7f364415c7f0 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.528+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c014950 con 0x7f364415c7f0 2023-11-24T08:11:49.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.528+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36441f7040 con 0x7f364415c7f0 2023-11-24T08:11:49.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.528+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36441f66a0 con 0x7f364415c7f0 2023-11-24T08:11:49.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.529+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3644067470 con 0x7f364415c7f0 2023-11-24T08:11:49.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.530+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f363c014ab0 con 0x7f364415c7f0 2023-11-24T08:11:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.530+0000 7f36417fa640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f362806b260 0x7f362806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.531+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f363c08a610 con 0x7f364415c7f0 2023-11-24T08:11:49.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.531+0000 7f3648c11640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f362806b260 0x7f362806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.532+0000 7f3648c11640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f362806b260 0x7f362806d720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3638036000 tx=0x7f363803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:49.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.535+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f363c055130 con 0x7f364415c7f0 2023-11-24T08:11:49.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.730+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f36440aca20 con 0x7f364415c7f0 2023-11-24T08:11:49.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.732+0000 7f36417fa640 1 -- 172.21.15.106:0/2703087515 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16721 (secure 0 0 0) 0x7f363c058750 con 0x7f364415c7f0 2023-11-24T08:11:49.735 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:49.735 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":63,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:46.321969+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.734+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f362806b260 msgr2=0x7f362806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.734+0000 7f364ae75640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f362806b260 0x7f362806d720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3638036000 tx=0x7f363803c040 comp rx=0 tx=0).stop 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.734+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 msgr2=0x7f364415f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.734+0000 7f364ae75640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364415f400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f363c007d90 tx=0x7f363c00e4b0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 shutdown_connections 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f364415c7f0 0x7f364415f400 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f362806b260 0x7f362806d720 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f36440cf450 0x7f364415eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 --2- 172.21.15.106:0/2703087515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f36440ce6c0 0x7f3644150aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 >> 172.21.15.106:0/2703087515 conn(0x7f364406a920 msgr2=0x7f364415cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 shutdown_connections 2023-11-24T08:11:49.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.735+0000 7f364ae75640 1 -- 172.21.15.106:0/2703087515 wait complete. 2023-11-24T08:11:49.756 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:11:29.872463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:11:31.045762+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 63, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:49.756 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:49.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.970+0000 7f09067f5640 1 Processor -- start 2023-11-24T08:11:49.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.970+0000 7f09067f5640 1 -- start start 2023-11-24T08:11:49.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.972+0000 7f09067f5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09000cc0f0 0x7f09000c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.972+0000 7f09067f5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000c57e0 0x7f090006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.972+0000 7f09067f5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090015eea0 con 0x7f090015b440 2023-11-24T08:11:49.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.972+0000 7f09067f5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090015b820 con 0x7f09000cc0f0 2023-11-24T08:11:49.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.972+0000 7f09067f5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000abf30 con 0x7f09000c57e0 2023-11-24T08:11:49.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09000cc0f0 0x7f09000c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09000cc0f0 0x7f09000c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:56590/0 (socket says 172.21.15.106:56590) 2023-11-24T08:11:49.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 -- 172.21.15.106:0/2924021240 learned_addr learned my addr 172.21.15.106:0/2924021240 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:49.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f0904d92640 1 --2- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000c57e0 0x7f090006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 -- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000c57e0 msgr2=0x7f090006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:49.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 --2- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000c57e0 0x7f090006c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 -- 172.21.15.106:0/2924021240 >> v1:172.21.15.156:6789/0 conn(0x7f090015b440 legacy=0x7f09000cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:49.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.973+0000 7f08f7fff640 1 -- 172.21.15.106:0/2924021240 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090015f0f0 con 0x7f09000cc0f0 2023-11-24T08:11:49.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.975+0000 7f08f7fff640 1 --2- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000cc0f0 0x7f09000c52a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009870 tx=0x7f08e802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a67000cb6e44d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:49.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.975+0000 7f08ff7fe640 1 -- 172.21.15.106:0/2924021240 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8031640 con 0x7f09000cc0f0 2023-11-24T08:11:49.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.975+0000 7f08ff7fe640 1 -- 172.21.15.106:0/2924021240 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e80317a0 con 0x7f09000cc0f0 2023-11-24T08:11:49.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.975+0000 7f08ff7fe640 1 -- 172.21.15.106:0/2924021240 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e8031b30 con 0x7f09000cc0f0 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 -- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000cc0f0 msgr2=0x7f09000c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 --2- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000cc0f0 0x7f09000c52a0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f08e8009870 tx=0x7f08e802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 -- 172.21.15.106:0/2924021240 shutdown_connections 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 --2- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000c57e0 0x7f090006c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 --2- 172.21.15.106:0/2924021240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000cc0f0 0x7f09000c52a0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 -- 172.21.15.106:0/2924021240 >> 172.21.15.106:0/2924021240 conn(0x7f09000b0c50 msgr2=0x7f09000b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 -- 172.21.15.106:0/2924021240 shutdown_connections 2023-11-24T08:11:49.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.976+0000 7f09067f5640 1 -- 172.21.15.106:0/2924021240 wait complete. 2023-11-24T08:11:49.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.978+0000 7f09067f5640 1 Processor -- start 2023-11-24T08:11:49.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.979+0000 7f09067f5640 1 -- start start 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f09067f5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000c57e0 0x7f09000af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f09067f5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000cc0f0 0x7f0900159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f09067f5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 0x7f0900147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f09067f5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000afc80 con 0x7f090015b440 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f09067f5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090015a3d0 con 0x7f09000c57e0 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f09067f5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000b0a80 con 0x7f09000cc0f0 2023-11-24T08:11:49.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f0904d92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 0x7f0900147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f0904d92640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 0x7f0900147800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33570/0 (socket says 172.21.15.106:33570) 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.980+0000 7f0904d92640 1 -- 172.21.15.106:0/1948127452 learned_addr learned my addr 172.21.15.106:0/1948127452 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f08fffff640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000c57e0 0x7f09000af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f08f7fff640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000cc0f0 0x7f0900159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f0904d92640 1 -- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000cc0f0 msgr2=0x7f0900159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f0904d92640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000cc0f0 0x7f0900159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f0904d92640 1 -- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000c57e0 msgr2=0x7f09000af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f0904d92640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000c57e0 0x7f09000af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:49.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f0904d92640 1 -- 172.21.15.106:0/1948127452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e80094d0 con 0x7f090015b440 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f0904d92640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 0x7f0900147800 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009dd0 tx=0x7f08f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800ced0 con 0x7f090015b440 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f800f040 con 0x7f090015b440 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80146a0 con 0x7f090015b440 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.981+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09000afe70 con 0x7f090015b440 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.982+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09000b0430 con 0x7f090015b440 2023-11-24T08:11:49.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.983+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090004e8b0 con 0x7f090015b440 2023-11-24T08:11:49.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.983+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f08f8014800 con 0x7f090015b440 2023-11-24T08:11:49.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.984+0000 7f08fd7fa640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08e006b230 0x7f08e006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:49.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.985+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f08f808ab00 con 0x7f090015b440 2023-11-24T08:11:49.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.985+0000 7f08fffff640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08e006b230 0x7f08e006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:49.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.985+0000 7f08fffff640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08e006b230 0x7f08e006d6f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f08f0000f60 tx=0x7f08f0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:49.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:49.990+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08f80553b0 con 0x7f090015b440 2023-11-24T08:11:50.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.183+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f090006dc80 con 0x7f090015b440 2023-11-24T08:11:50.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.184+0000 7f08fd7fa640 1 -- 172.21.15.106:0/1948127452 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08f8018020 con 0x7f090015b440 2023-11-24T08:11:50.187 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:50.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08e006b230 msgr2=0x7f08e006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08e006b230 0x7f08e006d6f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f08f0000f60 tx=0x7f08f0009040 comp rx=0 tx=0).stop 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 msgr2=0x7f0900147800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 0x7f0900147800 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009dd0 tx=0x7f08f800c490 comp rx=0 tx=0).stop 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 shutdown_connections 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090015b440 0x7f0900147800 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08e006b230 0x7f08e006d6f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09000cc0f0 0x7f0900159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 --2- 172.21.15.106:0/1948127452 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09000c57e0 0x7f09000af740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 >> 172.21.15.106:0/1948127452 conn(0x7f09000b0c50 msgr2=0x7f09000c4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:50.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 shutdown_connections 2023-11-24T08:11:50.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.187+0000 7f09067f5640 1 -- 172.21.15.106:0/1948127452 wait complete. 2023-11-24T08:11:50.207 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:11:50.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.416+0000 7fc250968640 1 Processor -- start 2023-11-24T08:11:50.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.417+0000 7fc250968640 1 -- start start 2023-11-24T08:11:50.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc250968640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc24c0ccbd0 0x7fc24c1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc250968640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc24c0ccfb0 0x7fc24c162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc250968640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c1655f0 con 0x7fc24c162540 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc250968640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c0cc380 con 0x7fc24c0ccfb0 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc250968640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc24c162b30 con 0x7fc24c0ccbd0 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc243fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc24c0ccfb0 0x7fc24c162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.418+0000 7fc24ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc24c162540 0x7fc24c0c9650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40762/0 (socket says 172.21.15.106:40762) 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc243fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc24c0ccfb0 0x7fc24c162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33586/0 (socket says 172.21.15.106:33586) 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc24ad9d640 1 -- 172.21.15.106:0/414937776 learned_addr learned my addr 172.21.15.106:0/414937776 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc24a59c640 1 --2- 172.21.15.106:0/414937776 >> v2:172.21.15.156:3300/0 conn(0x7fc24c0ccbd0 0x7fc24c1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc243fff640 1 -- 172.21.15.106:0/414937776 >> v2:172.21.15.156:3300/0 conn(0x7fc24c0ccbd0 msgr2=0x7fc24c1660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc243fff640 1 --2- 172.21.15.106:0/414937776 >> v2:172.21.15.156:3300/0 conn(0x7fc24c0ccbd0 0x7fc24c1660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc243fff640 1 -- 172.21.15.106:0/414937776 >> v1:172.21.15.106:6789/0 conn(0x7fc24c162540 legacy=0x7fc24c0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc243fff640 1 -- 172.21.15.106:0/414937776 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c162e10 con 0x7fc24c0ccfb0 2023-11-24T08:11:50.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.419+0000 7fc249d9b640 1 -- 172.21.15.106:0/414937776 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3954404743 0 0) 0x7fc24c1655f0 con 0x7fc24c162540 2023-11-24T08:11:50.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.420+0000 7fc243fff640 1 --2- 172.21.15.106:0/414937776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c0ccfb0 0x7fc24c162000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc24c06ea50 tx=0x7fc23402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6374984d6d843f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:50.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc249d9b640 1 -- 172.21.15.106:0/414937776 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234031830 con 0x7fc24c0ccfb0 2023-11-24T08:11:50.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc249d9b640 1 -- 172.21.15.106:0/414937776 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc234031990 con 0x7fc24c0ccfb0 2023-11-24T08:11:50.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc249d9b640 1 -- 172.21.15.106:0/414937776 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc234031d20 con 0x7fc24c0ccfb0 2023-11-24T08:11:50.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc250968640 1 -- 172.21.15.106:0/414937776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c0ccfb0 msgr2=0x7fc24c162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc250968640 1 --2- 172.21.15.106:0/414937776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c0ccfb0 0x7fc24c162000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc24c06ea50 tx=0x7fc23402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc250968640 1 -- 172.21.15.106:0/414937776 shutdown_connections 2023-11-24T08:11:50.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc250968640 1 --2- 172.21.15.106:0/414937776 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c0ccfb0 0x7fc24c162000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc250968640 1 --2- 172.21.15.106:0/414937776 >> v2:172.21.15.156:3300/0 conn(0x7fc24c0ccbd0 0x7fc24c1660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.421+0000 7fc250968640 1 -- 172.21.15.106:0/414937776 >> 172.21.15.106:0/414937776 conn(0x7fc24c0b0c50 msgr2=0x7fc24c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:50.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.422+0000 7fc250968640 1 -- 172.21.15.106:0/414937776 shutdown_connections 2023-11-24T08:11:50.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.422+0000 7fc250968640 1 -- 172.21.15.106:0/414937776 wait complete. 2023-11-24T08:11:50.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.423+0000 7fc250968640 1 Processor -- start 2023-11-24T08:11:50.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.423+0000 7fc250968640 1 -- start start 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.424+0000 7fc250968640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc24c0ccbd0 0x7fc24c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.424+0000 7fc250968640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc24c0ccfb0 0x7fc24c1f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.424+0000 7fc250968640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 0x7fc24c06c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.424+0000 7fc250968640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c163050 con 0x7fc24c162540 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.424+0000 7fc250968640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c1f8b80 con 0x7fc24c0ccbd0 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.424+0000 7fc250968640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c1632a0 con 0x7fc24c0ccfb0 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 0x7fc24c06c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 0x7fc24c06c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33590/0 (socket says 172.21.15.106:33590) 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 -- 172.21.15.106:0/2047911930 learned_addr learned my addr 172.21.15.106:0/2047911930 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24a59c640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc24c0ccbd0 0x7fc24c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc243fff640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc24c0ccfb0 0x7fc24c1f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 -- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc24c0ccfb0 msgr2=0x7fc24c1f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc24c0ccfb0 0x7fc24c1f8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 -- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc24c0ccbd0 msgr2=0x7fc24c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc24c0ccbd0 0x7fc24c152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 -- 172.21.15.106:0/2047911930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc234009400 con 0x7fc24c162540 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24a59c640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc24c0ccbd0 0x7fc24c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:50.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.425+0000 7fc24ad9d640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 0x7fc24c06c5f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c7e0 tx=0x7fc24400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:50.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.426+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24400a640 con 0x7fc24c162540 2023-11-24T08:11:50.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.426+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24400a7c0 con 0x7fc24c162540 2023-11-24T08:11:50.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.426+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24400ab70 con 0x7fc24c162540 2023-11-24T08:11:50.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.426+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c163580 con 0x7fc24c162540 2023-11-24T08:11:50.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.426+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c163b40 con 0x7fc24c162540 2023-11-24T08:11:50.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.427+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c061a50 con 0x7fc24c162540 2023-11-24T08:11:50.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.428+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc24400acd0 con 0x7fc24c162540 2023-11-24T08:11:50.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.429+0000 7fc242ffd640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc22c06b230 0x7fc22c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.429+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc2440893c0 con 0x7fc24c162540 2023-11-24T08:11:50.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.430+0000 7fc24a59c640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc22c06b230 0x7fc22c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.430+0000 7fc24a59c640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc22c06b230 0x7fc22c06d6f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23c006fd0 tx=0x7fc23c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:50.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.434+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc244053e50 con 0x7fc24c162540 2023-11-24T08:11:50.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.623+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc24c163d30 con 0x7fc24c162540 2023-11-24T08:11:50.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.624+0000 7fc242ffd640 1 -- 172.21.15.106:0/2047911930 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc244018020 con 0x7fc24c162540 2023-11-24T08:11:50.628 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:11:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.626+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc22c06b230 msgr2=0x7fc22c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.626+0000 7fc250968640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc22c06b230 0x7fc22c06d6f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fc23c006fd0 tx=0x7fc23c008040 comp rx=0 tx=0).stop 2023-11-24T08:11:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.626+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 msgr2=0x7fc24c06c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.626+0000 7fc250968640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 0x7fc24c06c5f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c7e0 tx=0x7fc24400cca0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 shutdown_connections 2023-11-24T08:11:50.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc24c162540 0x7fc24c06c5f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc22c06b230 0x7fc22c06d6f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc24c0ccfb0 0x7fc24c1f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 --2- 172.21.15.106:0/2047911930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc24c0ccbd0 0x7fc24c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 >> 172.21.15.106:0/2047911930 conn(0x7fc24c0b0c50 msgr2=0x7fc24c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 shutdown_connections 2023-11-24T08:11:50.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.627+0000 7fc250968640 1 -- 172.21.15.106:0/2047911930 wait complete. 2023-11-24T08:11:50.648 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:11:50.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.863+0000 7ff298b73640 1 Processor -- start 2023-11-24T08:11:50.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.863+0000 7ff298b73640 1 -- start start 2023-11-24T08:11:50.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.864+0000 7ff298b73640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 0x7ff29415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.864+0000 7ff298b73640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff2940c49f0 0x7ff2940705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.864+0000 7ff298b73640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff29415ec90 con 0x7ff294072160 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.864+0000 7ff298b73640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2940cd530 con 0x7ff2940c49f0 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.864+0000 7ff298b73640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940c4ed0 con 0x7ff29406de10 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.865+0000 7ff29259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff294072160 0x7ff29406d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40768/0 (socket says 172.21.15.106:40768) 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.865+0000 7ff29259c640 1 -- 172.21.15.106:0/4024691774 learned_addr learned my addr 172.21.15.106:0/4024691774 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.865+0000 7ff28bfff640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 0x7ff29415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.865+0000 7ff292d9d640 1 --2- 172.21.15.106:0/4024691774 >> v2:172.21.15.156:3300/0 conn(0x7ff2940c49f0 0x7ff2940705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.865+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420559253 0 0) 0x7ff284001110 con 0x7ff294072160 2023-11-24T08:11:50.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.865+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff280003520 con 0x7ff294072160 2023-11-24T08:11:50.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3183945743 0 0) 0x7ff280003520 con 0x7ff294072160 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 msgr2=0x7ff29415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 0x7ff29415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 >> v2:172.21.15.156:3300/0 conn(0x7ff2940c49f0 msgr2=0x7ff2940705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 --2- 172.21.15.106:0/4024691774 >> v2:172.21.15.156:3300/0 conn(0x7ff2940c49f0 0x7ff2940705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2940c51b0 con 0x7ff294072160 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7ff284001800 con 0x7ff294072160 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 >> v1:172.21.15.106:6789/0 conn(0x7ff294072160 legacy=0x7ff29406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280003970 0x7ff280003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800047c0 0x7ff280004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 0x7ff280005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940cd530 con 0x7ff280003970 2023-11-24T08:11:50.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940c4ed0 con 0x7ff2800047c0 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff28000b980 con 0x7ff29406de10 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff29259c640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280003970 0x7ff280003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.866+0000 7ff29259c640 1 -- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 msgr2=0x7ff280005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.867+0000 7ff29259c640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29406de10 0x7ff280005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.867+0000 7ff29259c640 1 -- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800047c0 msgr2=0x7ff280004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.867+0000 7ff29259c640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2800047c0 0x7ff280004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.867+0000 7ff29259c640 1 -- 172.21.15.106:0/4024691774 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280003520 con 0x7ff280003970 2023-11-24T08:11:50.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff29259c640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280003970 0x7ff280003d90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c210 tx=0x7ff2800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f164f071ee3ba0b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:50.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28402cae0 con 0x7ff280003970 2023-11-24T08:11:50.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28402cc60 con 0x7ff280003970 2023-11-24T08:11:50.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff291d9b640 1 -- 172.21.15.106:0/4024691774 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2840058d0 con 0x7ff280003970 2023-11-24T08:11:50.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff298b73640 1 -- 172.21.15.106:0/4024691774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280003970 msgr2=0x7ff280003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff298b73640 1 --2- 172.21.15.106:0/4024691774 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff280003970 0x7ff280003d90 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff28400c210 tx=0x7ff2800057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.868+0000 7ff292d9d640 1 -- 172.21.15.106:0/4024691774 reap_dead start 2023-11-24T08:11:50.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.869+0000 7ff298b73640 1 -- 172.21.15.106:0/4024691774 shutdown_connections 2023-11-24T08:11:50.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.869+0000 7ff298b73640 1 -- 172.21.15.106:0/4024691774 >> 172.21.15.106:0/4024691774 conn(0x7ff2940b0c50 msgr2=0x7ff2940b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:50.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.869+0000 7ff298b73640 1 -- 172.21.15.106:0/4024691774 shutdown_connections 2023-11-24T08:11:50.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.869+0000 7ff298b73640 1 -- 172.21.15.106:0/4024691774 wait complete. 2023-11-24T08:11:50.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.870+0000 7ff298b73640 1 Processor -- start 2023-11-24T08:11:50.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 -- start start 2023-11-24T08:11:50.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff280003970 0x7ff2940999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29409bb20 0x7ff2941560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 0x7ff2940afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29415a080 con 0x7ff2940af680 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff294099f10 con 0x7ff280003970 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.871+0000 7ff298b73640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2940b0040 con 0x7ff29409bb20 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 0x7ff2940afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 0x7ff2940afb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33612/0 (socket says 172.21.15.106:33612) 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 -- 172.21.15.106:0/1885391762 learned_addr learned my addr 172.21.15.106:0/1885391762 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff29259c640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff280003970 0x7ff2940999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff28bfff640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29409bb20 0x7ff2941560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 -- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29409bb20 msgr2=0x7ff2941560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29409bb20 0x7ff2941560d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 -- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff280003970 msgr2=0x7ff2940999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff280003970 0x7ff2940999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:50.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 -- 172.21.15.106:0/1885391762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29409a120 con 0x7ff2940af680 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff28bfff640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29409bb20 0x7ff2941560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.872+0000 7ff292d9d640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 0x7ff2940afb00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff2940b51a0 tx=0x7ff28c00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.873+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c010040 con 0x7ff2940af680 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.873+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff28c00ba80 con 0x7ff2940af680 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.873+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff28c00be30 con 0x7ff2940af680 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.873+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29409a400 con 0x7ff2940af680 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.873+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29409a9c0 con 0x7ff2940af680 2023-11-24T08:11:50.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.874+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29404e8b0 con 0x7ff2940af680 2023-11-24T08:11:50.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.875+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff28c01d400 con 0x7ff2940af680 2023-11-24T08:11:50.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.876+0000 7ff28affd640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff27406b230 0x7ff27406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:50.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.876+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff28c08a3e0 con 0x7ff2940af680 2023-11-24T08:11:50.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.876+0000 7ff29259c640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff27406b230 0x7ff27406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:50.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.877+0000 7ff29259c640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff27406b230 0x7ff27406d6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff284034000 tx=0x7ff284007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:50.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:50.880+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff28c054df0 con 0x7ff2940af680 2023-11-24T08:11:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.064+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff29415e720 con 0x7ff2940af680 2023-11-24T08:11:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.065+0000 7ff28affd640 1 -- 172.21.15.106:0/1885391762 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff28c058410 con 0x7ff2940af680 2023-11-24T08:11:51.068 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.067+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff27406b230 msgr2=0x7ff27406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.067+0000 7ff298b73640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff27406b230 0x7ff27406d6f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff284034000 tx=0x7ff284007000 comp rx=0 tx=0).stop 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 msgr2=0x7ff2940afb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 0x7ff2940afb00 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff2940b51a0 tx=0x7ff28c00b480 comp rx=0 tx=0).stop 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 shutdown_connections 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2940af680 0x7ff2940afb00 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff27406b230 0x7ff27406d6f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff29409bb20 0x7ff2941560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 --2- 172.21.15.106:0/1885391762 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff280003970 0x7ff2940999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 >> 172.21.15.106:0/1885391762 conn(0x7ff2940b0c50 msgr2=0x7ff2940cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 shutdown_connections 2023-11-24T08:11:51.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.068+0000 7ff298b73640 1 -- 172.21.15.106:0/1885391762 wait complete. 2023-11-24T08:11:51.088 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:11:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.299+0000 7f916ffff640 1 Processor -- start 2023-11-24T08:11:51.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.299+0000 7f916ffff640 1 -- start start 2023-11-24T08:11:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.300+0000 7f916ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f917015b1d0 0x7f9170071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:51.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.300+0000 7f916ffff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f91700b1250 0x7f917015edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916ffff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91700ca0e0 con 0x7f9170070550 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916ffff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9170072800 con 0x7f917015b1d0 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916ffff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91700c97b0 con 0x7f91700b1250 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f917015b1d0 0x7f9170071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f917015b1d0 0x7f9170071f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33620/0 (socket says 172.21.15.106:33620) 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916effd640 1 -- 172.21.15.106:0/3752642084 learned_addr learned my addr 172.21.15.106:0/3752642084 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:51.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916f7fe640 1 --2- 172.21.15.106:0/3752642084 >> v2:172.21.15.156:3300/0 conn(0x7f91700b1250 0x7f917015edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:51.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.301+0000 7f916effd640 1 -- 172.21.15.106:0/3752642084 >> v2:172.21.15.156:3300/0 conn(0x7f91700b1250 msgr2=0x7f917015edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:51.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.302+0000 7f916effd640 1 --2- 172.21.15.106:0/3752642084 >> v2:172.21.15.156:3300/0 conn(0x7f91700b1250 0x7f917015edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.302+0000 7f916effd640 1 -- 172.21.15.106:0/3752642084 >> v1:172.21.15.156:6789/0 conn(0x7f9170070550 legacy=0x7f91700b0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:51.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.302+0000 7f916effd640 1 -- 172.21.15.106:0/3752642084 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91700c9a90 con 0x7f917015b1d0 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.303+0000 7f916effd640 1 --2- 172.21.15.106:0/3752642084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f9170071f40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f91700b5170 tx=0x7f916002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3f2cff1e4a2f65 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.303+0000 7f916dffb640 1 -- 172.21.15.106:0/3752642084 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160031870 con 0x7f917015b1d0 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.303+0000 7f916dffb640 1 -- 172.21.15.106:0/3752642084 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91600319d0 con 0x7f917015b1d0 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.303+0000 7f916dffb640 1 -- 172.21.15.106:0/3752642084 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160031d60 con 0x7f917015b1d0 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.303+0000 7f916ffff640 1 -- 172.21.15.106:0/3752642084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 msgr2=0x7f9170071f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.303+0000 7f916ffff640 1 --2- 172.21.15.106:0/3752642084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f9170071f40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f91700b5170 tx=0x7f916002b910 comp rx=0 tx=0).stop 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.304+0000 7f916ffff640 1 -- 172.21.15.106:0/3752642084 shutdown_connections 2023-11-24T08:11:51.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.304+0000 7f916ffff640 1 --2- 172.21.15.106:0/3752642084 >> v2:172.21.15.156:3300/0 conn(0x7f91700b1250 0x7f917015edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.304+0000 7f916ffff640 1 --2- 172.21.15.106:0/3752642084 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f9170071f40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.304+0000 7f916ffff640 1 -- 172.21.15.106:0/3752642084 >> 172.21.15.106:0/3752642084 conn(0x7f91700cbb40 msgr2=0x7f91700cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.304+0000 7f916ffff640 1 -- 172.21.15.106:0/3752642084 shutdown_connections 2023-11-24T08:11:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.304+0000 7f916ffff640 1 -- 172.21.15.106:0/3752642084 wait complete. 2023-11-24T08:11:51.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.306+0000 7f916ffff640 1 Processor -- start 2023-11-24T08:11:51.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.306+0000 7f916ffff640 1 -- start start 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916ffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9170070550 0x7f91700c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91700b1250 0x7f91700b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f91700c8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916ffff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b30b0 con 0x7f917015b1d0 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916ffff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b3390 con 0x7f91700b1250 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916ffff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91700b3670 con 0x7f9170070550 2023-11-24T08:11:51.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.307+0000 7f916f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f91700c8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f91700c8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33628/0 (socket says 172.21.15.106:33628) 2023-11-24T08:11:51.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 -- 172.21.15.106:0/1924568575 learned_addr learned my addr 172.21.15.106:0/1924568575 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:51.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916e7fc640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91700b1250 0x7f91700b4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:51.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916effd640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9170070550 0x7f91700c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:51.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 -- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9170070550 msgr2=0x7f91700c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9170070550 0x7f91700c6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 -- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91700b1250 msgr2=0x7f91700b4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91700b1250 0x7f91700b4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 -- 172.21.15.106:0/1924568575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91600094d0 con 0x7f917015b1d0 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916e7fc640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91700b1250 0x7f91700b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f916f7fe640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f91700c8bb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9168007e40 tx=0x7f916800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.308+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168014620 con 0x7f917015b1d0 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.309+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916800ee90 con 0x7f917015b1d0 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.309+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91680149e0 con 0x7f917015b1d0 2023-11-24T08:11:51.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.309+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91700b3950 con 0x7f917015b1d0 2023-11-24T08:11:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.309+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91700c66d0 con 0x7f917015b1d0 2023-11-24T08:11:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.310+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91700ca550 con 0x7f917015b1d0 2023-11-24T08:11:51.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.311+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f9168014b40 con 0x7f917015b1d0 2023-11-24T08:11:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.312+0000 7f91577fe640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f915c06b230 0x7f915c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.312+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f916808a630 con 0x7f917015b1d0 2023-11-24T08:11:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.312+0000 7f916effd640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f915c06b230 0x7f915c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:51.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.313+0000 7f916effd640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f915c06b230 0x7f915c06d6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9160036000 tx=0x7f916003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:51.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.317+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f91700ca550 con 0x7f917015b1d0 2023-11-24T08:11:51.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.520+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f91700b3e70 con 0x7f917015b1d0 2023-11-24T08:11:51.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.521+0000 7f91577fe640 1 -- 172.21.15.106:0/1924568575 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7f91700b3e70 con 0x7f917015b1d0 2023-11-24T08:11:51.523 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:11:51.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f915c06b230 msgr2=0x7f915c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f915c06b230 0x7f915c06d6f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f9160036000 tx=0x7f916003c040 comp rx=0 tx=0).stop 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 msgr2=0x7f91700c8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f91700c8bb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f9168007e40 tx=0x7f916800e4b0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 shutdown_connections 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f917015b1d0 0x7f91700c8bb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f91700b1250 0x7f91700b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f915c06b230 0x7f915c06d6f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 --2- 172.21.15.106:0/1924568575 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9170070550 0x7f91700c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 >> 172.21.15.106:0/1924568575 conn(0x7f91700cbb40 msgr2=0x7f91700ce480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 shutdown_connections 2023-11-24T08:11:51.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:51.523+0000 7f916ffff640 1 -- 172.21.15.106:0/1924568575 wait complete. 2023-11-24T08:11:51.543 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:11:51.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:51.543 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T08:11:51.659 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:11:51.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:11:51.660 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:11:51.715 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:11:51.715 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:51.774 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:11:51.774 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:11:51.774 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:11:51.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:11:51.833 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:52.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.053+0000 7eff47fff640 1 Processor -- start 2023-11-24T08:11:52.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.053+0000 7eff47fff640 1 -- start start 2023-11-24T08:11:52.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff47fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7eff4815c410 0x7eff4815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff47fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7eff480b1de0 0x7eff480b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff47fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff4815bf80 con 0x7eff48144b30 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff47fff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff480b27b0 con 0x7eff480b1de0 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff47fff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff480b1560 con 0x7eff4815c410 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff46ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7eff48144b30 0x7eff480b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40784/0 (socket says 172.21.15.106:40784) 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff46ffd640 1 -- 172.21.15.106:0/3785995907 learned_addr learned my addr 172.21.15.106:0/3785995907 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:52.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.055+0000 7eff477fe640 1 --2- 172.21.15.106:0/3785995907 >> v2:172.21.15.106:3300/0 conn(0x7eff480b1de0 0x7eff480b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff467fc640 1 --2- 172.21.15.106:0/3785995907 >> v2:172.21.15.156:3300/0 conn(0x7eff4815c410 0x7eff4815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff477fe640 1 -- 172.21.15.106:0/3785995907 >> v2:172.21.15.156:3300/0 conn(0x7eff4815c410 msgr2=0x7eff4815f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff477fe640 1 --2- 172.21.15.106:0/3785995907 >> v2:172.21.15.156:3300/0 conn(0x7eff4815c410 0x7eff4815f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff477fe640 1 -- 172.21.15.106:0/3785995907 >> v1:172.21.15.106:6789/0 conn(0x7eff48144b30 legacy=0x7eff480b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff477fe640 1 -- 172.21.15.106:0/3785995907 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480b1280 con 0x7eff480b1de0 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff45ffb640 1 -- 172.21.15.106:0/3785995907 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2696270686 0 0) 0x7eff4815bf80 con 0x7eff48144b30 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.056+0000 7eff467fc640 1 --2- 172.21.15.106:0/3785995907 >> v2:172.21.15.156:3300/0 conn(0x7eff4815c410 0x7eff4815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:52.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.057+0000 7eff477fe640 1 --2- 172.21.15.106:0/3785995907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff480b2270 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7eff480cc7a0 tx=0x7eff4002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f01ad295219b28c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.057+0000 7eff45ffb640 1 -- 172.21.15.106:0/3785995907 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4002cc80 con 0x7eff480b1de0 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.057+0000 7eff45ffb640 1 -- 172.21.15.106:0/3785995907 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4002cde0 con 0x7eff480b1de0 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.057+0000 7eff45ffb640 1 -- 172.21.15.106:0/3785995907 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40034660 con 0x7eff480b1de0 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 -- 172.21.15.106:0/3785995907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 msgr2=0x7eff480b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 --2- 172.21.15.106:0/3785995907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff480b2270 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7eff480cc7a0 tx=0x7eff4002bed0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 -- 172.21.15.106:0/3785995907 shutdown_connections 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 --2- 172.21.15.106:0/3785995907 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff480b2270 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 --2- 172.21.15.106:0/3785995907 >> v2:172.21.15.156:3300/0 conn(0x7eff4815c410 0x7eff4815f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 -- 172.21.15.106:0/3785995907 >> 172.21.15.106:0/3785995907 conn(0x7eff480cb830 msgr2=0x7eff480cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 -- 172.21.15.106:0/3785995907 shutdown_connections 2023-11-24T08:11:52.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.058+0000 7eff47fff640 1 -- 172.21.15.106:0/3785995907 wait complete. 2023-11-24T08:11:52.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.060+0000 7eff47fff640 1 Processor -- start 2023-11-24T08:11:52.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.060+0000 7eff47fff640 1 -- start start 2023-11-24T08:11:52.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff47fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff4815aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff47fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff48144b30 0x7eff481f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff47fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff4815c410 0x7eff480ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff47fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff481f35e0 con 0x7eff480b1de0 2023-11-24T08:11:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff47fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff4815af90 con 0x7eff48144b30 2023-11-24T08:11:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff47fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff481f3f40 con 0x7eff4815c410 2023-11-24T08:11:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff4815aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff4815aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33638/0 (socket says 172.21.15.106:33638) 2023-11-24T08:11:52.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 -- 172.21.15.106:0/3444950588 learned_addr learned my addr 172.21.15.106:0/3444950588 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff467fc640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff48144b30 0x7eff481f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 -- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff4815c410 msgr2=0x7eff480ce930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff4815c410 0x7eff480ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 -- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff48144b30 msgr2=0x7eff481f3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff48144b30 0x7eff481f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.061+0000 7eff46ffd640 1 -- 172.21.15.106:0/3444950588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff40009a70 con 0x7eff480b1de0 2023-11-24T08:11:52.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.062+0000 7eff46ffd640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff4815aa50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c4d0 tx=0x7eff3800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.062+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff380074e0 con 0x7eff480b1de0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.062+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff38007660 con 0x7eff480b1de0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.062+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff48070f70 con 0x7eff480b1de0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.062+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff38004a20 con 0x7eff480b1de0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.062+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff481f2f20 con 0x7eff480b1de0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.063+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff480b3610 con 0x7eff480b1de0 2023-11-24T08:11:52.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.064+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7eff38004b80 con 0x7eff480b1de0 2023-11-24T08:11:52.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.065+0000 7eff2ffff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff3406b230 0x7eff3406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.065+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7eff380892e0 con 0x7eff480b1de0 2023-11-24T08:11:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.065+0000 7eff467fc640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff3406b230 0x7eff3406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.066+0000 7eff467fc640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff3406b230 0x7eff3406d6f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff480724a0 tx=0x7eff3c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.070+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7eff480b3610 con 0x7eff480b1de0 2023-11-24T08:11:52.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.297+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7eff480b3610 con 0x7eff480b1de0 2023-11-24T08:11:52.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.298+0000 7eff2ffff640 1 -- 172.21.15.106:0/3444950588 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4919 (secure 0 0 0) 0x7eff480b3610 con 0x7eff480b1de0 2023-11-24T08:11:52.301 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:52.301 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:11:32.053291+0000","modified":"2023-11-24T08:11:35.081758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/180703350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":180703350},{"type":"v1","addr":"172.21.15.106:6837","nonce":180703350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:11:52.302 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 34 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff3406b230 msgr2=0x7eff3406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff3406b230 0x7eff3406d6f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7eff480724a0 tx=0x7eff3c008040 comp rx=0 tx=0).stop 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 msgr2=0x7eff4815aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff4815aa50 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7eff3800c4d0 tx=0x7eff3800c990 comp rx=0 tx=0).stop 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 shutdown_connections 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7eff3406b230 0x7eff3406d6f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7eff4815c410 0x7eff480ce930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7eff48144b30 0x7eff481f3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 --2- 172.21.15.106:0/3444950588 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7eff480b1de0 0x7eff4815aa50 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 >> 172.21.15.106:0/3444950588 conn(0x7eff480cb830 msgr2=0x7eff4815ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:52.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 shutdown_connections 2023-11-24T08:11:52.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.301+0000 7eff47fff640 1 -- 172.21.15.106:0/3444950588 wait complete. 2023-11-24T08:11:52.322 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:52.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.528+0000 7f870b59e640 1 Processor -- start 2023-11-24T08:11:52.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.528+0000 7f870b59e640 1 -- start start 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.529+0000 7f870b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c06d5c0 0x7f870c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.529+0000 7f870b59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f870c15a970 0x7f870c163430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.529+0000 7f870b59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f870c0ce870 con 0x7f870c159e80 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870b59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870c0cdc60 con 0x7f870c15a970 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c0cdf40 con 0x7f870c06d5c0 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f870c15a970 0x7f870c163430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f870c15a970 0x7f870c163430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33654/0 (socket says 172.21.15.106:33654) 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 -- 172.21.15.106:0/2539432210 learned_addr learned my addr 172.21.15.106:0/2539432210 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f8703fff640 1 --2- 172.21.15.106:0/2539432210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c06d5c0 0x7f870c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 -- 172.21.15.106:0/2539432210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c06d5c0 msgr2=0x7f870c071610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:52.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 --2- 172.21.15.106:0/2539432210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c06d5c0 0x7f870c071610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 -- 172.21.15.106:0/2539432210 >> v1:172.21.15.156:6789/0 conn(0x7f870c159e80 legacy=0x7f870c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.530+0000 7f870ad9d640 1 -- 172.21.15.106:0/2539432210 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c0ce220 con 0x7f870c15a970 2023-11-24T08:11:52.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.532+0000 7f870ad9d640 1 --2- 172.21.15.106:0/2539432210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c15a970 0x7f870c163430 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f870400d150 tx=0x7f870402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3589b900f3cab4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.532+0000 7f8709d9b640 1 -- 172.21.15.106:0/2539432210 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870402ee90 con 0x7f870c15a970 2023-11-24T08:11:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.532+0000 7f8709d9b640 1 -- 172.21.15.106:0/2539432210 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8704032070 con 0x7f870c15a970 2023-11-24T08:11:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.532+0000 7f8709d9b640 1 -- 172.21.15.106:0/2539432210 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704038620 con 0x7f870c15a970 2023-11-24T08:11:52.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 -- 172.21.15.106:0/2539432210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c15a970 msgr2=0x7f870c163430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 --2- 172.21.15.106:0/2539432210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c15a970 0x7f870c163430 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f870400d150 tx=0x7f870402e0e0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 -- 172.21.15.106:0/2539432210 shutdown_connections 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 --2- 172.21.15.106:0/2539432210 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c15a970 0x7f870c163430 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 --2- 172.21.15.106:0/2539432210 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c06d5c0 0x7f870c071610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 -- 172.21.15.106:0/2539432210 >> 172.21.15.106:0/2539432210 conn(0x7f870c0cb830 msgr2=0x7f870c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 -- 172.21.15.106:0/2539432210 shutdown_connections 2023-11-24T08:11:52.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.533+0000 7f870b59e640 1 -- 172.21.15.106:0/2539432210 wait complete. 2023-11-24T08:11:52.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.535+0000 7f870b59e640 1 Processor -- start 2023-11-24T08:11:52.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.535+0000 7f870b59e640 1 -- start start 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f870b59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f870c06d5c0 0x7f870c144530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f870b59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 0x7f870c0a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f870b59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c15a970 0x7f870c1ee200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f870b59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c154ee0 con 0x7f870c159e80 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f870b59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870c1551c0 con 0x7f870c06d5c0 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f870b59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f870c1554a0 con 0x7f870c15a970 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f8703fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 0x7f870c0a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f8703fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 0x7f870c0a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33660/0 (socket says 172.21.15.106:33660) 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.536+0000 7f8703fff640 1 -- 172.21.15.106:0/4214821732 learned_addr learned my addr 172.21.15.106:0/4214821732 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:52.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8703fff640 1 -- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c15a970 msgr2=0x7f870c1ee200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8703fff640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c15a970 0x7f870c1ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f870a59c640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f870c06d5c0 0x7f870c144530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8703fff640 1 -- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f870c06d5c0 msgr2=0x7f870c144530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8703fff640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f870c06d5c0 0x7f870c144530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8703fff640 1 -- 172.21.15.106:0/4214821732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870400bd40 con 0x7f870c159e80 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f870a59c640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f870c06d5c0 0x7f870c144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8703fff640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 0x7f870c0a8430 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f86f400d5c0 tx=0x7f86f400da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f40125f0 con 0x7f870c159e80 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86f4012750 con 0x7f870c159e80 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f4012b00 con 0x7f870c159e80 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f870c155780 con 0x7f870c159e80 2023-11-24T08:11:52.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.537+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f870c144a70 con 0x7f870c159e80 2023-11-24T08:11:52.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.538+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870c071400 con 0x7f870c159e80 2023-11-24T08:11:52.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.539+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f86f4012c60 con 0x7f870c159e80 2023-11-24T08:11:52.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.540+0000 7f8702ffd640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f86ec06b230 0x7f86ec06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.540+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f86f408a290 con 0x7f870c159e80 2023-11-24T08:11:52.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.540+0000 7f870a59c640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f86ec06b230 0x7f86ec06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.541+0000 7f870a59c640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f86ec06b230 0x7f86ec06d6f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f86fc000f60 tx=0x7f86fc009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.547 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.544+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f86f4054ce0 con 0x7f870c159e80 2023-11-24T08:11:52.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.736+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f870c04e840 con 0x7f870c159e80 2023-11-24T08:11:52.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.738+0000 7f8702ffd640 1 -- 172.21.15.106:0/4214821732 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16721 (secure 0 0 0) 0x7f86f4058300 con 0x7f870c159e80 2023-11-24T08:11:52.741 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:52.741 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":63,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:46.321969+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f86ec06b230 msgr2=0x7f86ec06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f86ec06b230 0x7f86ec06d6f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f86fc000f60 tx=0x7f86fc009040 comp rx=0 tx=0).stop 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 msgr2=0x7f870c0a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 0x7f870c0a8430 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f86f400d5c0 tx=0x7f86f400da80 comp rx=0 tx=0).stop 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 shutdown_connections 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f86ec06b230 0x7f86ec06d6f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f870c15a970 0x7f870c1ee200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f870c159e80 0x7f870c0a8430 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 --2- 172.21.15.106:0/4214821732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f870c06d5c0 0x7f870c144530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.741+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 >> 172.21.15.106:0/4214821732 conn(0x7f870c0cb830 msgr2=0x7f870c15a260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:52.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.742+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 shutdown_connections 2023-11-24T08:11:52.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.742+0000 7f870b59e640 1 -- 172.21.15.106:0/4214821732 wait complete. 2023-11-24T08:11:52.762 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:11:29.872463+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:11:31.045762+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 63, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:11:52.763 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:11:52.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.974+0000 7f1d266d1640 1 Processor -- start 2023-11-24T08:11:52.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.974+0000 7f1d266d1640 1 -- start start 2023-11-24T08:11:52.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d266d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1d2015a1c0 0x7f1d2006d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d266d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d200b0b40 0x7f1d200b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d266d1640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d2006d430 con 0x7f1d2015ded0 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d266d1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d2015ea70 con 0x7f1d2015a1c0 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d266d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d2015ec50 con 0x7f1d200b0b40 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d256cf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1d2015a1c0 0x7f1d2006d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d25ed0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1d2015ded0 0x7f1d2015e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:40800/0 (socket says 172.21.15.106:40800) 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d256cf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1d2015a1c0 0x7f1d2006d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33662/0 (socket says 172.21.15.106:33662) 2023-11-24T08:11:52.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.976+0000 7f1d25ed0640 1 -- 172.21.15.106:0/2366393712 learned_addr learned my addr 172.21.15.106:0/2366393712 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:52.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.977+0000 7f1d24ece640 1 --2- 172.21.15.106:0/2366393712 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d200b0b40 0x7f1d200b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.977+0000 7f1d256cf640 1 -- 172.21.15.106:0/2366393712 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d200b0b40 msgr2=0x7f1d200b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:52.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.977+0000 7f1d256cf640 1 --2- 172.21.15.106:0/2366393712 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d200b0b40 0x7f1d200b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.977+0000 7f1d256cf640 1 -- 172.21.15.106:0/2366393712 >> v1:172.21.15.106:6789/0 conn(0x7f1d2015ded0 legacy=0x7f1d2015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.977+0000 7f1d256cf640 1 -- 172.21.15.106:0/2366393712 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d200b14e0 con 0x7f1d2015a1c0 2023-11-24T08:11:52.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.977+0000 7f1d17fff640 1 -- 172.21.15.106:0/2366393712 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812931476 0 0) 0x7f1d2006d430 con 0x7f1d2015ded0 2023-11-24T08:11:52.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.978+0000 7f1d256cf640 1 --2- 172.21.15.106:0/2366393712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015a1c0 0x7f1d2006d8c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00ba40 tx=0x7f1d1c02e080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d497c8081f3f54de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.978+0000 7f1d17fff640 1 -- 172.21.15.106:0/2366393712 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c02ee30 con 0x7f1d2015a1c0 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.978+0000 7f1d17fff640 1 -- 172.21.15.106:0/2366393712 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1c036440 con 0x7f1d2015a1c0 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.978+0000 7f1d17fff640 1 -- 172.21.15.106:0/2366393712 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1c0367d0 con 0x7f1d2015a1c0 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 -- 172.21.15.106:0/2366393712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015a1c0 msgr2=0x7f1d2006d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 --2- 172.21.15.106:0/2366393712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015a1c0 0x7f1d2006d8c0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c00ba40 tx=0x7f1d1c02e080 comp rx=0 tx=0).stop 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 -- 172.21.15.106:0/2366393712 shutdown_connections 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 --2- 172.21.15.106:0/2366393712 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d200b0b40 0x7f1d200b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 --2- 172.21.15.106:0/2366393712 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015a1c0 0x7f1d2006d8c0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 -- 172.21.15.106:0/2366393712 >> 172.21.15.106:0/2366393712 conn(0x7f1d200cbb40 msgr2=0x7f1d200cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 -- 172.21.15.106:0/2366393712 shutdown_connections 2023-11-24T08:11:52.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.979+0000 7f1d266d1640 1 -- 172.21.15.106:0/2366393712 wait complete. 2023-11-24T08:11:52.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.981+0000 7f1d266d1640 1 Processor -- start 2023-11-24T08:11:52.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.981+0000 7f1d266d1640 1 -- start start 2023-11-24T08:11:52.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d266d1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d200b0b40 0x7f1d200c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d266d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d2015a1c0 0x7f1d200c66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d266d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 0x7f1d201f2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d266d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200c7ad0 con 0x7f1d2015ded0 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d266d1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200c6c10 con 0x7f1d200b0b40 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d266d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d200c6ec0 con 0x7f1d2015a1c0 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d25ed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 0x7f1d201f2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d25ed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 0x7f1d201f2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33666/0 (socket says 172.21.15.106:33666) 2023-11-24T08:11:52.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d25ed0640 1 -- 172.21.15.106:0/3570038534 learned_addr learned my addr 172.21.15.106:0/3570038534 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:52.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d256cf640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d200b0b40 0x7f1d200c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d24ece640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d2015a1c0 0x7f1d200c66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.982+0000 7f1d25ed0640 1 -- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d2015a1c0 msgr2=0x7f1d200c66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d25ed0640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d2015a1c0 0x7f1d200c66d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d25ed0640 1 -- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d200b0b40 msgr2=0x7f1d200c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d25ed0640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d200b0b40 0x7f1d200c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d25ed0640 1 -- 172.21.15.106:0/3570038534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d1c00b6a0 con 0x7f1d2015ded0 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d256cf640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d200b0b40 0x7f1d200c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d25ed0640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 0x7f1d201f2cc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800c700 tx=0x7f1d1800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800ce10 con 0x7f1d2015ded0 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d1800a640 con 0x7f1d2015ded0 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d200c71a0 con 0x7f1d2015ded0 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d1800aa70 con 0x7f1d2015ded0 2023-11-24T08:11:52.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.983+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d200c7760 con 0x7f1d2015ded0 2023-11-24T08:11:52.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.984+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d200c7c40 con 0x7f1d2015ded0 2023-11-24T08:11:52.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.985+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1d1800abd0 con 0x7f1d2015ded0 2023-11-24T08:11:52.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.986+0000 7f1d15ffb640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1d0406b260 0x7f1d0406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:52.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.986+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1d18089520 con 0x7f1d2015ded0 2023-11-24T08:11:52.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.986+0000 7f1d256cf640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1d0406b260 0x7f1d0406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:52.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.987+0000 7f1d256cf640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1d0406b260 0x7f1d0406d720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c038000 tx=0x7f1d1c032000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:52.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:52.990+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1d18054000 con 0x7f1d2015ded0 2023-11-24T08:11:53.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.181+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1d2004e840 con 0x7f1d2015ded0 2023-11-24T08:11:53.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.182+0000 7f1d15ffb640 1 -- 172.21.15.106:0/3570038534 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1d18057620 con 0x7f1d2015ded0 2023-11-24T08:11:53.185 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:11:53.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1d0406b260 msgr2=0x7f1d0406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1d0406b260 0x7f1d0406d720 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1d1c038000 tx=0x7f1d1c032000 comp rx=0 tx=0).stop 2023-11-24T08:11:53.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 msgr2=0x7f1d201f2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 0x7f1d201f2cc0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f1d1800c700 tx=0x7f1d1800cbc0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 shutdown_connections 2023-11-24T08:11:53.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1d2015ded0 0x7f1d201f2cc0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1d0406b260 0x7f1d0406d720 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1d2015a1c0 0x7f1d200c66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 --2- 172.21.15.106:0/3570038534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d200b0b40 0x7f1d200c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 >> 172.21.15.106:0/3570038534 conn(0x7f1d200cbb40 msgr2=0x7f1d20071680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 shutdown_connections 2023-11-24T08:11:53.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.185+0000 7f1d266d1640 1 -- 172.21.15.106:0/3570038534 wait complete. 2023-11-24T08:11:53.207 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:11:53.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.425+0000 7fb827c32640 1 Processor -- start 2023-11-24T08:11:53.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.426+0000 7fb827c32640 1 -- start start 2023-11-24T08:11:53.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.427+0000 7fb827c32640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb82015d380 0x7fb8200cb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.427+0000 7fb827c32640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82006d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.427+0000 7fb827c32640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb82006d670 con 0x7fb82006f360 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.427+0000 7fb827c32640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb82006b980 con 0x7fb82015d380 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.427+0000 7fb827c32640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82006bb90 con 0x7fb8200ce2c0 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.427+0000 7fb8259ce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb82015d380 0x7fb8200cb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8259ce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb82015d380 0x7fb8200cb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33672/0 (socket says 172.21.15.106:33672) 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8259ce640 1 -- 172.21.15.106:0/2340507591 learned_addr learned my addr 172.21.15.106:0/2340507591 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8261cf640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82006d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8259ce640 1 -- 172.21.15.106:0/2340507591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 msgr2=0x7fb82006d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8259ce640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82006d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8259ce640 1 -- 172.21.15.106:0/2340507591 >> v1:172.21.15.156:6789/0 conn(0x7fb82006f360 legacy=0x7fb8200cdbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8259ce640 1 -- 172.21.15.106:0/2340507591 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82006be70 con 0x7fb82015d380 2023-11-24T08:11:53.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.428+0000 7fb8261cf640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:53.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.429+0000 7fb8259ce640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb8200cb850 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb810009870 tx=0x7fb81002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aeaa960dc84037d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:53.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb8249cc640 1 -- 172.21.15.106:0/2340507591 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810031640 con 0x7fb82015d380 2023-11-24T08:11:53.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb8249cc640 1 -- 172.21.15.106:0/2340507591 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8100317a0 con 0x7fb82015d380 2023-11-24T08:11:53.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb8249cc640 1 -- 172.21.15.106:0/2340507591 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb810031b30 con 0x7fb82015d380 2023-11-24T08:11:53.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 -- 172.21.15.106:0/2340507591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 msgr2=0x7fb8200cb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb8200cb850 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fb810009870 tx=0x7fb81002b8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 -- 172.21.15.106:0/2340507591 shutdown_connections 2023-11-24T08:11:53.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82006d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 --2- 172.21.15.106:0/2340507591 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb8200cb850 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 -- 172.21.15.106:0/2340507591 >> 172.21.15.106:0/2340507591 conn(0x7fb82006e7d0 msgr2=0x7fb82006ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:53.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 -- 172.21.15.106:0/2340507591 shutdown_connections 2023-11-24T08:11:53.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.430+0000 7fb827c32640 1 -- 172.21.15.106:0/2340507591 wait complete. 2023-11-24T08:11:53.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.432+0000 7fb827c32640 1 Processor -- start 2023-11-24T08:11:53.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.432+0000 7fb827c32640 1 -- start start 2023-11-24T08:11:53.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb827c32640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb82006f360 0x7fb8201485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb827c32640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb827c32640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb82015a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb827c32640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb820159d70 con 0x7fb82015d380 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb827c32640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82015ac10 con 0x7fb82006f360 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb827c32640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8201601e0 con 0x7fb8200ce2c0 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb8261cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb82015a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb8261cf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb82015a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33680/0 (socket says 172.21.15.106:33680) 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.433+0000 7fb8261cf640 1 -- 172.21.15.106:0/4002574771 learned_addr learned my addr 172.21.15.106:0/4002574771 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8251cd640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8259ce640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb82006f360 0x7fb8201485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8261cf640 1 -- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 msgr2=0x7fb82015a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8261cf640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82015a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8261cf640 1 -- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb82006f360 msgr2=0x7fb8201485c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8261cf640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb82006f360 0x7fb8201485c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8261cf640 1 -- 172.21.15.106:0/4002574771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100094d0 con 0x7fb82015d380 2023-11-24T08:11:53.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8251cd640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:53.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8259ce640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb82006f360 0x7fb8201485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:53.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8261cf640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb82015a6d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c7f0 tx=0x7fb81800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.434+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818007820 con 0x7fb82015d380 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.435+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb818007980 con 0x7fb82015d380 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.435+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb818007d70 con 0x7fb82015d380 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.435+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8201604c0 con 0x7fb82015d380 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.435+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb820159960 con 0x7fb82015d380 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.436+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8200cec20 con 0x7fb82015d380 2023-11-24T08:11:53.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.437+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb8180113f0 con 0x7fb82015d380 2023-11-24T08:11:53.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.438+0000 7fb8167fc640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb80c06b230 0x7fb80c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.438+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb81808a190 con 0x7fb82015d380 2023-11-24T08:11:53.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.438+0000 7fb8259ce640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb80c06b230 0x7fb80c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.439+0000 7fb8259ce640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb80c06b230 0x7fb80c06d6f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb810036000 tx=0x7fb81003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:53.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.442+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb818054b80 con 0x7fb82015d380 2023-11-24T08:11:53.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.639+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb820159eb0 con 0x7fb82015d380 2023-11-24T08:11:53.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.640+0000 7fb8167fc640 1 -- 172.21.15.106:0/4002574771 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb820159eb0 con 0x7fb82015d380 2023-11-24T08:11:53.643 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb80c06b230 msgr2=0x7fb80c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb80c06b230 0x7fb80c06d6f0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb810036000 tx=0x7fb81003c040 comp rx=0 tx=0).stop 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 msgr2=0x7fb82015a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb82015a6d0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c7f0 tx=0x7fb81800ccb0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 shutdown_connections 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb82015d380 0x7fb82015a6d0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb80c06b230 0x7fb80c06d6f0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8200ce2c0 0x7fb82015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 --2- 172.21.15.106:0/4002574771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb82006f360 0x7fb8201485c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 >> 172.21.15.106:0/4002574771 conn(0x7fb82006e7d0 msgr2=0x7fb82006cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:53.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 shutdown_connections 2023-11-24T08:11:53.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.643+0000 7fb827c32640 1 -- 172.21.15.106:0/4002574771 wait complete. 2023-11-24T08:11:53.665 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:11:53.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.878+0000 7fe449fc9640 1 Processor -- start 2023-11-24T08:11:53.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.878+0000 7fe449fc9640 1 -- start start 2023-11-24T08:11:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.879+0000 7fe449fc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe444072160 0x7fe44415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.879+0000 7fe449fc9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4440c49f0 0x7fe44406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.879+0000 7fe449fc9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe44415ec90 con 0x7fe44406dc40 2023-11-24T08:11:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.879+0000 7fe449fc9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44406e0f0 con 0x7fe4440c49f0 2023-11-24T08:11:53.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.879+0000 7fe449fc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440cd530 con 0x7fe444072160 2023-11-24T08:11:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.880+0000 7fe442ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4440c49f0 0x7fe44406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.880+0000 7fe442ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4440c49f0 0x7fe44406d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33692/0 (socket says 172.21.15.106:33692) 2023-11-24T08:11:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.880+0000 7fe442ffd640 1 -- 172.21.15.106:0/4059277415 learned_addr learned my addr 172.21.15.106:0/4059277415 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.880+0000 7fe4437fe640 1 --2- 172.21.15.106:0/4059277415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe444072160 0x7fe44415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.880+0000 7fe442ffd640 1 -- 172.21.15.106:0/4059277415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe444072160 msgr2=0x7fe44415e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:53.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.880+0000 7fe442ffd640 1 --2- 172.21.15.106:0/4059277415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe444072160 0x7fe44415e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.881+0000 7fe442ffd640 1 -- 172.21.15.106:0/4059277415 >> v1:172.21.15.156:6789/0 conn(0x7fe44406dc40 legacy=0x7fe4440cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.881+0000 7fe442ffd640 1 -- 172.21.15.106:0/4059277415 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4440c4ed0 con 0x7fe4440c49f0 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.882+0000 7fe442ffd640 1 --2- 172.21.15.106:0/4059277415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4440c49f0 0x7fe44406d700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009870 tx=0x7fe42c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c975484f65023 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.882+0000 7fe4427fc640 1 -- 172.21.15.106:0/4059277415 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c030840 con 0x7fe4440c49f0 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.882+0000 7fe4427fc640 1 -- 172.21.15.106:0/4059277415 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c0309a0 con 0x7fe4440c49f0 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.882+0000 7fe4427fc640 1 -- 172.21.15.106:0/4059277415 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c030d30 con 0x7fe4440c49f0 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 -- 172.21.15.106:0/4059277415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4440c49f0 msgr2=0x7fe44406d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 --2- 172.21.15.106:0/4059277415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4440c49f0 0x7fe44406d700 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fe42c009870 tx=0x7fe42c02b980 comp rx=0 tx=0).stop 2023-11-24T08:11:53.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 -- 172.21.15.106:0/4059277415 shutdown_connections 2023-11-24T08:11:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 --2- 172.21.15.106:0/4059277415 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4440c49f0 0x7fe44406d700 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 --2- 172.21.15.106:0/4059277415 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe444072160 0x7fe44415e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 -- 172.21.15.106:0/4059277415 >> 172.21.15.106:0/4059277415 conn(0x7fe4440b0c50 msgr2=0x7fe4440b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 -- 172.21.15.106:0/4059277415 shutdown_connections 2023-11-24T08:11:53.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.883+0000 7fe449fc9640 1 -- 172.21.15.106:0/4059277415 wait complete. 2023-11-24T08:11:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.884+0000 7fe449fc9640 1 Processor -- start 2023-11-24T08:11:53.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.885+0000 7fe449fc9640 1 -- start start 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe449fc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe44406dc40 0x7fe444150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe449fc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 0x7fe4440b3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe449fc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4440c49f0 0x7fe44415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe449fc9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4441f34a0 con 0x7fe444072160 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe449fc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b4860 con 0x7fe44406dc40 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe449fc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4440b4b40 con 0x7fe4440c49f0 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 0x7fe4440b3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 0x7fe4440b3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:33706/0 (socket says 172.21.15.106:33706) 2023-11-24T08:11:53.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 -- 172.21.15.106:0/3759941832 learned_addr learned my addr 172.21.15.106:0/3759941832 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe4437fe640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe44406dc40 0x7fe444150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe443fff640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4440c49f0 0x7fe44415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 -- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4440c49f0 msgr2=0x7fe44415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4440c49f0 0x7fe44415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 -- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe44406dc40 msgr2=0x7fe444150210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.886+0000 7fe442ffd640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe44406dc40 0x7fe444150210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe442ffd640 1 -- 172.21.15.106:0/3759941832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe42c0094d0 con 0x7fe444072160 2023-11-24T08:11:53.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe443fff640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4440c49f0 0x7fe44415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe442ffd640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 0x7fe4440b3da0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000c00 tx=0x7fe42c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c039a90 con 0x7fe444072160 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe42c039bf0 con 0x7fe444072160 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe42c043410 con 0x7fe444072160 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4440b4e20 con 0x7fe444072160 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.887+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44409ad00 con 0x7fe444072160 2023-11-24T08:11:53.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.888+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44404e8b0 con 0x7fe444072160 2023-11-24T08:11:53.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.889+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe42c02f390 con 0x7fe444072160 2023-11-24T08:11:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.891+0000 7fe43bfff640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe42406b230 0x7fe42406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.891+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe42c0ac000 con 0x7fe444072160 2023-11-24T08:11:53.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.891+0000 7fe4437fe640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe42406b230 0x7fe42406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:53.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.892+0000 7fe4437fe640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe42406b230 0x7fe42406d6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe434009a60 tx=0x7fe434003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:53.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:53.895+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe42c076a30 con 0x7fe444072160 2023-11-24T08:11:54.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.084+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe44415dab0 con 0x7fe444072160 2023-11-24T08:11:54.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.085+0000 7fe43bfff640 1 -- 172.21.15.106:0/3759941832 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v64) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe42c048020 con 0x7fe444072160 2023-11-24T08:11:54.087 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:11:54.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe42406b230 msgr2=0x7fe42406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe42406b230 0x7fe42406d6f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe434009a60 tx=0x7fe434003990 comp rx=0 tx=0).stop 2023-11-24T08:11:54.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 msgr2=0x7fe4440b3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 0x7fe4440b3da0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe42c000c00 tx=0x7fe42c004730 comp rx=0 tx=0).stop 2023-11-24T08:11:54.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 shutdown_connections 2023-11-24T08:11:54.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe42406b230 0x7fe42406d6f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4440c49f0 0x7fe44415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe444072160 0x7fe4440b3da0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 --2- 172.21.15.106:0/3759941832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe44406dc40 0x7fe444150210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 >> 172.21.15.106:0/3759941832 conn(0x7fe4440b0c50 msgr2=0x7fe4440712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:54.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 shutdown_connections 2023-11-24T08:11:54.092 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.087+0000 7fe449fc9640 1 -- 172.21.15.106:0/3759941832 wait complete. 2023-11-24T08:11:54.109 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:54.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.318+0000 7fe5fef8a640 1 Processor -- start 2023-11-24T08:11:54.321 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.319+0000 7fe5fef8a640 1 -- start start 2023-11-24T08:11:54.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.320+0000 7fe5fef8a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 0x7fe5f815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.320+0000 7fe5fef8a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f815ab80 con 0x7fe5f80b2a70 2023-11-24T08:11:54.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.320+0000 7fe5fef8a640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f815e680 con 0x7fe5f80b2630 2023-11-24T08:11:54.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.320+0000 7fe5fef8a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f815a990 con 0x7fe5f80b2250 2023-11-24T08:11:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.320+0000 7fe5fd527640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe5f80b2a70 0x7fe5f815b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48984/0 (socket says 172.21.15.106:48984) 2023-11-24T08:11:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.320+0000 7fe5fd527640 1 -- 172.21.15.106:0/1737989303 learned_addr learned my addr 172.21.15.106:0/1737989303 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5fcd26640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 0x7fe5f815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028553227 0 0) 0x7fe5f815ab80 con 0x7fe5f80b2a70 2023-11-24T08:11:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5e8003520 con 0x7fe5f80b2a70 2023-11-24T08:11:54.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2260410200 0 0) 0x7fe5e8003520 con 0x7fe5f80b2a70 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 msgr2=0x7fe5f815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 0x7fe5f815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 >> v1:172.21.15.156:6789/0 conn(0x7fe5f80b2630 legacy=0x7fe5f80ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f815e960 con 0x7fe5f80b2a70 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fe5f00019f0 con 0x7fe5f80b2a70 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 >> v1:172.21.15.106:6789/0 conn(0x7fe5f80b2a70 legacy=0x7fe5f815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 0x7fe5e80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e8003f70 0x7fe5e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800ba90 con 0x7fe5e8004d00 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800bc00 con 0x7fe5e8003f70 2023-11-24T08:11:54.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.321+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5e800bdc0 con 0x7fe5f80b2250 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fd527640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fcd26640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 0x7fe5e80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fd527640 1 -- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 msgr2=0x7fe5e80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5f7fff640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e8003f70 0x7fe5e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fd527640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f80b2250 0x7fe5e80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fd527640 1 -- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e8003f70 msgr2=0x7fe5e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fd527640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e8003f70 0x7fe5e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5fd527640 1 -- 172.21.15.106:0/1737989303 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5e8003520 con 0x7fe5e8004d00 2023-11-24T08:11:54.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.322+0000 7fe5f7fff640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5e8003f70 0x7fe5e80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:54.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.323+0000 7fe5fd527640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5e8005180 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000a760 tx=0x7fe5e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382436c1427d9552 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f0034630 con 0x7fe5e8004d00 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5f002f070 con 0x7fe5e8004d00 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5f77fe640 1 -- 172.21.15.106:0/1737989303 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5f00349f0 con 0x7fe5e8004d00 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1737989303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 msgr2=0x7fe5e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1737989303 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5e8005180 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fe5f000a760 tx=0x7fe5e80057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fd527640 1 -- 172.21.15.106:0/1737989303 reap_dead start 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1737989303 shutdown_connections 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1737989303 >> 172.21.15.106:0/1737989303 conn(0x7fe5f806a920 msgr2=0x7fe5f806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:54.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1737989303 shutdown_connections 2023-11-24T08:11:54.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.324+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1737989303 wait complete. 2023-11-24T08:11:54.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.326+0000 7fe5fef8a640 1 Processor -- start 2023-11-24T08:11:54.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.326+0000 7fe5fef8a640 1 -- start start 2023-11-24T08:11:54.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fef8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5f8144e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fef8a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f80705c0 0x7fe5f814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fef8a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f8070a40 0x7fe5f814ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fef8a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f8158360 con 0x7fe5e8004d00 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fef8a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f81454f0 con 0x7fe5f80705c0 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fef8a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f81457d0 con 0x7fe5f8070a40 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fcd26640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5f8144e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fcd26640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5f8144e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45724/0 (socket says 172.21.15.106:45724) 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.327+0000 7fe5fcd26640 1 -- 172.21.15.106:0/1618398628 learned_addr learned my addr 172.21.15.106:0/1618398628 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5f7fff640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f80705c0 0x7fe5f814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fcd26640 1 -- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f8070a40 msgr2=0x7fe5f814ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fcd26640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f8070a40 0x7fe5f814ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fcd26640 1 -- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f80705c0 msgr2=0x7fe5f814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fcd26640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f80705c0 0x7fe5f814b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fcd26640 1 -- 172.21.15.106:0/1618398628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f80cd7c0 con 0x7fe5e8004d00 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fcd26640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5f8144e60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec011600 tx=0x7fe5ec011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec015870 con 0x7fe5e8004d00 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5ec0159d0 con 0x7fe5e8004d00 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ec015d40 con 0x7fe5e8004d00 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.328+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f8145ab0 con 0x7fe5e8004d00 2023-11-24T08:11:54.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.329+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f8146070 con 0x7fe5e8004d00 2023-11-24T08:11:54.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.330+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f815e4c0 con 0x7fe5e8004d00 2023-11-24T08:11:54.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.331+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe5ec00f490 con 0x7fe5e8004d00 2023-11-24T08:11:54.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.332+0000 7fe5f57fa640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5dc06b230 0x7fe5dc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.332+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fe5ec08a2a0 con 0x7fe5e8004d00 2023-11-24T08:11:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.332+0000 7fe5f7fff640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5dc06b230 0x7fe5dc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.333+0000 7fe5f7fff640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5dc06b230 0x7fe5dc06d6f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:54.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.336+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5ec054d10 con 0x7fe5e8004d00 2023-11-24T08:11:54.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.566+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f8067470 con 0x7fe5e8004d00 2023-11-24T08:11:54.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.568+0000 7fe5f57fa640 1 -- 172.21.15.106:0/1618398628 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fe5ec058330 con 0x7fe5e8004d00 2023-11-24T08:11:54.570 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:54.571 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:11:32.053291+0000","modified":"2023-11-24T08:11:35.081758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/180703350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":180703350},{"type":"v1","addr":"172.21.15.106:6837","nonce":180703350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:11:54.571 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 34 2023-11-24T08:11:54.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5dc06b230 msgr2=0x7fe5dc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.573 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5dc06b230 0x7fe5dc06d6f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe5e4000c00 tx=0x7fe5e4009040 comp rx=0 tx=0).stop 2023-11-24T08:11:54.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 msgr2=0x7fe5f8144e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5f8144e60 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec011600 tx=0x7fe5ec011ac0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 shutdown_connections 2023-11-24T08:11:54.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5dc06b230 0x7fe5dc06d6f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5f8070a40 0x7fe5f814ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5f80705c0 0x7fe5f814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 --2- 172.21.15.106:0/1618398628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5e8004d00 0x7fe5f8144e60 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.570+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 >> 172.21.15.106:0/1618398628 conn(0x7fe5f806a920 msgr2=0x7fe5f80cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:54.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.571+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 shutdown_connections 2023-11-24T08:11:54.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.571+0000 7fe5fef8a640 1 -- 172.21.15.106:0/1618398628 wait complete. 2023-11-24T08:11:54.591 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:54.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.802+0000 7f1f52f7c640 1 Processor -- start 2023-11-24T08:11:54.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.803+0000 7f1f52f7c640 1 -- start start 2023-11-24T08:11:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.804+0000 7f1f52f7c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f4c0cf450 0x7f1f4c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.804+0000 7f1f52f7c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c0cece0 con 0x7f1f4c15c7f0 2023-11-24T08:11:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.804+0000 7f1f52f7c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f4c06d040 con 0x7f1f4c0cdfb0 2023-11-24T08:11:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.804+0000 7f1f52f7c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f4c0cfe40 con 0x7f1f4c0cf450 2023-11-24T08:11:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.804+0000 7f1f50d18640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1f4c15c7f0 0x7f1f4c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48994/0 (socket says 172.21.15.106:48994) 2023-11-24T08:11:54.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.804+0000 7f1f50d18640 1 -- 172.21.15.106:0/1463272031 learned_addr learned my addr 172.21.15.106:0/1463272031 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1463272031 >> v2:172.21.15.156:3300/0 conn(0x7f1f4c0cf450 0x7f1f4c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374235381 0 0) 0x7f1f4c0cece0 con 0x7f1f4c15c7f0 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f38003520 con 0x7f1f4c15c7f0 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1361802032 0 0) 0x7f1f38003520 con 0x7f1f4c15c7f0 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 >> v2:172.21.15.156:3300/0 conn(0x7f1f4c0cf450 msgr2=0x7f1f4c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 --2- 172.21.15.106:0/1463272031 >> v2:172.21.15.156:3300/0 conn(0x7f1f4c0cf450 0x7f1f4c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 >> v1:172.21.15.156:6789/0 conn(0x7f1f4c0cdfb0 legacy=0x7f1f4c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c06d320 con 0x7f1f4c15c7f0 2023-11-24T08:11:54.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f1f34001670 con 0x7f1f4c15c7f0 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 >> v1:172.21.15.106:6789/0 conn(0x7f1f4c15c7f0 legacy=0x7f1f4c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f38003970 0x7f1f38003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f380047c0 0x7f1f38004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38005180 0x7f1f3800fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0cfe40 con 0x7f1f380047c0 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f380104c0 con 0x7f1f38005180 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.805+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38010680 con 0x7f1f38003970 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f380047c0 0x7f1f38004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f51519640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38005180 0x7f1f3800fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f50d18640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f38003970 0x7f1f38003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f38003970 msgr2=0x7f1f38003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f38003970 0x7f1f38003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38005180 msgr2=0x7f1f3800fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f38005180 0x7f1f3800fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f51519640 1 -- 172.21.15.106:0/1463272031 reap_dead start 2023-11-24T08:11:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.806+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1463272031 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f38003520 con 0x7f1f380047c0 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.807+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f380047c0 0x7f1f38004c40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c007330 tx=0x7f1f380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5413ae9ae00b26a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c02cca0 con 0x7f1f380047c0 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c02ce00 con 0x7f1f380047c0 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f4b7fe640 1 -- 172.21.15.106:0/1463272031 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c034660 con 0x7f1f380047c0 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1463272031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f380047c0 msgr2=0x7f1f38004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f380047c0 0x7f1f38004c40 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c007330 tx=0x7f1f380057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1463272031 shutdown_connections 2023-11-24T08:11:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1463272031 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f380047c0 0x7f1f38004c40 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1463272031 >> 172.21.15.106:0/1463272031 conn(0x7f1f4c06a920 msgr2=0x7f1f4c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1463272031 shutdown_connections 2023-11-24T08:11:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.808+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1463272031 wait complete. 2023-11-24T08:11:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.810+0000 7f1f52f7c640 1 Processor -- start 2023-11-24T08:11:54.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.810+0000 7f1f52f7c640 1 -- start start 2023-11-24T08:11:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.811+0000 7f1f52f7c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f380047c0 0x7f1f4c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.811+0000 7f1f52f7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 0x7f1f4c152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.811+0000 7f1f52f7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c149a70 0x7f1f4c1529a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.811+0000 7f1f52f7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c151b60 con 0x7f1f4c149520 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.811+0000 7f1f52f7c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c151e40 con 0x7f1f380047c0 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.811+0000 7f1f52f7c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f4c0b1e50 con 0x7f1f4c149a70 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 0x7f1f4c152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 0x7f1f4c152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45746/0 (socket says 172.21.15.106:45746) 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1427451801 learned_addr learned my addr 172.21.15.106:0/1427451801 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f50d18640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f380047c0 0x7f1f4c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c149a70 msgr2=0x7f1f4c1529a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c149a70 0x7f1f4c1529a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f380047c0 msgr2=0x7f1f4c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f380047c0 0x7f1f4c148d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 -- 172.21.15.106:0/1427451801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c0b2130 con 0x7f1f4c149520 2023-11-24T08:11:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f4bfff640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 0x7f1f4c152460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c037040 tx=0x7f1f3c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.812+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c003b90 con 0x7f1f4c149520 2023-11-24T08:11:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.813+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f3c003cf0 con 0x7f1f4c149520 2023-11-24T08:11:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.813+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c003ea0 con 0x7f1f4c149520 2023-11-24T08:11:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.813+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f4c0b2410 con 0x7f1f4c149520 2023-11-24T08:11:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.813+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f4c151810 con 0x7f1f4c149520 2023-11-24T08:11:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.814+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f4c067470 con 0x7f1f4c149520 2023-11-24T08:11:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.815+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1f3c004260 con 0x7f1f4c149520 2023-11-24T08:11:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.816+0000 7f1f497fa640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f2c06b260 0x7f1f2c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.816+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1f3c0abff0 con 0x7f1f4c149520 2023-11-24T08:11:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.816+0000 7f1f50d18640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f2c06b260 0x7f1f2c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.817+0000 7f1f50d18640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f2c06b260 0x7f1f2c06d720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1f34001cb0 tx=0x7f1f340014b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:54.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:54.820+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f3c076a80 con 0x7f1f4c149520 2023-11-24T08:11:55.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.012+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f4c15e290 con 0x7f1f4c149520 2023-11-24T08:11:55.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.014+0000 7f1f497fa640 1 -- 172.21.15.106:0/1427451801 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16678 (secure 0 0 0) 0x7f1f3c039090 con 0x7f1f4c149520 2023-11-24T08:11:55.017 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:55.017 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:53.317419+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:55.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.016+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f2c06b260 msgr2=0x7f1f2c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.016+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f2c06b260 0x7f1f2c06d720 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1f34001cb0 tx=0x7f1f340014b0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.016+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 msgr2=0x7f1f4c152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.016+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 0x7f1f4c152460 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c037040 tx=0x7f1f3c004ab0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 shutdown_connections 2023-11-24T08:11:55.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f2c06b260 0x7f1f2c06d720 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f4c149a70 0x7f1f4c1529a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f4c149520 0x7f1f4c152460 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 --2- 172.21.15.106:0/1427451801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f380047c0 0x7f1f4c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 >> 172.21.15.106:0/1427451801 conn(0x7f1f4c06a920 msgr2=0x7f1f4c15eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:55.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 shutdown_connections 2023-11-24T08:11:55.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.017+0000 7f1f52f7c640 1 -- 172.21.15.106:0/1427451801 wait complete. 2023-11-24T08:11:55.039 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:11:55.040 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.250+0000 7fd4dafe4640 1 Processor -- start 2023-11-24T08:11:55.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.250+0000 7fd4dafe4640 1 -- start start 2023-11-24T08:11:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.251+0000 7fd4dafe4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4d4144b30 0x7fd4d415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.251+0000 7fd4dafe4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd4d40b1de0 0x7fd4d40b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.251+0000 7fd4dafe4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d415bf80 con 0x7fd4d415c410 2023-11-24T08:11:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.251+0000 7fd4dafe4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40b27b0 con 0x7fd4d4144b30 2023-11-24T08:11:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.251+0000 7fd4dafe4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d40b1560 con 0x7fd4d40b1de0 2023-11-24T08:11:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4d9fe2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4d4144b30 0x7fd4d415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4d9fe2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd4d4144b30 0x7fd4d415f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45760/0 (socket says 172.21.15.106:45760) 2023-11-24T08:11:55.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4d9fe2640 1 -- 172.21.15.106:0/164488970 learned_addr learned my addr 172.21.15.106:0/164488970 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:55.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4da7e3640 1 --2- 172.21.15.106:0/164488970 >> v2:172.21.15.156:3300/0 conn(0x7fd4d40b1de0 0x7fd4d40b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4d9fe2640 1 -- 172.21.15.106:0/164488970 >> v2:172.21.15.156:3300/0 conn(0x7fd4d40b1de0 msgr2=0x7fd4d40b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4d9fe2640 1 --2- 172.21.15.106:0/164488970 >> v2:172.21.15.156:3300/0 conn(0x7fd4d40b1de0 0x7fd4d40b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.252+0000 7fd4d9fe2640 1 -- 172.21.15.106:0/164488970 >> v1:172.21.15.106:6789/0 conn(0x7fd4d415c410 legacy=0x7fd4d40b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.253+0000 7fd4d9fe2640 1 -- 172.21.15.106:0/164488970 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d40b1280 con 0x7fd4d4144b30 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.253+0000 7fd4d8fe0640 1 -- 172.21.15.106:0/164488970 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539426524 0 0) 0x7fd4d415bf80 con 0x7fd4d415c410 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.253+0000 7fd4da7e3640 1 --2- 172.21.15.106:0/164488970 >> v2:172.21.15.156:3300/0 conn(0x7fd4d40b1de0 0x7fd4d40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.254+0000 7fd4d9fe2640 1 --2- 172.21.15.106:0/164488970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d415f920 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4072a50 tx=0x7fd4d002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e714533c32a0b19 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.254+0000 7fd4d8fe0640 1 -- 172.21.15.106:0/164488970 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0030860 con 0x7fd4d4144b30 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.254+0000 7fd4d8fe0640 1 -- 172.21.15.106:0/164488970 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00309c0 con 0x7fd4d4144b30 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.254+0000 7fd4d8fe0640 1 -- 172.21.15.106:0/164488970 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0030d50 con 0x7fd4d4144b30 2023-11-24T08:11:55.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 -- 172.21.15.106:0/164488970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 msgr2=0x7fd4d415f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/164488970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d415f920 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4072a50 tx=0x7fd4d002b9a0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 -- 172.21.15.106:0/164488970 shutdown_connections 2023-11-24T08:11:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/164488970 >> v2:172.21.15.156:3300/0 conn(0x7fd4d40b1de0 0x7fd4d40b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/164488970 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d415f920 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 -- 172.21.15.106:0/164488970 >> 172.21.15.106:0/164488970 conn(0x7fd4d40cb830 msgr2=0x7fd4d40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 -- 172.21.15.106:0/164488970 shutdown_connections 2023-11-24T08:11:55.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.255+0000 7fd4dafe4640 1 -- 172.21.15.106:0/164488970 wait complete. 2023-11-24T08:11:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.257+0000 7fd4dafe4640 1 Processor -- start 2023-11-24T08:11:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.257+0000 7fd4dafe4640 1 -- start start 2023-11-24T08:11:55.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4dafe4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d40b1de0 0x7fd4d413bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4dafe4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d413c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4dafe4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4d415c410 0x7fd4d4151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4dafe4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d413b090 con 0x7fd4d4144b30 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4dafe4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4141db0 con 0x7fd4d40b1de0 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4dafe4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d4141480 con 0x7fd4d415c410 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4d97e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d413c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.258+0000 7fd4d97e1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d413c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45762/0 (socket says 172.21.15.106:45762) 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 -- 172.21.15.106:0/1327844838 learned_addr learned my addr 172.21.15.106:0/1327844838 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 -- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4d415c410 msgr2=0x7fd4d4151fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4d415c410 0x7fd4d4151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d9fe2640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d40b1de0 0x7fd4d413bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 -- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d40b1de0 msgr2=0x7fd4d413bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d40b1de0 0x7fd4d413bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 -- 172.21.15.106:0/1327844838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00094d0 con 0x7fd4d4144b30 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4d97e1640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d413c020 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c410 tx=0x7fd4c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c40035f0 con 0x7fd4d4144b30 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c4003770 con 0x7fd4d4144b30 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.259+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c4003b20 con 0x7fd4d4144b30 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.260+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d4141760 con 0x7fd4d4144b30 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.260+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d4152520 con 0x7fd4d4144b30 2023-11-24T08:11:55.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.261+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d404e840 con 0x7fd4d4144b30 2023-11-24T08:11:55.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.262+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd4c4003c80 con 0x7fd4d4144b30 2023-11-24T08:11:55.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.263+0000 7fd4ca7fc640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4b806b230 0x7fd4b806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.263+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fd4c40893c0 con 0x7fd4d4144b30 2023-11-24T08:11:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.263+0000 7fd4d9fe2640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4b806b230 0x7fd4b806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.264+0000 7fd4d9fe2640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4b806b230 0x7fd4b806d6f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0036000 tx=0x7fd4d003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:55.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.267+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd4c4053e30 con 0x7fd4d4144b30 2023-11-24T08:11:55.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.498+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4d4141ef0 con 0x7fd4d4144b30 2023-11-24T08:11:55.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.499+0000 7fd4ca7fc640 1 -- 172.21.15.106:0/1327844838 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4919 (secure 0 0 0) 0x7fd4c4057450 con 0x7fd4d4144b30 2023-11-24T08:11:55.502 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:55.502 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:11:32.053291+0000","modified":"2023-11-24T08:11:35.081758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/180703350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":180703350},{"type":"v1","addr":"172.21.15.106:6837","nonce":180703350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:11:55.503 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 34 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4b806b230 msgr2=0x7fd4b806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4b806b230 0x7fd4b806d6f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0036000 tx=0x7fd4d003b040 comp rx=0 tx=0).stop 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 msgr2=0x7fd4d413c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d413c020 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fd4c400c410 tx=0x7fd4c400c8d0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 shutdown_connections 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd4b806b230 0x7fd4b806d6f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd4d415c410 0x7fd4d4151fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4d4144b30 0x7fd4d413c020 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 --2- 172.21.15.106:0/1327844838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d40b1de0 0x7fd4d413bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 >> 172.21.15.106:0/1327844838 conn(0x7fd4d40cb830 msgr2=0x7fd4d40cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:55.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 shutdown_connections 2023-11-24T08:11:55.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.502+0000 7fd4dafe4640 1 -- 172.21.15.106:0/1327844838 wait complete. 2023-11-24T08:11:55.523 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:11:55.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.736+0000 7f2f15690640 1 Processor -- start 2023-11-24T08:11:55.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.736+0000 7f2f15690640 1 -- start start 2023-11-24T08:11:55.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.737+0000 7f2f15690640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f1015c7f0 0x7f2f1006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.737+0000 7f2f15690640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2f100ce6c0 0x7f2f100cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.737+0000 7f2f15690640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f100cece0 con 0x7f2f100cf450 2023-11-24T08:11:55.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.737+0000 7f2f15690640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f10156010 con 0x7f2f100ce6c0 2023-11-24T08:11:55.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.737+0000 7f2f15690640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1006ce30 con 0x7f2f1015c7f0 2023-11-24T08:11:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2f100ce6c0 0x7f2f100cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2f100ce6c0 0x7f2f100cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45768/0 (socket says 172.21.15.106:45768) 2023-11-24T08:11:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 -- 172.21.15.106:0/2864133883 learned_addr learned my addr 172.21.15.106:0/2864133883 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0effd640 1 --2- 172.21.15.106:0/2864133883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f1015c7f0 0x7f2f1006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 -- 172.21.15.106:0/2864133883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f1015c7f0 msgr2=0x7f2f1006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 --2- 172.21.15.106:0/2864133883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f1015c7f0 0x7f2f1006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 -- 172.21.15.106:0/2864133883 >> v1:172.21.15.156:6789/0 conn(0x7f2f100cf450 legacy=0x7f2f100cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.738+0000 7f2f0f7fe640 1 -- 172.21.15.106:0/2864133883 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1006d040 con 0x7f2f100ce6c0 2023-11-24T08:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.739+0000 7f2f0f7fe640 1 --2- 172.21.15.106:0/2864133883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100ce6c0 0x7f2f100cf8d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009e50 tx=0x7f2f0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6c84784d2818ab8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.740+0000 7f2f0e7fc640 1 -- 172.21.15.106:0/2864133883 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0802ca70 con 0x7f2f100ce6c0 2023-11-24T08:11:55.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.740+0000 7f2f0e7fc640 1 -- 172.21.15.106:0/2864133883 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0802cbd0 con 0x7f2f100ce6c0 2023-11-24T08:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.740+0000 7f2f0e7fc640 1 -- 172.21.15.106:0/2864133883 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f080343f0 con 0x7f2f100ce6c0 2023-11-24T08:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.740+0000 7f2f15690640 1 -- 172.21.15.106:0/2864133883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100ce6c0 msgr2=0x7f2f100cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.740+0000 7f2f15690640 1 --2- 172.21.15.106:0/2864133883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100ce6c0 0x7f2f100cf8d0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009e50 tx=0x7f2f0802bed0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.741+0000 7f2f15690640 1 -- 172.21.15.106:0/2864133883 shutdown_connections 2023-11-24T08:11:55.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.741+0000 7f2f15690640 1 --2- 172.21.15.106:0/2864133883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100ce6c0 0x7f2f100cf8d0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.741+0000 7f2f15690640 1 --2- 172.21.15.106:0/2864133883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f1015c7f0 0x7f2f1006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.741+0000 7f2f15690640 1 -- 172.21.15.106:0/2864133883 >> 172.21.15.106:0/2864133883 conn(0x7f2f1006a920 msgr2=0x7f2f1006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.741+0000 7f2f15690640 1 -- 172.21.15.106:0/2864133883 shutdown_connections 2023-11-24T08:11:55.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.741+0000 7f2f15690640 1 -- 172.21.15.106:0/2864133883 wait complete. 2023-11-24T08:11:55.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.742+0000 7f2f15690640 1 Processor -- start 2023-11-24T08:11:55.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.743+0000 7f2f15690640 1 -- start start 2023-11-24T08:11:55.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.743+0000 7f2f15690640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f100ce6c0 0x7f2f100ace60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f15690640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 0x7f2f10149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f15690640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f1015c7f0 0x7f2f10152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f15690640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100addd0 con 0x7f2f100cf450 2023-11-24T08:11:55.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f15690640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10149d60 con 0x7f2f1015c7f0 2023-11-24T08:11:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f15690640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100ac9d0 con 0x7f2f100ce6c0 2023-11-24T08:11:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 0x7f2f10149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 0x7f2f10149820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45778/0 (socket says 172.21.15.106:45778) 2023-11-24T08:11:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 -- 172.21.15.106:0/1922968729 learned_addr learned my addr 172.21.15.106:0/1922968729 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f0effd640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f100ce6c0 0x7f2f100ace60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 -- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f100ce6c0 msgr2=0x7f2f100ace60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f100ce6c0 0x7f2f100ace60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 -- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f1015c7f0 msgr2=0x7f2f10152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:55.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f0f7fe640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f1015c7f0 0x7f2f10152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f1015c7f0 0x7f2f10152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:55.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.744+0000 7f2f067fc640 1 -- 172.21.15.106:0/1922968729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08009b30 con 0x7f2f100cf450 2023-11-24T08:11:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.745+0000 7f2f067fc640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 0x7f2f10149820 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c430 tx=0x7f2ef800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.745+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8011790 con 0x7f2f100cf450 2023-11-24T08:11:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.745+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef80118f0 con 0x7f2f100cf450 2023-11-24T08:11:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.745+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef8011ce0 con 0x7f2f100cf450 2023-11-24T08:11:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.745+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f10156810 con 0x7f2f100cf450 2023-11-24T08:11:55.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.745+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f100adfb0 con 0x7f2f100cf450 2023-11-24T08:11:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.746+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f10067470 con 0x7f2f100cf450 2023-11-24T08:11:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.747+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f2ef801c460 con 0x7f2f100cf450 2023-11-24T08:11:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.748+0000 7f2f07fff640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ef006b230 0x7f2ef006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.748+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f2ef8089de0 con 0x7f2f100cf450 2023-11-24T08:11:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.748+0000 7f2f0effd640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ef006b230 0x7f2ef006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:55.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.749+0000 7f2f0effd640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ef006b230 0x7f2ef006d6f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006fd0 tx=0x7f2f00008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:55.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.752+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2ef8054800 con 0x7f2f100cf450 2023-11-24T08:11:55.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.981+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f10159280 con 0x7f2f100cf450 2023-11-24T08:11:56.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.983+0000 7f2f07fff640 1 -- 172.21.15.106:0/1922968729 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4919 (secure 0 0 0) 0x7f2ef8057e20 con 0x7f2f100cf450 2023-11-24T08:11:56.057 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:56.057 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:11:32.053291+0000","modified":"2023-11-24T08:11:35.081758+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6075},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6075":{"gid":6075,"name":"c","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/180703350","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":180703350},{"type":"v1","addr":"172.21.15.106:6837","nonce":180703350}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 34 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ef006b230 msgr2=0x7f2ef006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ef006b230 0x7f2ef006d6f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2f00006fd0 tx=0x7f2f00008040 comp rx=0 tx=0).stop 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 msgr2=0x7f2f10149820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 0x7f2f10149820 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f2ef800c430 tx=0x7f2ef800c8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 shutdown_connections 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f1015c7f0 0x7f2f10152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2f100cf450 0x7f2f10149820 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ef006b230 0x7f2ef006d6f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 --2- 172.21.15.106:0/1922968729 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2f100ce6c0 0x7f2f100ace60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.985+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 >> 172.21.15.106:0/1922968729 conn(0x7f2f1006a920 msgr2=0x7f2f100718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.986+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 shutdown_connections 2023-11-24T08:11:56.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:55.986+0000 7f2f15690640 1 -- 172.21.15.106:0/1922968729 wait complete. 2023-11-24T08:11:56.060 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:11:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.280+0000 7fa753701640 1 Processor -- start 2023-11-24T08:11:56.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.281+0000 7fa753701640 1 -- start start 2023-11-24T08:11:56.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa753701640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 0x7fa74c161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa753701640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa74c0cf450 0x7fa74c0cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa753701640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa74c0ce0b0 con 0x7fa74c15dc80 2023-11-24T08:11:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa753701640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa74c0cfe10 con 0x7fa74c0cf450 2023-11-24T08:11:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa753701640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c161960 con 0x7fa74c157c70 2023-11-24T08:11:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa751c9e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa74c15dc80 0x7fa74c15e110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49010/0 (socket says 172.21.15.106:49010) 2023-11-24T08:11:56.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.282+0000 7fa751c9e640 1 -- 172.21.15.106:0/605795239 learned_addr learned my addr 172.21.15.106:0/605795239 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa75149d640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 0x7fa74c161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa750c9c640 1 --2- 172.21.15.106:0/605795239 >> v2:172.21.15.156:3300/0 conn(0x7fa74c0cf450 0x7fa74c0cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1114576543 0 0) 0x7fa74c0ce0b0 con 0x7fa74c15dc80 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa738003520 con 0x7fa74c15dc80 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1023945383 0 0) 0x7fa738003520 con 0x7fa74c15dc80 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 msgr2=0x7fa74c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 0x7fa74c161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 >> v2:172.21.15.156:3300/0 conn(0x7fa74c0cf450 msgr2=0x7fa74c0cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 --2- 172.21.15.106:0/605795239 >> v2:172.21.15.156:3300/0 conn(0x7fa74c0cf450 0x7fa74c0cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c15e8d0 con 0x7fa74c15dc80 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fa7440019f0 con 0x7fa74c15dc80 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 >> v1:172.21.15.106:6789/0 conn(0x7fa74c15dc80 legacy=0x7fa74c15e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 0x7fa738003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa738003f40 0x7fa7380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d00 0x7fa738005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c0cfe10 con 0x7fa738004d00 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c161960 con 0x7fa738003f40 2023-11-24T08:11:56.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.283+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73800ba30 con 0x7fa74c157c70 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa751c9e640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d00 0x7fa738005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa75149d640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 0x7fa738003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa750c9c640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa738003f40 0x7fa7380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa751c9e640 1 -- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 msgr2=0x7fa738003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa751c9e640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c157c70 0x7fa738003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa751c9e640 1 -- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa738003f40 msgr2=0x7fa7380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa751c9e640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa738003f40 0x7fa7380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa751c9e640 1 -- 172.21.15.106:0/605795239 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa738003520 con 0x7fa738004d00 2023-11-24T08:11:56.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.284+0000 7fa750c9c640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa738003f40 0x7fa7380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.285+0000 7fa751c9e640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d00 0x7fa738005180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa74400a790 tx=0x7fa7380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86840a6273aadfa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744034630 con 0x7fa738004d00 2023-11-24T08:11:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa74402f070 con 0x7fa738004d00 2023-11-24T08:11:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa743fff640 1 -- 172.21.15.106:0/605795239 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa744034a30 con 0x7fa738004d00 2023-11-24T08:11:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa753701640 1 -- 172.21.15.106:0/605795239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d00 msgr2=0x7fa738005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa753701640 1 --2- 172.21.15.106:0/605795239 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa738004d00 0x7fa738005180 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fa74400a790 tx=0x7fa7380057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa751c9e640 1 -- 172.21.15.106:0/605795239 reap_dead start 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa753701640 1 -- 172.21.15.106:0/605795239 shutdown_connections 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa753701640 1 -- 172.21.15.106:0/605795239 >> 172.21.15.106:0/605795239 conn(0x7fa74c06a920 msgr2=0x7fa74c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa753701640 1 -- 172.21.15.106:0/605795239 shutdown_connections 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.286+0000 7fa753701640 1 -- 172.21.15.106:0/605795239 wait complete. 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.288+0000 7fa753701640 1 Processor -- start 2023-11-24T08:11:56.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.288+0000 7fa753701640 1 -- start start 2023-11-24T08:11:56.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa753701640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c1f2b00 0x7fa74c099140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa753701640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 0x7fa74c0ad6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa753701640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa74c0adc30 0x7fa74c0ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa753701640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c152e40 con 0x7fa74c0ad270 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa753701640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c09d070 con 0x7fa74c0adc30 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa753701640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa74c09cab0 con 0x7fa74c1f2b00 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.289+0000 7fa750c9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 0x7fa74c0ad6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 0x7fa74c0ad6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45800/0 (socket says 172.21.15.106:45800) 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 -- 172.21.15.106:0/2657729017 learned_addr learned my addr 172.21.15.106:0/2657729017 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 -- 172.21.15.106:0/2657729017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c1f2b00 msgr2=0x7fa74c099140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c1f2b00 0x7fa74c099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 -- 172.21.15.106:0/2657729017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa74c0adc30 msgr2=0x7fa74c0ae0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa74c0adc30 0x7fa74c0ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 -- 172.21.15.106:0/2657729017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa74c09cd90 con 0x7fa74c0ad270 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa750c9c640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 0x7fa74c0ad6f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0109d0 tx=0x7fa73c010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c00eba0 con 0x7fa74c0ad270 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73c00ed00 con 0x7fa74c0ad270 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.290+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa74c0aca70 con 0x7fa74c0ad270 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.291+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73c01c620 con 0x7fa74c0ad270 2023-11-24T08:11:56.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.291+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa74c1f2ee0 con 0x7fa74c0ad270 2023-11-24T08:11:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.292+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa74c067470 con 0x7fa74c0ad270 2023-11-24T08:11:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.293+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa73c01c780 con 0x7fa74c0ad270 2023-11-24T08:11:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.294+0000 7fa741ffb640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73006b260 0x7fa73006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.294+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa73c08a3c0 con 0x7fa74c0ad270 2023-11-24T08:11:56.296 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.294+0000 7fa75149d640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73006b260 0x7fa73006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.295+0000 7fa75149d640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73006b260 0x7fa73006d720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0b3d00 tx=0x7fa748008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:56.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.298+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa73c054ee0 con 0x7fa74c0ad270 2023-11-24T08:11:56.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.492+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa74c0cd910 con 0x7fa74c0ad270 2023-11-24T08:11:56.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.495+0000 7fa741ffb640 1 -- 172.21.15.106:0/2657729017 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16678 (secure 0 0 0) 0x7fa73c058500 con 0x7fa74c0ad270 2023-11-24T08:11:56.498 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:11:56.498 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":64,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:11:53.317419+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:11:29.872463+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-24T08:11:31.045762+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":63,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":9,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:11:56.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.497+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73006b260 msgr2=0x7fa73006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.497+0000 7fa753701640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73006b260 0x7fa73006d720 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fa74c0b3d00 tx=0x7fa748008040 comp rx=0 tx=0).stop 2023-11-24T08:11:56.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.497+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 msgr2=0x7fa74c0ad6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.497+0000 7fa753701640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 0x7fa74c0ad6f0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0109d0 tx=0x7fa73c010e90 comp rx=0 tx=0).stop 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 shutdown_connections 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa74c0adc30 0x7fa74c0ae0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa74c0ad270 0x7fa74c0ad6f0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa73006b260 0x7fa73006d720 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 --2- 172.21.15.106:0/2657729017 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa74c1f2b00 0x7fa74c099140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 >> 172.21.15.106:0/2657729017 conn(0x7fa74c06a920 msgr2=0x7fa74c0b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 shutdown_connections 2023-11-24T08:11:56.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.498+0000 7fa753701640 1 -- 172.21.15.106:0/2657729017 wait complete. 2023-11-24T08:11:56.518 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:11:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.732+0000 7f72e5c94640 1 Processor -- start 2023-11-24T08:11:56.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.732+0000 7f72e5c94640 1 -- start start 2023-11-24T08:11:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.733+0000 7f72e5c94640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72e01684e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.733+0000 7f72e5c94640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e0161a10 con 0x7f72e006e160 2023-11-24T08:11:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.733+0000 7f72e5c94640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e0161cf0 con 0x7f72e0146eb0 2023-11-24T08:11:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.733+0000 7f72e5c94640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0161fd0 con 0x7f72e00cd2e0 2023-11-24T08:11:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.734+0000 7f72deffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f72e006e160 0x7f72e00ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49012/0 (socket says 172.21.15.106:49012) 2023-11-24T08:11:56.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.734+0000 7f72deffd640 1 -- 172.21.15.106:0/520794291 learned_addr learned my addr 172.21.15.106:0/520794291 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.734+0000 7f72dffff640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72e01684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.734+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3083407686 0 0) 0x7f72e0161a10 con 0x7f72e006e160 2023-11-24T08:11:56.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.734+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72cc003520 con 0x7f72e006e160 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.734+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1591533560 0 0) 0x7f72cc003520 con 0x7f72e006e160 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 msgr2=0x7f72e01684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72e01684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 >> v1:172.21.15.156:6789/0 conn(0x7f72e0146eb0 legacy=0x7f72e0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e0070660 con 0x7f72e006e160 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f72c8001690 con 0x7f72e006e160 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 >> v1:172.21.15.106:6789/0 conn(0x7f72e006e160 legacy=0x7f72e00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72cc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72cc003f20 0x7f72cc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e0161fd0 con 0x7f72cc004d00 2023-11-24T08:11:56.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc00ba60 con 0x7f72cc003f20 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc00bc20 con 0x7f72e00cd2e0 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72dffff640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72deffd640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72cc003f20 0x7f72cc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72df7fe640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72cc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72dffff640 1 -- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 msgr2=0x7f72cc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72dffff640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72cc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72dffff640 1 -- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72cc003f20 msgr2=0x7f72cc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72dffff640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72cc003f20 0x7f72cc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72dffff640 1 -- 172.21.15.106:0/520794291 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc003520 con 0x7f72cc004d00 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72df7fe640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00cd2e0 0x7f72cc0039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:56.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.735+0000 7f72deffd640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72cc003f20 0x7f72cc0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72dffff640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72cc005180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009ac0 tx=0x7f72cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffb42f4d1e8300f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d802c6b0 con 0x7f72cc004d00 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d802c810 con 0x7f72cc004d00 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72de7fc640 1 -- 172.21.15.106:0/520794291 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d802cba0 con 0x7f72cc004d00 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72e5c94640 1 -- 172.21.15.106:0/520794291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 msgr2=0x7f72cc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72e5c94640 1 --2- 172.21.15.106:0/520794291 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72cc005180 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f72d8009ac0 tx=0x7f72cc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72dffff640 1 -- 172.21.15.106:0/520794291 reap_dead start 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72e5c94640 1 -- 172.21.15.106:0/520794291 shutdown_connections 2023-11-24T08:11:56.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72e5c94640 1 -- 172.21.15.106:0/520794291 >> 172.21.15.106:0/520794291 conn(0x7f72e00b0c50 msgr2=0x7f72e00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72e5c94640 1 -- 172.21.15.106:0/520794291 shutdown_connections 2023-11-24T08:11:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.737+0000 7f72e5c94640 1 -- 172.21.15.106:0/520794291 wait complete. 2023-11-24T08:11:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.739+0000 7f72e5c94640 1 Processor -- start 2023-11-24T08:11:56.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.739+0000 7f72e5c94640 1 -- start start 2023-11-24T08:11:56.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.740+0000 7f72e5c94640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72e015cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.740+0000 7f72e5c94640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00b4650 0x7f72e015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.740+0000 7f72e5c94640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e015e420 0x7f72e00acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.740+0000 7f72e5c94640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b4230 con 0x7f72cc004d00 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.740+0000 7f72e5c94640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e015d420 con 0x7f72e015e420 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.740+0000 7f72e5c94640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e00b32d0 con 0x7f72e00b4650 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72e015cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72e015cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45824/0 (socket says 172.21.15.106:45824) 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 -- 172.21.15.106:0/1057282804 learned_addr learned my addr 172.21.15.106:0/1057282804 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72deffd640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00b4650 0x7f72e015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 -- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00b4650 msgr2=0x7f72e015dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00b4650 0x7f72e015dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 -- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e015e420 msgr2=0x7f72e00acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e015e420 0x7f72e00acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 -- 172.21.15.106:0/1057282804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00b35b0 con 0x7f72cc004d00 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.741+0000 7f72df7fe640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72e015cee0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f72d000db10 tx=0x7f72d00020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.742+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0003da0 con 0x7f72cc004d00 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.742+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0006b50 con 0x7f72cc004d00 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.742+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d000b740 con 0x7f72cc004d00 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.742+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e00b3890 con 0x7f72cc004d00 2023-11-24T08:11:56.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.742+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e00b3e50 con 0x7f72cc004d00 2023-11-24T08:11:56.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.743+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e00b44a0 con 0x7f72cc004d00 2023-11-24T08:11:56.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.744+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f72d000b8a0 con 0x7f72cc004d00 2023-11-24T08:11:56.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.745+0000 7f72d7fff640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72c006b230 0x7f72c006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:56.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.745+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f72d00893e0 con 0x7f72cc004d00 2023-11-24T08:11:56.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.745+0000 7f72deffd640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72c006b230 0x7f72c006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:56.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.746+0000 7f72deffd640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72c006b230 0x7f72c006d6f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f72c80017c0 tx=0x7f72c8001490 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:56.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.750+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f72d0053df0 con 0x7f72cc004d00 2023-11-24T08:11:56.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.977+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f72e00cb560 con 0x7f72cc004d00 2023-11-24T08:11:56.980 INFO:tasks.ceph.mon.a.smithi106.stderr:2023-11-24T08:11:56.977+0000 7fb1127dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:11:56.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.980+0000 7f72d7fff640 1 -- 172.21.15.106:0/1057282804 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f72d0057410 con 0x7f72cc004d00 2023-11-24T08:11:56.983 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:11:56.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72c006b230 msgr2=0x7f72c006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72c006b230 0x7f72c006d6f0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f72c80017c0 tx=0x7f72c8001490 comp rx=0 tx=0).stop 2023-11-24T08:11:56.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 msgr2=0x7f72e015cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:56.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72e015cee0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f72d000db10 tx=0x7f72d00020c0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 shutdown_connections 2023-11-24T08:11:56.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72e015e420 0x7f72e00acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f72c006b230 0x7f72c006d6f0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72e00b4650 0x7f72e015dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 --2- 172.21.15.106:0/1057282804 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72cc004d00 0x7f72e015cee0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.984+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 >> 172.21.15.106:0/1057282804 conn(0x7f72e00b0c50 msgr2=0x7f72e00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.985+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 shutdown_connections 2023-11-24T08:11:56.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:56.985+0000 7f72e5c94640 1 -- 172.21.15.106:0/1057282804 wait complete. 2023-11-24T08:11:57.008 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:11:57.063 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:11:57.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.200+0000 7ff5ef0aa640 1 Processor -- start 2023-11-24T08:11:57.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.200+0000 7ff5ef0aa640 1 -- start start 2023-11-24T08:11:57.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ef0aa640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff5e8143d30 0x7ff5e8071fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:57.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ef0aa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e81450d0 con 0x7ff5e80b1260 2023-11-24T08:11:57.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ef0aa640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5e81422b0 con 0x7ff5e8141b90 2023-11-24T08:11:57.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ef0aa640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5e80725e0 con 0x7ff5e8143d30 2023-11-24T08:11:57.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ee8a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff5e8143d30 0x7ff5e8071fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:57.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ed8a7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff5e80b1260 0x7ff5e80718d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49014/0 (socket says 172.21.15.106:49014) 2023-11-24T08:11:57.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ee8a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff5e8143d30 0x7ff5e8071fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45842/0 (socket says 172.21.15.106:45842) 2023-11-24T08:11:57.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ed8a7640 1 -- 172.21.15.106:0/4230757874 learned_addr learned my addr 172.21.15.106:0/4230757874 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:57.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ee8a9640 1 -- 172.21.15.106:0/4230757874 >> v1:172.21.15.156:6789/0 conn(0x7ff5e8141b90 legacy=0x7ff5e80b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:57.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.202+0000 7ff5ed0a6640 1 -- 172.21.15.106:0/4230757874 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961214861 0 0) 0x7ff5e81450d0 con 0x7ff5e80b1260 2023-11-24T08:11:57.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.203+0000 7ff5ee8a9640 1 -- 172.21.15.106:0/4230757874 >> v1:172.21.15.106:6789/0 conn(0x7ff5e80b1260 legacy=0x7ff5e80718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:57.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.203+0000 7ff5ee8a9640 1 -- 172.21.15.106:0/4230757874 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8142520 con 0x7ff5e8143d30 2023-11-24T08:11:57.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.204+0000 7ff5ee8a9640 1 --2- 172.21.15.106:0/4230757874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8143d30 0x7ff5e8071fe0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009d90 tx=0x7ff5e002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be12520a516b7eb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:57.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.204+0000 7ff5ed0a6640 1 -- 172.21.15.106:0/4230757874 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e002cc60 con 0x7ff5e8143d30 2023-11-24T08:11:57.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.204+0000 7ff5ed0a6640 1 -- 172.21.15.106:0/4230757874 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e002cdc0 con 0x7ff5e8143d30 2023-11-24T08:11:57.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.204+0000 7ff5ed0a6640 1 -- 172.21.15.106:0/4230757874 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0034660 con 0x7ff5e8143d30 2023-11-24T08:11:57.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.204+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/4230757874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8143d30 msgr2=0x7ff5e8071fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:57.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.204+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/4230757874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8143d30 0x7ff5e8071fe0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0009d90 tx=0x7ff5e002beb0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.205+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/4230757874 shutdown_connections 2023-11-24T08:11:57.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.205+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/4230757874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8143d30 0x7ff5e8071fe0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.205+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/4230757874 >> 172.21.15.106:0/4230757874 conn(0x7ff5e806a500 msgr2=0x7ff5e806a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:57.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.205+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/4230757874 shutdown_connections 2023-11-24T08:11:57.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.205+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/4230757874 wait complete. 2023-11-24T08:11:57.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.206+0000 7ff5ef0aa640 1 Processor -- start 2023-11-24T08:11:57.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.207+0000 7ff5ef0aa640 1 -- start start 2023-11-24T08:11:57.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.207+0000 7ff5ef0aa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e80b1260 0x7ff5e813d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:57.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.207+0000 7ff5ef0aa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 0x7ff5e81548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:57.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ef0aa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5e8143d30 0x7ff5e81401d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:57.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ef0aa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e80cd960 con 0x7ff5e8141b90 2023-11-24T08:11:57.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ef0aa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e81556c0 con 0x7ff5e80b1260 2023-11-24T08:11:57.211 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ef0aa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e815bfd0 con 0x7ff5e8143d30 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 0x7ff5e81548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 0x7ff5e81548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45852/0 (socket says 172.21.15.106:45852) 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 -- 172.21.15.106:0/2627889785 learned_addr learned my addr 172.21.15.106:0/2627889785 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 -- 172.21.15.106:0/2627889785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5e8143d30 msgr2=0x7ff5e81401d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5e8143d30 0x7ff5e81401d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 -- 172.21.15.106:0/2627889785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e80b1260 msgr2=0x7ff5e813d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e80b1260 0x7ff5e813d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 -- 172.21.15.106:0/2627889785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e0009a70 con 0x7ff5e8141b90 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.208+0000 7ff5ed8a7640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 0x7ff5e81548b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c420 tx=0x7ff5d800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.209+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d80074e0 con 0x7ff5e8141b90 2023-11-24T08:11:57.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.209+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8007640 con 0x7ff5e8141b90 2023-11-24T08:11:57.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.209+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8004a00 con 0x7ff5e8141b90 2023-11-24T08:11:57.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.209+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8155100 con 0x7ff5e8141b90 2023-11-24T08:11:57.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.209+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e815bc60 con 0x7ff5e8141b90 2023-11-24T08:11:57.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.210+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8067de0 con 0x7ff5e8141b90 2023-11-24T08:11:57.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.211+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff5d8004b60 con 0x7ff5e8141b90 2023-11-24T08:11:57.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.212+0000 7ff5deffd640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5cc06b230 0x7ff5cc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:57.215 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.212+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff5d8089a20 con 0x7ff5e8141b90 2023-11-24T08:11:57.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.213+0000 7ff5ee0a8640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5cc06b230 0x7ff5cc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:57.216 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.213+0000 7ff5ee0a8640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5cc06b230 0x7ff5cc06d6f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000f60 tx=0x7ff5e40079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:57.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.218+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff5d80545f0 con 0x7ff5e8141b90 2023-11-24T08:11:57.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.443+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff5e804e840 con 0x7ff5e8141b90 2023-11-24T08:11:57.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.992+0000 7ff5deffd640 1 -- 172.21.15.106:0/2627889785 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff5d8057c10 con 0x7ff5e8141b90 2023-11-24T08:11:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5cc06b230 msgr2=0x7ff5cc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5cc06b230 0x7ff5cc06d6f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7ff5e4000f60 tx=0x7ff5e40079b0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 msgr2=0x7ff5e81548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 0x7ff5e81548b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c420 tx=0x7ff5d800c8e0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 shutdown_connections 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff5cc06b230 0x7ff5cc06d6f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff5e8143d30 0x7ff5e81401d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff5e8141b90 0x7ff5e81548b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 --2- 172.21.15.106:0/2627889785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5e80b1260 0x7ff5e813d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 >> 172.21.15.106:0/2627889785 conn(0x7ff5e806a500 msgr2=0x7ff5e813e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 shutdown_connections 2023-11-24T08:11:57.998 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:57.995+0000 7ff5ef0aa640 1 -- 172.21.15.106:0/2627889785 wait complete. 2023-11-24T08:11:58.018 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:11:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.221+0000 7fb229161640 1 Processor -- start 2023-11-24T08:11:58.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.221+0000 7fb229161640 1 -- start start 2023-11-24T08:11:58.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb229161640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb224144e80 0x7fb224070a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:58.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb229161640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240b5d10 con 0x7fb2240b47b0 2023-11-24T08:11:58.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb229161640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2240b4c70 con 0x7fb2240ce690 2023-11-24T08:11:58.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb229161640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2240b4f50 con 0x7fb224144e80 2023-11-24T08:11:58.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb222d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb224144e80 0x7fb224070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:58.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb22259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb2240b47b0 0x7fb2240cdf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49024/0 (socket says 172.21.15.106:49024) 2023-11-24T08:11:58.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb222d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb224144e80 0x7fb224070a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45860/0 (socket says 172.21.15.106:45860) 2023-11-24T08:11:58.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.223+0000 7fb22259c640 1 -- 172.21.15.106:0/3902850840 learned_addr learned my addr 172.21.15.106:0/3902850840 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:11:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.224+0000 7fb222d9d640 1 -- 172.21.15.106:0/3902850840 >> v1:172.21.15.156:6789/0 conn(0x7fb2240ce690 legacy=0x7fb2240d00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.224+0000 7fb222d9d640 1 -- 172.21.15.106:0/3902850840 >> v1:172.21.15.106:6789/0 conn(0x7fb2240b47b0 legacy=0x7fb2240cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.224+0000 7fb221d9b640 1 -- 172.21.15.106:0/3902850840 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3232255654 0 0) 0x7fb2240b5d10 con 0x7fb2240b47b0 2023-11-24T08:11:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.224+0000 7fb222d9d640 1 -- 172.21.15.106:0/3902850840 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2240b5230 con 0x7fb224144e80 2023-11-24T08:11:58.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.225+0000 7fb222d9d640 1 --2- 172.21.15.106:0/3902850840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb224144e80 0x7fb224070a80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb214009d90 tx=0x7fb21402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca788cfc370bda4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.225+0000 7fb221d9b640 1 -- 172.21.15.106:0/3902850840 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21402cc60 con 0x7fb224144e80 2023-11-24T08:11:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.225+0000 7fb221d9b640 1 -- 172.21.15.106:0/3902850840 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb21402cdc0 con 0x7fb224144e80 2023-11-24T08:11:58.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.225+0000 7fb221d9b640 1 -- 172.21.15.106:0/3902850840 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb214034660 con 0x7fb224144e80 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 -- 172.21.15.106:0/3902850840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb224144e80 msgr2=0x7fb224070a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 --2- 172.21.15.106:0/3902850840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb224144e80 0x7fb224070a80 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb214009d90 tx=0x7fb21402beb0 comp rx=0 tx=0).stop 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 -- 172.21.15.106:0/3902850840 shutdown_connections 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 --2- 172.21.15.106:0/3902850840 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb224144e80 0x7fb224070a80 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 -- 172.21.15.106:0/3902850840 >> 172.21.15.106:0/3902850840 conn(0x7fb22406e8a0 msgr2=0x7fb22406ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 -- 172.21.15.106:0/3902850840 shutdown_connections 2023-11-24T08:11:58.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.226+0000 7fb229161640 1 -- 172.21.15.106:0/3902850840 wait complete. 2023-11-24T08:11:58.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.228+0000 7fb229161640 1 Processor -- start 2023-11-24T08:11:58.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.228+0000 7fb229161640 1 -- start start 2023-11-24T08:11:58.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb229161640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2240b47b0 0x7fb2240c9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:58.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb229161640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 0x7fb224140e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:58.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb229161640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb224144e80 0x7fb22415b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:58.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb229161640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224159bc0 con 0x7fb2240ce690 2023-11-24T08:11:58.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb229161640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb224141360 con 0x7fb2240b47b0 2023-11-24T08:11:58.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb229161640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2241592b0 con 0x7fb224144e80 2023-11-24T08:11:58.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb22259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 0x7fb224140e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:58.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb22259c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 0x7fb224140e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45864/0 (socket says 172.21.15.106:45864) 2023-11-24T08:11:58.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb22259c640 1 -- 172.21.15.106:0/746617628 learned_addr learned my addr 172.21.15.106:0/746617628 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:58.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb222d9d640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2240b47b0 0x7fb2240c9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:58.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb22259c640 1 -- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb224144e80 msgr2=0x7fb22415b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb22259c640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb224144e80 0x7fb22415b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.229+0000 7fb22259c640 1 -- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2240b47b0 msgr2=0x7fb2240c9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb22259c640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2240b47b0 0x7fb2240c9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb22259c640 1 -- 172.21.15.106:0/746617628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb214009a70 con 0x7fb2240ce690 2023-11-24T08:11:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb22259c640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 0x7fb224140e20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c430 tx=0x7fb20c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:58.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c0074e0 con 0x7fb2240ce690 2023-11-24T08:11:58.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb20c007660 con 0x7fb2240ce690 2023-11-24T08:11:58.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb20c004a20 con 0x7fb2240ce690 2023-11-24T08:11:58.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2241f5410 con 0x7fb2240ce690 2023-11-24T08:11:58.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.230+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb224159850 con 0x7fb2240ce690 2023-11-24T08:11:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.231+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb224067470 con 0x7fb2240ce690 2023-11-24T08:11:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.232+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb20c004b80 con 0x7fb2240ce690 2023-11-24T08:11:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.233+0000 7fb21b7fe640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb20406b230 0x7fb20406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.234+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb20c089a70 con 0x7fb2240ce690 2023-11-24T08:11:58.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.234+0000 7fb222d9d640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb20406b230 0x7fb20406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:58.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.234+0000 7fb222d9d640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb20406b230 0x7fb20406d6f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb214036000 tx=0x7fb214030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:58.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.238+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb20c054640 con 0x7fb2240ce690 2023-11-24T08:11:58.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:58.434+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2240d2260 con 0x7fb2240ce690 2023-11-24T08:11:59.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.423+0000 7fb21b7fe640 1 -- 172.21.15.106:0/746617628 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v66) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb20c057c60 con 0x7fb2240ce690 2023-11-24T08:11:59.425 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:11:59.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.426+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb20406b230 msgr2=0x7fb20406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.426+0000 7fb229161640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb20406b230 0x7fb20406d6f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb214036000 tx=0x7fb214030000 comp rx=0 tx=0).stop 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.426+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 msgr2=0x7fb224140e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.426+0000 7fb229161640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 0x7fb224140e20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb20c00c430 tx=0x7fb20c00c8f0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 shutdown_connections 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb20406b230 0x7fb20406d6f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb224144e80 0x7fb22415b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2240ce690 0x7fb224140e20 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 --2- 172.21.15.106:0/746617628 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2240b47b0 0x7fb2240c9150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 >> 172.21.15.106:0/746617628 conn(0x7fb22406e8a0 msgr2=0x7fb2241440d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 shutdown_connections 2023-11-24T08:11:59.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.427+0000 7fb229161640 1 -- 172.21.15.106:0/746617628 wait complete. 2023-11-24T08:11:59.451 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:11:59.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.662+0000 7ff0a3236640 1 Processor -- start 2023-11-24T08:11:59.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.663+0000 7ff0a3236640 1 -- start start 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff0a3236640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c15d740 0x7ff09c0d1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff0a3236640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff09c0cf080 0x7ff09c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff0a3236640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff09c0d0ad0 0x7ff09c0d1f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff0a3236640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0cec60 con 0x7ff09c0cf080 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff0a3236640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09c0ce2c0 con 0x7ff09c0d0ad0 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff0a3236640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0ce5a0 con 0x7ff09c15d740 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.664+0000 7ff09bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff09c0cf080 0x7ff09c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff09c0cf080 0x7ff09c0d19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45874/0 (socket says 172.21.15.106:45874) 2023-11-24T08:11:59.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 -- 172.21.15.106:0/3005515803 learned_addr learned my addr 172.21.15.106:0/3005515803 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:59.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 -- 172.21.15.106:0/3005515803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c15d740 msgr2=0x7ff09c0d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:59.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 --2- 172.21.15.106:0/3005515803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c15d740 0x7ff09c0d1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 -- 172.21.15.106:0/3005515803 >> v2:172.21.15.156:3300/0 conn(0x7ff09c0d0ad0 msgr2=0x7ff09c0d1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:11:59.667 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 --2- 172.21.15.106:0/3005515803 >> v2:172.21.15.156:3300/0 conn(0x7ff09c0d0ad0 0x7ff09c0d1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.665+0000 7ff09bfff640 1 -- 172.21.15.106:0/3005515803 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09c0ce880 con 0x7ff09c0cf080 2023-11-24T08:11:59.668 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.666+0000 7ff09bfff640 1 --2- 172.21.15.106:0/3005515803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c0cf080 0x7ff09c0d19f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0d33d0 tx=0x7ff08402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3df005a7ba7e3f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.666+0000 7ff09b7fe640 1 -- 172.21.15.106:0/3005515803 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084031880 con 0x7ff09c0cf080 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.666+0000 7ff09b7fe640 1 -- 172.21.15.106:0/3005515803 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0840319e0 con 0x7ff09c0cf080 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.666+0000 7ff09b7fe640 1 -- 172.21.15.106:0/3005515803 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff084031d70 con 0x7ff09c0cf080 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 -- 172.21.15.106:0/3005515803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c0cf080 msgr2=0x7ff09c0d19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 --2- 172.21.15.106:0/3005515803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c0cf080 0x7ff09c0d19f0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7ff09c0d33d0 tx=0x7ff08402b920 comp rx=0 tx=0).stop 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 -- 172.21.15.106:0/3005515803 shutdown_connections 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 --2- 172.21.15.106:0/3005515803 >> v2:172.21.15.156:3300/0 conn(0x7ff09c0d0ad0 0x7ff09c0d1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 --2- 172.21.15.106:0/3005515803 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c0cf080 0x7ff09c0d19f0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 --2- 172.21.15.106:0/3005515803 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c15d740 0x7ff09c0d1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 -- 172.21.15.106:0/3005515803 >> 172.21.15.106:0/3005515803 conn(0x7ff09c08af60 msgr2=0x7ff09c0d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:11:59.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 -- 172.21.15.106:0/3005515803 shutdown_connections 2023-11-24T08:11:59.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.667+0000 7ff0a3236640 1 -- 172.21.15.106:0/3005515803 wait complete. 2023-11-24T08:11:59.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.669+0000 7ff0a3236640 1 Processor -- start 2023-11-24T08:11:59.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.669+0000 7ff0a3236640 1 -- start start 2023-11-24T08:11:59.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a3236640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c0cf080 0x7ff09c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a3236640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09c0d0ad0 0x7ff09c1602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a3236640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 0x7ff09c0b6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a3236640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0b7090 con 0x7ff09c15d740 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a3236640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c1549d0 con 0x7ff09c0d0ad0 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a3236640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c0b6ad0 con 0x7ff09c0cf080 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 0x7ff09c0b6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 0x7ff09c0b6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45886/0 (socket says 172.21.15.106:45886) 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 -- 172.21.15.106:0/1381339736 learned_addr learned my addr 172.21.15.106:0/1381339736 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a0fd2640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c0cf080 0x7ff09c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff09bfff640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09c0d0ad0 0x7ff09c1602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 -- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c0cf080 msgr2=0x7ff09c154490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:59.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c0cf080 0x7ff09c154490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 -- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09c0d0ad0 msgr2=0x7ff09c1602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09c0d0ad0 0x7ff09c1602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a17d3640 1 -- 172.21.15.106:0/1381339736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0840094d0 con 0x7ff09c15d740 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff0a0fd2640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c0cf080 0x7ff09c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.670+0000 7ff09bfff640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09c0d0ad0 0x7ff09c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.671+0000 7ff0a17d3640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 0x7ff09c0b6010 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff094009fd0 tx=0x7ff09400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.671+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094011830 con 0x7ff09c15d740 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.671+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff094011990 con 0x7ff09c15d740 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.671+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c0b6db0 con 0x7ff09c15d740 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.671+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff094011d20 con 0x7ff09c15d740 2023-11-24T08:11:59.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.671+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c15fdc0 con 0x7ff09c15d740 2023-11-24T08:11:59.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.672+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c15ffb0 con 0x7ff09c15d740 2023-11-24T08:11:59.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.673+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7ff09401b460 con 0x7ff09c15d740 2023-11-24T08:11:59.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.674+0000 7ff0997fa640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff07c06b230 0x7ff07c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:11:59.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.674+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 5357+0+0 (secure 0 0 0) 0x7ff0940899d0 con 0x7ff09c15d740 2023-11-24T08:11:59.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.674+0000 7ff0a0fd2640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff07c06b230 0x7ff07c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:11:59.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.675+0000 7ff0a0fd2640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff07c06b230 0x7ff07c06d6f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff08c007eb0 tx=0x7ff08c0073c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:11:59.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.679+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff0940580e0 con 0x7ff09c15d740 2023-11-24T08:11:59.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:11:59.868+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff09c067470 con 0x7ff09c15d740 2023-11-24T08:12:00.436 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.434+0000 7ff0997fa640 1 -- 172.21.15.106:0/1381339736 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v67) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff094057c70 con 0x7ff09c15d740 2023-11-24T08:12:00.436 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2023-11-24T08:12:00.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff07c06b230 msgr2=0x7ff07c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff07c06b230 0x7ff07c06d6f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff08c007eb0 tx=0x7ff08c0073c0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 msgr2=0x7ff09c0b6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 0x7ff09c0b6010 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7ff094009fd0 tx=0x7ff09400b8e0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 shutdown_connections 2023-11-24T08:12:00.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff09c15d740 0x7ff09c0b6010 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff09c0d0ad0 0x7ff09c1602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff07c06b230 0x7ff07c06d6f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.439+0000 7ff0a3236640 1 --2- 172.21.15.106:0/1381339736 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff09c0cf080 0x7ff09c154490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.440+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 >> 172.21.15.106:0/1381339736 conn(0x7ff09c08af60 msgr2=0x7ff09c0d1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:00.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.440+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 shutdown_connections 2023-11-24T08:12:00.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.440+0000 7ff0a3236640 1 -- 172.21.15.106:0/1381339736 wait complete. 2023-11-24T08:12:00.466 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:12:00.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.674+0000 7f2423490640 1 Processor -- start 2023-11-24T08:12:00.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.675+0000 7f2423490640 1 -- start start 2023-11-24T08:12:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.676+0000 7f2423490640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f241c15b440 0x7f241c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.676+0000 7f2423490640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 0x7f241c0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.676+0000 7f2423490640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f241c15eea0 con 0x7f241c0c57e0 2023-11-24T08:12:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.676+0000 7f2423490640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0cc3d0 con 0x7f241c15b440 2023-11-24T08:12:00.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.676+0000 7f2423490640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c15b820 con 0x7f241c0cbf20 2023-11-24T08:12:00.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.677+0000 7f2421a2d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f241c0c57e0 0x7f241c0c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49026/0 (socket says 172.21.15.106:49026) 2023-11-24T08:12:00.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.677+0000 7f2421a2d640 1 -- 172.21.15.106:0/218346241 learned_addr learned my addr 172.21.15.106:0/218346241 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:00.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.677+0000 7f242122c640 1 --2- 172.21.15.106:0/218346241 >> v2:172.21.15.156:3300/0 conn(0x7f241c15b440 0x7f241c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.677+0000 7f2420a2b640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 0x7f241c0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.677+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3078518874 0 0) 0x7f241c15eea0 con 0x7f241c0c57e0 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.677+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2408003520 con 0x7f241c0c57e0 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1317593240 0 0) 0x7f2408003520 con 0x7f241c0c57e0 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 msgr2=0x7f241c0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 0x7f241c0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 >> v2:172.21.15.156:3300/0 conn(0x7f241c15b440 msgr2=0x7f241c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 --2- 172.21.15.106:0/218346241 >> v2:172.21.15.156:3300/0 conn(0x7f241c15b440 0x7f241c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c06d600 con 0x7f241c0c57e0 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f24140019f0 con 0x7f241c0c57e0 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 >> v1:172.21.15.106:6789/0 conn(0x7f241c0c57e0 legacy=0x7f241c0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 0x7f2408003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2408003f40 0x7f24080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2408004d00 0x7f2408005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0cc3d0 con 0x7f2408003f40 2023-11-24T08:12:00.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c15b820 con 0x7f2408004d00 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f240800ba30 con 0x7f241c0cbf20 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2420a2b640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2408003f40 0x7f24080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2420a2b640 1 -- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 msgr2=0x7f2408003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2420a2b640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f241c0cbf20 0x7f2408003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2420a2b640 1 -- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2408004d00 msgr2=0x7f2408005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2420a2b640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2408004d00 0x7f2408005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.678+0000 7f2420a2b640 1 -- 172.21.15.106:0/218346241 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2408003520 con 0x7f2408003f40 2023-11-24T08:12:00.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2420a2b640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2408003f40 0x7f24080047c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f240c009ac0 tx=0x7f24080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae031349dfcc5f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:00.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c02c710 con 0x7f2408003f40 2023-11-24T08:12:00.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c02c870 con 0x7f2408003f40 2023-11-24T08:12:00.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2413fff640 1 -- 172.21.15.106:0/218346241 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c02cc00 con 0x7f2408003f40 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2423490640 1 -- 172.21.15.106:0/218346241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2408003f40 msgr2=0x7f24080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2423490640 1 --2- 172.21.15.106:0/218346241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2408003f40 0x7f24080047c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f240c009ac0 tx=0x7f24080057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.680+0000 7f2421a2d640 1 -- 172.21.15.106:0/218346241 reap_dead start 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.681+0000 7f2423490640 1 -- 172.21.15.106:0/218346241 shutdown_connections 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.681+0000 7f2423490640 1 -- 172.21.15.106:0/218346241 >> 172.21.15.106:0/218346241 conn(0x7f241c0b0c50 msgr2=0x7f241c0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.681+0000 7f2423490640 1 -- 172.21.15.106:0/218346241 shutdown_connections 2023-11-24T08:12:00.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.681+0000 7f2423490640 1 -- 172.21.15.106:0/218346241 wait complete. 2023-11-24T08:12:00.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.683+0000 7f2423490640 1 Processor -- start 2023-11-24T08:12:00.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.683+0000 7f2423490640 1 -- start start 2023-11-24T08:12:00.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2423490640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2408003f40 0x7f241c151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2423490640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 0x7f241c0c8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2423490640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f241c0c8b40 0x7f241c0b3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2423490640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0c8fc0 con 0x7f241c1f3a00 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2423490640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c152450 con 0x7f241c0c8b40 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2423490640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0b1e30 con 0x7f2408003f40 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 0x7f241c0c8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 0x7f241c0c8600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45896/0 (socket says 172.21.15.106:45896) 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 -- 172.21.15.106:0/2630294142 learned_addr learned my addr 172.21.15.106:0/2630294142 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f242122c640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2408003f40 0x7f241c151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2421a2d640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f241c0c8b40 0x7f241c0b3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 -- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2408003f40 msgr2=0x7f241c151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2408003f40 0x7f241c151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 -- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f241c0c8b40 msgr2=0x7f241c0b3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f241c0c8b40 0x7f241c0b3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2420a2b640 1 -- 172.21.15.106:0/2630294142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c0b2110 con 0x7f241c1f3a00 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f242122c640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2408003f40 0x7f241c151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:00.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.684+0000 7f2421a2d640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f241c0c8b40 0x7f241c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.685+0000 7f2420a2b640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 0x7f241c0c8600 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f240c02c2b0 tx=0x7f240c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.685+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c003810 con 0x7f241c1f3a00 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.685+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f240c02de90 con 0x7f241c1f3a00 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.685+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c0b23f0 con 0x7f241c1f3a00 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.685+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f240c003b90 con 0x7f241c1f3a00 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.685+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c0b29b0 con 0x7f241c1f3a00 2023-11-24T08:12:00.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.686+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c0c8f20 con 0x7f241c1f3a00 2023-11-24T08:12:00.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.687+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f240c003cf0 con 0x7f241c1f3a00 2023-11-24T08:12:00.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.688+0000 7f2411ffb640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240006b260 0x7f240006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:00.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.688+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f240c0ab970 con 0x7f241c1f3a00 2023-11-24T08:12:00.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.689+0000 7f242122c640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240006b260 0x7f240006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:00.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.689+0000 7f242122c640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240006b260 0x7f240006d720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f241c0cde50 tx=0x7f2418008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:00.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.693+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f240c0768d0 con 0x7f241c1f3a00 2023-11-24T08:12:00.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.883+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f241c04e8b0 con 0x7f241c1f3a00 2023-11-24T08:12:00.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.889+0000 7f2411ffb640 1 -- 172.21.15.106:0/2630294142 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f240c079ef0 con 0x7f241c1f3a00 2023-11-24T08:12:00.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240006b260 msgr2=0x7f240006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240006b260 0x7f240006d720 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f241c0cde50 tx=0x7f2418008040 comp rx=0 tx=0).stop 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 msgr2=0x7f241c0c8600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 0x7f241c0c8600 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f240c02c2b0 tx=0x7f240c004730 comp rx=0 tx=0).stop 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 shutdown_connections 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f241c0c8b40 0x7f241c0b3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f241c1f3a00 0x7f241c0c8600 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f240006b260 0x7f240006d720 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 --2- 172.21.15.106:0/2630294142 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2408003f40 0x7f241c151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 >> 172.21.15.106:0/2630294142 conn(0x7f241c0b0c50 msgr2=0x7f241c0cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 shutdown_connections 2023-11-24T08:12:00.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:00.892+0000 7f2423490640 1 -- 172.21.15.106:0/2630294142 wait complete. 2023-11-24T08:12:00.912 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm' 2023-11-24T08:12:01.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.128+0000 7f47053ef640 1 Processor -- start 2023-11-24T08:12:01.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.128+0000 7f47053ef640 1 -- start start 2023-11-24T08:12:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f47053ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 0x7f47000d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f47053ef640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47000d0240 0x7f47000b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f47053ef640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47000ce990 con 0x7f47000d38e0 2023-11-24T08:12:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f47053ef640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47000d4110 con 0x7f47000d0240 2023-11-24T08:12:01.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f47053ef640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47000cec70 con 0x7f470015d4e0 2023-11-24T08:12:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f46fffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 0x7f47000d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f4704bee640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f47000d0240 0x7f47000b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.130+0000 7f46fffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 0x7f47000d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:58722/0 (socket says 172.21.15.106:58722) 2023-11-24T08:12:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.131+0000 7f46fffff640 1 -- 172.21.15.106:0/2715895325 learned_addr learned my addr 172.21.15.106:0/2715895325 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:01.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.131+0000 7f46fffff640 1 -- 172.21.15.106:0/2715895325 >> v2:172.21.15.156:3300/0 conn(0x7f47000d0240 msgr2=0x7f47000b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:01.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.131+0000 7f46fffff640 1 --2- 172.21.15.106:0/2715895325 >> v2:172.21.15.156:3300/0 conn(0x7f47000d0240 0x7f47000b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.131+0000 7f46fffff640 1 -- 172.21.15.106:0/2715895325 >> v1:172.21.15.156:6789/0 conn(0x7f47000d38e0 legacy=0x7f47000cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.131+0000 7f46fffff640 1 -- 172.21.15.106:0/2715895325 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47000cef50 con 0x7f470015d4e0 2023-11-24T08:12:01.134 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.133+0000 7f46fffff640 1 --2- 172.21.15.106:0/2715895325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 0x7f47000d1e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009870 tx=0x7f46f002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e77416614f22519 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:01.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.133+0000 7f46feffd640 1 -- 172.21.15.106:0/2715895325 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0030840 con 0x7f470015d4e0 2023-11-24T08:12:01.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f46feffd640 1 -- 172.21.15.106:0/2715895325 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f00309a0 con 0x7f470015d4e0 2023-11-24T08:12:01.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f46feffd640 1 -- 172.21.15.106:0/2715895325 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f0030d30 con 0x7f470015d4e0 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 -- 172.21.15.106:0/2715895325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 msgr2=0x7f47000d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 --2- 172.21.15.106:0/2715895325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 0x7f47000d1e20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f46f0009870 tx=0x7f46f002b980 comp rx=0 tx=0).stop 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 -- 172.21.15.106:0/2715895325 shutdown_connections 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 --2- 172.21.15.106:0/2715895325 >> v2:172.21.15.156:3300/0 conn(0x7f47000d0240 0x7f47000b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 --2- 172.21.15.106:0/2715895325 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f470015d4e0 0x7f47000d1e20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 -- 172.21.15.106:0/2715895325 >> 172.21.15.106:0/2715895325 conn(0x7f470009c510 msgr2=0x7f47000d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:01.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 -- 172.21.15.106:0/2715895325 shutdown_connections 2023-11-24T08:12:01.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.134+0000 7f47053ef640 1 -- 172.21.15.106:0/2715895325 wait complete. 2023-11-24T08:12:01.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.136+0000 7f47053ef640 1 Processor -- start 2023-11-24T08:12:01.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.136+0000 7f47053ef640 1 -- start start 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.137+0000 7f47053ef640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47000d0240 0x7f47001606c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.137+0000 7f47053ef640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47000d38e0 0x7f47000b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.137+0000 7f47053ef640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 0x7f4700158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.137+0000 7f47053ef640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4700160230 con 0x7f470015d4e0 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.137+0000 7f47053ef640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470015f620 con 0x7f47000d0240 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.137+0000 7f47053ef640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470015f900 con 0x7f47000d38e0 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 0x7f4700158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 0x7f4700158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45898/0 (socket says 172.21.15.106:45898) 2023-11-24T08:12:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 -- 172.21.15.106:0/1160577205 learned_addr learned my addr 172.21.15.106:0/1160577205 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:01.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f46fffff640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47000d0240 0x7f47001606c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f46ff7fe640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47000d38e0 0x7f47000b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 -- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47000d38e0 msgr2=0x7f47000b5df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47000d38e0 0x7f47000b5df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 -- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47000d0240 msgr2=0x7f47001606c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.141 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47000d0240 0x7f47001606c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 -- 172.21.15.106:0/1160577205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46f00094d0 con 0x7f470015d4e0 2023-11-24T08:12:01.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.138+0000 7f4704bee640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 0x7f4700158de0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f46f800efc0 tx=0x7f46f800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.139+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8014480 con 0x7f470015d4e0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.139+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46f800f040 con 0x7f470015d4e0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.139+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46f8014820 con 0x7f470015d4e0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.139+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f470015fbe0 con 0x7f470015d4e0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.139+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f470014e690 con 0x7f470015d4e0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.140+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f470004e840 con 0x7f470015d4e0 2023-11-24T08:12:01.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.141+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f46f8014980 con 0x7f470015d4e0 2023-11-24T08:12:01.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.143+0000 7f46fcff9640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46ec06b280 0x7f46ec06d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.143+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f46f808a9d0 con 0x7f470015d4e0 2023-11-24T08:12:01.144 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.143+0000 7f46fffff640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46ec06b280 0x7f46ec06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.144+0000 7f46fffff640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46ec06b280 0x7f46ec06d740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f46f0036000 tx=0x7f46f003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:01.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.147+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46f8055760 con 0x7f470015d4e0 2023-11-24T08:12:01.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.408+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]} v 0) v1 -- 0x7f470006caa0 con 0x7f470015d4e0 2023-11-24T08:12:01.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.437+0000 7f46fcff9640 1 -- 172.21.15.106:0/1160577205 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_nonexistent_subvolume_snapshot_rm"]}]=0 v198) v1 ==== 166+0+0 (secure 0 0 0) 0x7f46f8058d80 con 0x7f470015d4e0 2023-11-24T08:12:01.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46ec06b280 msgr2=0x7f46ec06d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46ec06b280 0x7f46ec06d740 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f46f0036000 tx=0x7f46f003b040 comp rx=0 tx=0).stop 2023-11-24T08:12:01.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 msgr2=0x7f4700158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 0x7f4700158de0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f46f800efc0 tx=0x7f46f800c490 comp rx=0 tx=0).stop 2023-11-24T08:12:01.442 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 shutdown_connections 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f470015d4e0 0x7f4700158de0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f46ec06b280 0x7f46ec06d740 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f47000d38e0 0x7f47000b5df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 --2- 172.21.15.106:0/1160577205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f47000d0240 0x7f47001606c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 >> 172.21.15.106:0/1160577205 conn(0x7f470009c510 msgr2=0x7f47000b3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.440+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 shutdown_connections 2023-11-24T08:12:01.443 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.441+0000 7f47053ef640 1 -- 172.21.15.106:0/1160577205 wait complete. 2023-11-24T08:12:01.460 INFO:tasks.cephfs_test_runner:test_nonexistent_subvolume_snapshot_rm (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ok 2023-11-24T08:12:01.460 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) tests the scenario where 'fs subvolume snapshot info' command 2023-11-24T08:12:01.460 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_info (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:12:01.460 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:01.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.672+0000 7f7a2f59e640 1 Processor -- start 2023-11-24T08:12:01.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.673+0000 7f7a2f59e640 1 -- start start 2023-11-24T08:12:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.674+0000 7f7a2f59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7a3013d120 0x7f7a3015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.674+0000 7f7a2f59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7a3006d6c0 0x7f7a3006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.674+0000 7f7a2f59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a300b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.674+0000 7f7a2f59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3006e060 con 0x7f7a3006d6c0 2023-11-24T08:12:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a2f59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a3013f9e0 con 0x7f7a3013d120 2023-11-24T08:12:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a2f59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3013fcc0 con 0x7f7a300b0a70 2023-11-24T08:12:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7a3006d6c0 0x7f7a3006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7a3006d6c0 0x7f7a3006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45914/0 (socket says 172.21.15.106:45914) 2023-11-24T08:12:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 -- 172.21.15.106:0/3957351694 learned_addr learned my addr 172.21.15.106:0/3957351694 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a2e59c640 1 --2- 172.21.15.106:0/3957351694 >> v2:172.21.15.156:3300/0 conn(0x7f7a3013d120 0x7f7a3015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 -- 172.21.15.106:0/3957351694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 msgr2=0x7f7a300b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 --2- 172.21.15.106:0/3957351694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a300b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 -- 172.21.15.106:0/3957351694 >> v2:172.21.15.156:3300/0 conn(0x7f7a3013d120 msgr2=0x7f7a3015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 --2- 172.21.15.106:0/3957351694 >> v2:172.21.15.156:3300/0 conn(0x7f7a3013d120 0x7f7a3015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a27fff640 1 -- 172.21.15.106:0/3957351694 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3013ffa0 con 0x7f7a3006d6c0 2023-11-24T08:12:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.675+0000 7f7a2e59c640 1 --2- 172.21.15.106:0/3957351694 >> v2:172.21.15.156:3300/0 conn(0x7f7a3013d120 0x7f7a3015af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:01.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.676+0000 7f7a27fff640 1 --2- 172.21.15.106:0/3957351694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3006d6c0 0x7f7a3006db20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7a300b1b80 tx=0x7f7a1802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=214a075301263c70 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:01.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2dd9b640 1 -- 172.21.15.106:0/3957351694 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a180317d0 con 0x7f7a3006d6c0 2023-11-24T08:12:01.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2dd9b640 1 -- 172.21.15.106:0/3957351694 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a18031930 con 0x7f7a3006d6c0 2023-11-24T08:12:01.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2dd9b640 1 -- 172.21.15.106:0/3957351694 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a18031cc0 con 0x7f7a3006d6c0 2023-11-24T08:12:01.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3957351694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3006d6c0 msgr2=0x7f7a3006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3957351694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3006d6c0 0x7f7a3006db20 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7a300b1b80 tx=0x7f7a1802b870 comp rx=0 tx=0).stop 2023-11-24T08:12:01.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3957351694 shutdown_connections 2023-11-24T08:12:01.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3957351694 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a300b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3957351694 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3006d6c0 0x7f7a3006db20 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3957351694 >> v2:172.21.15.156:3300/0 conn(0x7f7a3013d120 0x7f7a3015af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3957351694 >> 172.21.15.106:0/3957351694 conn(0x7f7a300cb830 msgr2=0x7f7a300cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3957351694 shutdown_connections 2023-11-24T08:12:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.677+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3957351694 wait complete. 2023-11-24T08:12:01.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.680+0000 7f7a2f59e640 1 Processor -- start 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.680+0000 7f7a2f59e640 1 -- start start 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2f59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a3006d6c0 0x7f7a301397d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2f59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a301520d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 0x7f7a30159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2f59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a301539a0 con 0x7f7a3013d120 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2f59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30139d10 con 0x7f7a3006d6c0 2023-11-24T08:12:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2f59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a30153510 con 0x7f7a300b0a70 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 0x7f7a30159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2ed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 0x7f7a30159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45928/0 (socket says 172.21.15.106:45928) 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.681+0000 7f7a2ed9d640 1 -- 172.21.15.106:0/3782812319 learned_addr learned my addr 172.21.15.106:0/3782812319 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a27fff640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a301520d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2e59c640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a3006d6c0 0x7f7a301397d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2ed9d640 1 -- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 msgr2=0x7f7a301520d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2ed9d640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a301520d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2ed9d640 1 -- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a3006d6c0 msgr2=0x7f7a301397d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2ed9d640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a3006d6c0 0x7f7a301397d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2ed9d640 1 -- 172.21.15.106:0/3782812319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a180094d0 con 0x7f7a3013d120 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2e59c640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a3006d6c0 0x7f7a301397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2ed9d640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 0x7f7a30159210 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b9a0 tx=0x7f7a2800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2800cae0 con 0x7f7a3013d120 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2800cc60 con 0x7f7a3013d120 2023-11-24T08:12:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28014440 con 0x7f7a3013d120 2023-11-24T08:12:01.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.682+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a30152900 con 0x7f7a3013d120 2023-11-24T08:12:01.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.683+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a30152ec0 con 0x7f7a3013d120 2023-11-24T08:12:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.684+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3004e840 con 0x7f7a3013d120 2023-11-24T08:12:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.685+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f7a2801d3b0 con 0x7f7a3013d120 2023-11-24T08:12:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.685+0000 7f7a26ffd640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7a1006b230 0x7f7a1006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.686+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a280899f0 con 0x7f7a3013d120 2023-11-24T08:12:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.686+0000 7f7a2e59c640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7a1006b230 0x7f7a1006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.686+0000 7f7a2e59c640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7a1006b230 0x7f7a1006d6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006fd0 tx=0x7f7a20008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:01.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.690+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7a280546f0 con 0x7f7a3013d120 2023-11-24T08:12:01.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.881+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7a30153650 con 0x7f7a3013d120 2023-11-24T08:12:01.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.884+0000 7f7a26ffd640 1 -- 172.21.15.106:0/3782812319 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+13034 (secure 0 0 0) 0x7f7a28019070 con 0x7f7a3013d120 2023-11-24T08:12:01.886 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:01.886 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":67,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:00.424212+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/180703350":"2023-11-25T08:11:56.979344+0000","172.21.15.106:6836/180703350":"2023-11-25T08:11:56.979344+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:01.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7a1006b230 msgr2=0x7f7a1006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7a1006b230 0x7f7a1006d6f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7a20006fd0 tx=0x7f7a20008040 comp rx=0 tx=0).stop 2023-11-24T08:12:01.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 msgr2=0x7f7a30159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:01.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 0x7f7a30159210 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7a2800b9a0 tx=0x7f7a2800be60 comp rx=0 tx=0).stop 2023-11-24T08:12:01.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 shutdown_connections 2023-11-24T08:12:01.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7a3013d120 0x7f7a30159210 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7a1006b230 0x7f7a1006d6f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7a300b0a70 0x7f7a301520d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 --2- 172.21.15.106:0/3782812319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7a3006d6c0 0x7f7a301397d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:01.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.886+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 >> 172.21.15.106:0/3782812319 conn(0x7f7a300cb830 msgr2=0x7f7a300aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:01.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.887+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 shutdown_connections 2023-11-24T08:12:01.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:01.887+0000 7f7a2f59e640 1 -- 172.21.15.106:0/3782812319 wait complete. 2023-11-24T08:12:01.908 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:01.908 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:02.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.122+0000 7fc0fecd1640 1 Processor -- start 2023-11-24T08:12:02.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.122+0000 7fc0fecd1640 1 -- start start 2023-11-24T08:12:02.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fecd1640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc0f8157850 0x7fc0f80b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fecd1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc0f8157c30 0x7fc0f8072320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fecd1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f815e4a0 0x7fc0f806a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fecd1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f815dad0 con 0x7fc0f8157c30 2023-11-24T08:12:02.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fecd1640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f8072860 con 0x7fc0f8157850 2023-11-24T08:12:02.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fecd1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f815dce0 con 0x7fc0f815e4a0 2023-11-24T08:12:02.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fd4ce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc0f8157c30 0x7fc0f8072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fd4ce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc0f8157c30 0x7fc0f8072320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45932/0 (socket says 172.21.15.106:45932) 2023-11-24T08:12:02.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.124+0000 7fc0fd4ce640 1 -- 172.21.15.106:0/4002875692 learned_addr learned my addr 172.21.15.106:0/4002875692 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:02.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fdccf640 1 --2- 172.21.15.106:0/4002875692 >> v2:172.21.15.156:3300/0 conn(0x7fc0f8157850 0x7fc0f80b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fe4d0640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f815e4a0 0x7fc0f806a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fd4ce640 1 -- 172.21.15.106:0/4002875692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f815e4a0 msgr2=0x7fc0f806a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:02.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fd4ce640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f815e4a0 0x7fc0f806a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fd4ce640 1 -- 172.21.15.106:0/4002875692 >> v2:172.21.15.156:3300/0 conn(0x7fc0f8157850 msgr2=0x7fc0f80b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:02.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fd4ce640 1 --2- 172.21.15.106:0/4002875692 >> v2:172.21.15.156:3300/0 conn(0x7fc0f8157850 0x7fc0f80b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fd4ce640 1 -- 172.21.15.106:0/4002875692 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f80b3b70 con 0x7fc0f8157c30 2023-11-24T08:12:02.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fdccf640 1 --2- 172.21.15.106:0/4002875692 >> v2:172.21.15.156:3300/0 conn(0x7fc0f8157850 0x7fc0f80b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:02.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.125+0000 7fc0fe4d0640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f815e4a0 0x7fc0f806a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:02.130 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.126+0000 7fc0fd4ce640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157c30 0x7fc0f8072320 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8061b80 tx=0x7fc0e802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5db96b66f9bbbfc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:02.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.126+0000 7fc0fcccd640 1 -- 172.21.15.106:0/4002875692 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8031880 con 0x7fc0f8157c30 2023-11-24T08:12:02.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.126+0000 7fc0fcccd640 1 -- 172.21.15.106:0/4002875692 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e80319e0 con 0x7fc0f8157c30 2023-11-24T08:12:02.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.126+0000 7fc0fcccd640 1 -- 172.21.15.106:0/4002875692 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8031d70 con 0x7fc0f8157c30 2023-11-24T08:12:02.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 -- 172.21.15.106:0/4002875692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157c30 msgr2=0x7fc0f8072320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.131 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157c30 0x7fc0f8072320 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fc0f8061b80 tx=0x7fc0e802b920 comp rx=0 tx=0).stop 2023-11-24T08:12:02.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 -- 172.21.15.106:0/4002875692 shutdown_connections 2023-11-24T08:12:02.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f815e4a0 0x7fc0f806a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/4002875692 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157c30 0x7fc0f8072320 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/4002875692 >> v2:172.21.15.156:3300/0 conn(0x7fc0f8157850 0x7fc0f80b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 -- 172.21.15.106:0/4002875692 >> 172.21.15.106:0/4002875692 conn(0x7fc0f80b1bd0 msgr2=0x7fc0f80b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:02.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 -- 172.21.15.106:0/4002875692 shutdown_connections 2023-11-24T08:12:02.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.127+0000 7fc0fecd1640 1 -- 172.21.15.106:0/4002875692 wait complete. 2023-11-24T08:12:02.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.129+0000 7fc0fecd1640 1 Processor -- start 2023-11-24T08:12:02.135 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.129+0000 7fc0fecd1640 1 -- start start 2023-11-24T08:12:02.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fecd1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 0x7fc0f814ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fecd1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f8157c30 0x7fc0f8161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fecd1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f815e4a0 0x7fc0f80ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fecd1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f815f0b0 con 0x7fc0f8157850 2023-11-24T08:12:02.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fecd1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f815f390 con 0x7fc0f815e4a0 2023-11-24T08:12:02.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fecd1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f815f670 con 0x7fc0f8157c30 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fdccf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 0x7fc0f814ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fdccf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 0x7fc0f814ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45946/0 (socket says 172.21.15.106:45946) 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.130+0000 7fc0fdccf640 1 -- 172.21.15.106:0/3636786382 learned_addr learned my addr 172.21.15.106:0/3636786382 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fd4ce640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f8157c30 0x7fc0f8161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fe4d0640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f815e4a0 0x7fc0f80ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fdccf640 1 -- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f8157c30 msgr2=0x7fc0f8161260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fdccf640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f8157c30 0x7fc0f8161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fdccf640 1 -- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f815e4a0 msgr2=0x7fc0f80ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fdccf640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f815e4a0 0x7fc0f80ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fdccf640 1 -- 172.21.15.106:0/3636786382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e80094d0 con 0x7fc0f8157850 2023-11-24T08:12:02.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fdccf640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 0x7fc0f814ba30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c7f0 tx=0x7fc0f400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:02.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4007820 con 0x7fc0f8157850 2023-11-24T08:12:02.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0f4007980 con 0x7fc0f8157850 2023-11-24T08:12:02.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4007d30 con 0x7fc0f8157850 2023-11-24T08:12:02.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f815f950 con 0x7fc0f8157850 2023-11-24T08:12:02.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.131+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f81617a0 con 0x7fc0f8157850 2023-11-24T08:12:02.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.133+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f8161990 con 0x7fc0f8157850 2023-11-24T08:12:02.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.134+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc0f40113f0 con 0x7fc0f8157850 2023-11-24T08:12:02.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.134+0000 7fc0ee7fc640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc0dc06b230 0x7fc0dc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.135+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0f4089f60 con 0x7fc0f8157850 2023-11-24T08:12:02.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.135+0000 7fc0fd4ce640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc0dc06b230 0x7fc0dc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.135+0000 7fc0fd4ce640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc0dc06b230 0x7fc0dc06d6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8036000 tx=0x7fc0e803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:02.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.139+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc0f8161990 con 0x7fc0f8157850 2023-11-24T08:12:02.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.330+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc0f814c800 con 0x7fc0f8157850 2023-11-24T08:12:02.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.331+0000 7fc0ee7fc640 1 -- 172.21.15.106:0/3636786382 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v67) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc0f814c800 con 0x7fc0f8157850 2023-11-24T08:12:02.333 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:02.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc0dc06b230 msgr2=0x7fc0dc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc0dc06b230 0x7fc0dc06d6f0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8036000 tx=0x7fc0e803c040 comp rx=0 tx=0).stop 2023-11-24T08:12:02.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 msgr2=0x7fc0f814ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 0x7fc0f814ba30 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fc0f400c7f0 tx=0x7fc0f400ccb0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 shutdown_connections 2023-11-24T08:12:02.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0f815e4a0 0x7fc0f80ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc0dc06b230 0x7fc0dc06d6f0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc0f8157c30 0x7fc0f8161260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 --2- 172.21.15.106:0/3636786382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc0f8157850 0x7fc0f814ba30 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 >> 172.21.15.106:0/3636786382 conn(0x7fc0f80b1bd0 msgr2=0x7fc0f80af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:02.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 shutdown_connections 2023-11-24T08:12:02.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.334+0000 7fc0fecd1640 1 -- 172.21.15.106:0/3636786382 wait complete. 2023-11-24T08:12:02.353 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T08:12:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.554+0000 7fd4a3dfb640 1 Processor -- start 2023-11-24T08:12:02.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.555+0000 7fd4a3dfb640 1 -- start start 2023-11-24T08:12:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.556+0000 7fd4a3dfb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd49c0ce6c0 0x7fd49c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.556+0000 7fd4a3dfb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49c0cece0 con 0x7fd49c0cf450 2023-11-24T08:12:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.556+0000 7fd4a3dfb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49c06d040 con 0x7fd49c15c7f0 2023-11-24T08:12:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.556+0000 7fd4a3dfb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd49c0cfe40 con 0x7fd49c0ce6c0 2023-11-24T08:12:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.556+0000 7fd4a1396640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd49c0cf450 0x7fd49c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49028/0 (socket says 172.21.15.106:49028) 2023-11-24T08:12:02.558 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.556+0000 7fd4a1396640 1 -- 172.21.15.106:0/1868220186 learned_addr learned my addr 172.21.15.106:0/1868220186 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a2398640 1 --2- 172.21.15.106:0/1868220186 >> v2:172.21.15.156:3300/0 conn(0x7fd49c0ce6c0 0x7fd49c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 658081563 0 0) 0x7fd49c0cece0 con 0x7fd49c0cf450 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd488003520 con 0x7fd49c0cf450 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 21853570 0 0) 0x7fd488003520 con 0x7fd49c0cf450 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 >> v2:172.21.15.156:3300/0 conn(0x7fd49c0ce6c0 msgr2=0x7fd49c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 --2- 172.21.15.106:0/1868220186 >> v2:172.21.15.156:3300/0 conn(0x7fd49c0ce6c0 0x7fd49c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 >> v1:172.21.15.156:6789/0 conn(0x7fd49c15c7f0 legacy=0x7fd49c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:02.560 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c06d320 con 0x7fd49c0cf450 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fd48c001670 con 0x7fd49c0cf450 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 >> v1:172.21.15.106:6789/0 conn(0x7fd49c0cf450 legacy=0x7fd49c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd488003970 0x7fd488003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4880047c0 0x7fd488004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd488005180 0x7fd48800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0cfe40 con 0x7fd4880047c0 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48800c0b0 con 0x7fd488005180 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.557+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48800c270 con 0x7fd488003970 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1396640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4880047c0 0x7fd488004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1b97640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd488003970 0x7fd488003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1396640 1 -- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd488003970 msgr2=0x7fd488003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1396640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd488003970 0x7fd488003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1396640 1 -- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd488005180 msgr2=0x7fd48800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:02.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1396640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd488005180 0x7fd48800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1396640 1 -- 172.21.15.106:0/1868220186 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd488003520 con 0x7fd4880047c0 2023-11-24T08:12:02.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a2398640 1 -- 172.21.15.106:0/1868220186 reap_dead start 2023-11-24T08:12:02.562 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.558+0000 7fd4a1b97640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd488003970 0x7fd488003d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.559+0000 7fd4a1396640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4880047c0 0x7fd488004c40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd48c001eb0 tx=0x7fd4880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37072d8e146390ca server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c005ad0 con 0x7fd4880047c0 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c005c50 con 0x7fd4880047c0 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a0b95640 1 -- 172.21.15.106:0/1868220186 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c02cad0 con 0x7fd4880047c0 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/1868220186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4880047c0 msgr2=0x7fd488004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4880047c0 0x7fd488004c40 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fd48c001eb0 tx=0x7fd4880057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/1868220186 shutdown_connections 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/1868220186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd4880047c0 0x7fd488004c40 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/1868220186 >> 172.21.15.106:0/1868220186 conn(0x7fd49c06a920 msgr2=0x7fd49c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/1868220186 shutdown_connections 2023-11-24T08:12:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.560+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/1868220186 wait complete. 2023-11-24T08:12:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.562+0000 7fd4a3dfb640 1 Processor -- start 2023-11-24T08:12:02.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.562+0000 7fd4a3dfb640 1 -- start start 2023-11-24T08:12:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a3dfb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4880047c0 0x7fd49c151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a3dfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 0x7fd49c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a3dfb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd49c0b3130 0x7fd49c0b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a3dfb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0b3af0 con 0x7fd49c156cd0 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a3dfb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c0ac6b0 con 0x7fd4880047c0 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a3dfb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd49c1577c0 con 0x7fd49c0b3130 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.563+0000 7fd4a1396640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 0x7fd49c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 0x7fd49c157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45954/0 (socket says 172.21.15.106:45954) 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 -- 172.21.15.106:0/2137434278 learned_addr learned my addr 172.21.15.106:0/2137434278 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a2398640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd49c0b3130 0x7fd49c0b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1b97640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4880047c0 0x7fd49c151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 -- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd49c0b3130 msgr2=0x7fd49c0b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd49c0b3130 0x7fd49c0b35b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 -- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4880047c0 msgr2=0x7fd49c151460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:02.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4880047c0 0x7fd49c151460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:02.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 -- 172.21.15.106:0/2137434278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd49c0ac060 con 0x7fd49c156cd0 2023-11-24T08:12:02.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.564+0000 7fd4a1396640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 0x7fd49c157150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd48c001eb0 tx=0x7fd48c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:02.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.565+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c0049e0 con 0x7fd49c156cd0 2023-11-24T08:12:02.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.565+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c03a050 con 0x7fd49c156cd0 2023-11-24T08:12:02.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.565+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd49c0ac340 con 0x7fd49c156cd0 2023-11-24T08:12:02.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.565+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c004d60 con 0x7fd49c156cd0 2023-11-24T08:12:02.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.565+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd49c157aa0 con 0x7fd49c156cd0 2023-11-24T08:12:02.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.566+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd49c067470 con 0x7fd49c156cd0 2023-11-24T08:12:02.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.567+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd48c032070 con 0x7fd49c156cd0 2023-11-24T08:12:02.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.568+0000 7fd4927fc640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd48006b260 0x7fd48006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:02.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.568+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd48c0ab680 con 0x7fd49c156cd0 2023-11-24T08:12:02.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.568+0000 7fd4a1b97640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd48006b260 0x7fd48006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:02.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.569+0000 7fd4a1b97640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd48006b260 0x7fd48006d720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0aee90 tx=0x7fd498009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:02.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.574+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd48c076460 con 0x7fd49c156cd0 2023-11-24T08:12:02.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:02.834+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7fd49c0ac7f0 con 0x7fd49c156cd0 2023-11-24T08:12:03.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.442+0000 7fd4927fc640 1 -- 172.21.15.106:0/2137434278 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v200) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd48c079a80 con 0x7fd49c156cd0 2023-11-24T08:12:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.446+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd48006b260 msgr2=0x7fd48006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.446+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd48006b260 0x7fd48006d720 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0aee90 tx=0x7fd498009040 comp rx=0 tx=0).stop 2023-11-24T08:12:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.446+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 msgr2=0x7fd49c157150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.446+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 0x7fd49c157150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fd48c001eb0 tx=0x7fd48c001a00 comp rx=0 tx=0).stop 2023-11-24T08:12:03.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.446+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 shutdown_connections 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.446+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd48006b260 0x7fd48006d720 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.447+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd49c0b3130 0x7fd49c0b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.447+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd49c156cd0 0x7fd49c157150 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.447+0000 7fd4a3dfb640 1 --2- 172.21.15.106:0/2137434278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4880047c0 0x7fd49c151460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.447+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 >> 172.21.15.106:0/2137434278 conn(0x7fd49c06a920 msgr2=0x7fd49c071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.447+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 shutdown_connections 2023-11-24T08:12:03.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.447+0000 7fd4a3dfb640 1 -- 172.21.15.106:0/2137434278 wait complete. 2023-11-24T08:12:03.471 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:12:03.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.685+0000 7f5325789640 1 Processor -- start 2023-11-24T08:12:03.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.685+0000 7f5325789640 1 -- start start 2023-11-24T08:12:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.687+0000 7f5325789640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f53200c59b0 0x7f5320166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.687+0000 7f5325789640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53200c5dc0 con 0x7f532015b440 2023-11-24T08:12:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.687+0000 7f5325789640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f532015eea0 con 0x7f53200cc0f0 2023-11-24T08:12:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.687+0000 7f5325789640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f532015b820 con 0x7f53200c59b0 2023-11-24T08:12:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.687+0000 7f531effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f532015b440 0x7f53200cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:49044/0 (socket says 172.21.15.106:49044) 2023-11-24T08:12:03.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.687+0000 7f531effd640 1 -- 172.21.15.106:0/126646410 learned_addr learned my addr 172.21.15.106:0/126646410 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3040443230 0 0) 0x7f53200c5dc0 con 0x7f532015b440 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c003520 con 0x7f532015b440 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531f7fe640 1 --2- 172.21.15.106:0/126646410 >> v2:172.21.15.156:3300/0 conn(0x7f53200c59b0 0x7f5320166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4208977452 0 0) 0x7f530c003520 con 0x7f532015b440 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 >> v2:172.21.15.156:3300/0 conn(0x7f53200c59b0 msgr2=0x7f5320166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 --2- 172.21.15.106:0/126646410 >> v2:172.21.15.156:3300/0 conn(0x7f53200c59b0 0x7f5320166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 >> v1:172.21.15.156:6789/0 conn(0x7f53200cc0f0 legacy=0x7f53200c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532006d600 con 0x7f532015b440 2023-11-24T08:12:03.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f5310001670 con 0x7f532015b440 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 >> v1:172.21.15.106:6789/0 conn(0x7f532015b440 legacy=0x7f53200cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530c0039c0 0x7f530c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c005250 0x7f530c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532015b820 con 0x7f530c0039c0 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c1a0 con 0x7f530c0047c0 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.688+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c00c360 con 0x7f530c005250 2023-11-24T08:12:03.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531effd640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530c0039c0 0x7f530c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531effd640 1 -- 172.21.15.106:0/126646410 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c005250 msgr2=0x7f530c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531effd640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c005250 0x7f530c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531effd640 1 -- 172.21.15.106:0/126646410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f530c0047c0 msgr2=0x7f530c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531effd640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f530c0047c0 0x7f530c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531effd640 1 -- 172.21.15.106:0/126646410 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c003520 con 0x7f530c0039c0 2023-11-24T08:12:03.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.689+0000 7f531f7fe640 1 -- 172.21.15.106:0/126646410 reap_dead start 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.690+0000 7f531effd640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530c0039c0 0x7f530c003dc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f531000c0e0 tx=0x7f530c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c09269c29695847 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.690+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310005ad0 con 0x7f530c0039c0 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.690+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310005c30 con 0x7f530c0039c0 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.690+0000 7f531e7fc640 1 -- 172.21.15.106:0/126646410 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f531002c810 con 0x7f530c0039c0 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 -- 172.21.15.106:0/126646410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530c0039c0 msgr2=0x7f530c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530c0039c0 0x7f530c003dc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f531000c0e0 tx=0x7f530c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 -- 172.21.15.106:0/126646410 shutdown_connections 2023-11-24T08:12:03.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 --2- 172.21.15.106:0/126646410 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f530c0039c0 0x7f530c003dc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 -- 172.21.15.106:0/126646410 >> 172.21.15.106:0/126646410 conn(0x7f53200b0c50 msgr2=0x7f53200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:03.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 -- 172.21.15.106:0/126646410 shutdown_connections 2023-11-24T08:12:03.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.691+0000 7f5325789640 1 -- 172.21.15.106:0/126646410 wait complete. 2023-11-24T08:12:03.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.693+0000 7f5325789640 1 Processor -- start 2023-11-24T08:12:03.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.693+0000 7f5325789640 1 -- start start 2023-11-24T08:12:03.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5325789640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c0039c0 0x7f53201f6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5325789640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 0x7f532009afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5325789640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5320155940 0x7f5320155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5325789640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53201f72c0 con 0x7f53201f2d80 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5325789640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5320156300 con 0x7f5320155940 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5325789640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53200ccbd0 con 0x7f530c0039c0 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5317fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 0x7f532009afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5317fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 0x7f532009afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45958/0 (socket says 172.21.15.106:45958) 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.694+0000 7f5317fff640 1 -- 172.21.15.106:0/3982336132 learned_addr learned my addr 172.21.15.106:0/3982336132 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5317fff640 1 -- 172.21.15.106:0/3982336132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c0039c0 msgr2=0x7f53201f6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5317fff640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c0039c0 0x7f53201f6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5317fff640 1 -- 172.21.15.106:0/3982336132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5320155940 msgr2=0x7f5320155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5317fff640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5320155940 0x7f5320155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5317fff640 1 -- 172.21.15.106:0/3982336132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53201f32d0 con 0x7f53201f2d80 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5317fff640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 0x7f532009afd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f530800c3c0 tx=0x7f530800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308011790 con 0x7f53201f2d80 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53080118f0 con 0x7f53201f2d80 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5308011ce0 con 0x7f53201f2d80 2023-11-24T08:12:03.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53201f35b0 con 0x7f53201f2d80 2023-11-24T08:12:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.695+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53201f3b70 con 0x7f53201f2d80 2023-11-24T08:12:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.697+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53200ccdd0 con 0x7f53201f2d80 2023-11-24T08:12:03.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.697+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f530801c460 con 0x7f53201f2d80 2023-11-24T08:12:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.698+0000 7f53177fe640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f530006b230 0x7f530006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.698+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5308089a10 con 0x7f53201f2d80 2023-11-24T08:12:03.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.699+0000 7f531effd640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f530006b230 0x7f530006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:03.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.699+0000 7f531effd640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f530006b230 0x7f530006d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5310034000 tx=0x7f5310039040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:03.705 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.703+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f53200ccdd0 con 0x7f53201f2d80 2023-11-24T08:12:03.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.893+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f53200ccda0 con 0x7f53201f2d80 2023-11-24T08:12:03.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.899+0000 7f53177fe640 1 -- 172.21.15.106:0/3982336132 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f53080547a0 con 0x7f53201f2d80 2023-11-24T08:12:03.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f530006b230 msgr2=0x7f530006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f530006b230 0x7f530006d6f0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f5310034000 tx=0x7f5310039040 comp rx=0 tx=0).stop 2023-11-24T08:12:03.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 msgr2=0x7f532009afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:03.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 0x7f532009afd0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f530800c3c0 tx=0x7f530800c7c0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 shutdown_connections 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5320155940 0x7f5320155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f53201f2d80 0x7f532009afd0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f530006b230 0x7f530006d6f0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 --2- 172.21.15.106:0/3982336132 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f530c0039c0 0x7f53201f6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.902+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 >> 172.21.15.106:0/3982336132 conn(0x7f53200b0c50 msgr2=0x7f53200c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.903+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 shutdown_connections 2023-11-24T08:12:03.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:03.903+0000 7f5325789640 1 -- 172.21.15.106:0/3982336132 wait complete. 2023-11-24T08:12:03.924 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:12:03.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:12:04.039 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:12:04.162 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:12:04.162 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:04.163 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:04.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:12:04.222 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:04.222 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:04.284 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:12:04.285 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:04.285 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:04.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:12:04.346 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:04.346 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:04.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:12:04.406 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:12:04.406 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:04.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.629+0000 7f0868d4c640 1 Processor -- start 2023-11-24T08:12:04.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.629+0000 7f0868d4c640 1 -- start start 2023-11-24T08:12:04.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f0868d4c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f08640ce6c0 0x7f08640cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f0868d4c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08640cece0 con 0x7f086415c7f0 2023-11-24T08:12:04.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f0868d4c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f086406d040 con 0x7f08640cf450 2023-11-24T08:12:04.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f0868d4c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08640cfe40 con 0x7f08640ce6c0 2023-11-24T08:12:04.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f086259c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f086415c7f0 0x7f086406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48586/0 (socket says 172.21.15.106:48586) 2023-11-24T08:12:04.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f086259c640 1 -- 172.21.15.106:0/4027798145 learned_addr learned my addr 172.21.15.106:0/4027798145 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:04.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f0862d9d640 1 --2- 172.21.15.106:0/4027798145 >> v2:172.21.15.156:3300/0 conn(0x7f08640ce6c0 0x7f08640cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928262505 0 0) 0x7f08640cece0 con 0x7f086415c7f0 2023-11-24T08:12:04.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.631+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0850003520 con 0x7f086415c7f0 2023-11-24T08:12:04.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401104314 0 0) 0x7f0850003520 con 0x7f086415c7f0 2023-11-24T08:12:04.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 >> v2:172.21.15.156:3300/0 conn(0x7f08640ce6c0 msgr2=0x7f08640cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:04.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 --2- 172.21.15.106:0/4027798145 >> v2:172.21.15.156:3300/0 conn(0x7f08640ce6c0 0x7f08640cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 >> v1:172.21.15.156:6789/0 conn(0x7f08640cf450 legacy=0x7f08640cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f086406d320 con 0x7f086415c7f0 2023-11-24T08:12:04.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f08540019f0 con 0x7f086415c7f0 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 >> v1:172.21.15.106:6789/0 conn(0x7f086415c7f0 legacy=0x7f086406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0850003970 0x7f0850003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08500047c0 0x7f0850004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0850005180 0x7f085000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08640cfe40 con 0x7f08500047c0 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085000c0d0 con 0x7f0850005180 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085000c290 con 0x7f0850003970 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f0861d9b640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08500047c0 0x7f0850004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.632+0000 7f086259c640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0850003970 0x7f0850003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.633+0000 7f0861d9b640 1 -- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0850003970 msgr2=0x7f0850003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:04.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.633+0000 7f0861d9b640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0850003970 0x7f0850003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.633+0000 7f0861d9b640 1 -- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0850005180 msgr2=0x7f085000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:04.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.633+0000 7f0861d9b640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0850005180 0x7f085000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.633+0000 7f0861d9b640 1 -- 172.21.15.106:0/4027798145 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0850003520 con 0x7f08500047c0 2023-11-24T08:12:04.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.633+0000 7f0862d9d640 1 -- 172.21.15.106:0/4027798145 reap_dead start 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.634+0000 7f0861d9b640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08500047c0 0x7f0850004c40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f084c001750 tx=0x7f08500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=385ec059c7d26523 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.634+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c0052d0 con 0x7f08500047c0 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.634+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c02e050 con 0x7f08500047c0 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.634+0000 7f086159a640 1 -- 172.21.15.106:0/4027798145 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c003830 con 0x7f08500047c0 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.634+0000 7f0868d4c640 1 -- 172.21.15.106:0/4027798145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08500047c0 msgr2=0x7f0850004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.634+0000 7f0868d4c640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08500047c0 0x7f0850004c40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f084c001750 tx=0x7f08500057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.635+0000 7f0868d4c640 1 -- 172.21.15.106:0/4027798145 shutdown_connections 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.635+0000 7f0868d4c640 1 --2- 172.21.15.106:0/4027798145 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08500047c0 0x7f0850004c40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.635+0000 7f0868d4c640 1 -- 172.21.15.106:0/4027798145 >> 172.21.15.106:0/4027798145 conn(0x7f086406a920 msgr2=0x7f086406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.635+0000 7f0868d4c640 1 -- 172.21.15.106:0/4027798145 shutdown_connections 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.635+0000 7f0868d4c640 1 -- 172.21.15.106:0/4027798145 wait complete. 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.636+0000 7f0868d4c640 1 Processor -- start 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.637+0000 7f0868d4c640 1 -- start start 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0868d4c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08500047c0 0x7f08641483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0868d4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 0x7f08640ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0868d4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640aec10 0x7f08640af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0868d4c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08640af600 con 0x7f08640addb0 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0868d4c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641561f0 con 0x7f08640aec10 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0868d4c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08640ad8c0 con 0x7f08500047c0 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 0x7f08640ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 0x7f08640ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47782/0 (socket says 172.21.15.106:47782) 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 -- 172.21.15.106:0/1621784660 learned_addr learned my addr 172.21.15.106:0/1621784660 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:04.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f086259c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08500047c0 0x7f08641483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0862d9d640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640aec10 0x7f08640af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 -- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08500047c0 msgr2=0x7f08641483d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08500047c0 0x7f08641483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 -- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640aec10 msgr2=0x7f08640af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640aec10 0x7f08640af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0861d9b640 1 -- 172.21.15.106:0/1621784660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08640ad300 con 0x7f08640addb0 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.638+0000 7f0862d9d640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640aec10 0x7f08640af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.639+0000 7f0861d9b640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 0x7f08640ac9d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f084c02bc30 tx=0x7f084c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:04.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.639+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c0076c0 con 0x7f08640addb0 2023-11-24T08:12:04.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.639+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f084c007a30 con 0x7f08640addb0 2023-11-24T08:12:04.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.639+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f084c006be0 con 0x7f08640addb0 2023-11-24T08:12:04.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.639+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08640ad5e0 con 0x7f08640addb0 2023-11-24T08:12:04.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.639+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08640acf10 con 0x7f08640addb0 2023-11-24T08:12:04.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.640+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0864067470 con 0x7f08640addb0 2023-11-24T08:12:04.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.641+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f084c006d40 con 0x7f08640addb0 2023-11-24T08:12:04.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.642+0000 7f085affd640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f084406b260 0x7f084406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:04.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.642+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f084c0ab6f0 con 0x7f08640addb0 2023-11-24T08:12:04.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.642+0000 7f086259c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f084406b260 0x7f084406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:04.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.643+0000 7f086259c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f084406b260 0x7f084406d720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f08540013d0 tx=0x7f08540020c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:04.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.646+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f084c076370 con 0x7f08640addb0 2023-11-24T08:12:04.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.875+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0864156330 con 0x7f08640addb0 2023-11-24T08:12:04.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.876+0000 7f085affd640 1 -- 172.21.15.106:0/1621784660 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f084c079990 con 0x7f08640addb0 2023-11-24T08:12:04.878 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:04.878 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5059,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/1031368631","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":1031368631},{"type":"v1","addr":"172.21.15.156:6834","nonce":1031368631}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":5062,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/585891050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":585891050},{"type":"v1","addr":"172.21.15.156:6835","nonce":585891050}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6072,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6835/1622871643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":1622871643},{"type":"v1","addr":"172.21.15.106:6835","nonce":1622871643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":29},{"gid":6438,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/1752671708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6836","nonce":1752671708},{"type":"v1","addr":"172.21.15.106:6837","nonce":1752671708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36}],"filesystems":[]} 2023-11-24T08:12:04.879 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 36 2023-11-24T08:12:04.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.879+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f084406b260 msgr2=0x7f084406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.879+0000 7f0868d4c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f084406b260 0x7f084406d720 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f08540013d0 tx=0x7f08540020c0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.879+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 msgr2=0x7f08640ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:04.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.879+0000 7f0868d4c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 0x7f08640ac9d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f084c02bc30 tx=0x7f084c001120 comp rx=0 tx=0).stop 2023-11-24T08:12:04.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 shutdown_connections 2023-11-24T08:12:04.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08640aec10 0x7f08640af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08640addb0 0x7f08640ac9d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f084406b260 0x7f084406d720 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 --2- 172.21.15.106:0/1621784660 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08500047c0 0x7f08641483d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:04.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 >> 172.21.15.106:0/1621784660 conn(0x7f086406a920 msgr2=0x7f086415eec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:04.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 shutdown_connections 2023-11-24T08:12:04.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:04.880+0000 7f0868d4c640 1 -- 172.21.15.106:0/1621784660 wait complete. 2023-11-24T08:12:04.902 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:12:04.902 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:12:04.902 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:12:04.902 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:04.902 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:12:04.902 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:12:04.903 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:12:04.903 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:04.903 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:12:04.903 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:12:04.903 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:12:04.903 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:04.903 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:12:04.904 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:12:04.904 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:12:04.904 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:04.923 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:12:04.923 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:12:04.924 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:12:04.924 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:12:04.925 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:12:04.925 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:12:04.930 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:12:04.930 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:12:04.972 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:12:04.973 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:04.977 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:12:04.978 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:04.979 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:12:04.979 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:04.985 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:12:04.985 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:04.985 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:12:05.132 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:12:05.133 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:12:05.134 INFO:tasks.ceph.mds.c.smithi106.stderr:2023-11-24T08:12:05.133+0000 7fa4ab8a8640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-24T08:12:05.144 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-11-24T08:12:05.144 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:12:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.174+0000 7fb0cab01640 1 Processor -- start 2023-11-24T08:12:05.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.175+0000 7fb0cab01640 1 -- start start 2023-11-24T08:12:05.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0cab01640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0cab01640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0c40cd2e0 0x7fb0c4070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0cab01640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c4161a10 con 0x7fb0c406e160 2023-11-24T08:12:05.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0cab01640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c4161cf0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0cab01640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4161fd0 con 0x7fb0c4146eb0 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0c909e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0c40cd2e0 0x7fb0c4070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0c909e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb0c40cd2e0 0x7fb0c4070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47814/0 (socket says 172.21.15.106:47814) 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0c909e640 1 -- 172.21.15.106:0/1603705474 learned_addr learned my addr 172.21.15.106:0/1603705474 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.176+0000 7fb0c889d640 1 --2- 172.21.15.106:0/1603705474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.177+0000 7fb0c909e640 1 -- 172.21.15.106:0/1603705474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 msgr2=0x7fb0c4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.177+0000 7fb0c909e640 1 --2- 172.21.15.106:0/1603705474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c4161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.177+0000 7fb0c909e640 1 -- 172.21.15.106:0/1603705474 >> v1:172.21.15.156:6789/0 conn(0x7fb0c406e160 legacy=0x7fb0c40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.177+0000 7fb0c909e640 1 -- 172.21.15.106:0/1603705474 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c4070660 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.177+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1603705474 <== mon.1 v1:172.21.15.156:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983060288 0 0) 0x7fb0c4161a10 con 0x7fb0c406e160 2023-11-24T08:12:05.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.178+0000 7fb0c909e640 1 --2- 172.21.15.106:0/1603705474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c4070e40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009e50 tx=0x7fb0bc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b826c85513c1d5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:05.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.178+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1603705474 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc02cb00 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.178+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1603705474 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0bc02cc60 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.178+0000 7fb0c37fe640 1 -- 172.21.15.106:0/1603705474 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0bc0343f0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 -- 172.21.15.106:0/1603705474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 msgr2=0x7fb0c4070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 --2- 172.21.15.106:0/1603705474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c4070e40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fb0bc009e50 tx=0x7fb0bc02bf60 comp rx=0 tx=0).stop 2023-11-24T08:12:05.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 -- 172.21.15.106:0/1603705474 shutdown_connections 2023-11-24T08:12:05.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 --2- 172.21.15.106:0/1603705474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c4070e40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 --2- 172.21.15.106:0/1603705474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 -- 172.21.15.106:0/1603705474 >> 172.21.15.106:0/1603705474 conn(0x7fb0c40b0c50 msgr2=0x7fb0c40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:05.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 -- 172.21.15.106:0/1603705474 shutdown_connections 2023-11-24T08:12:05.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.179+0000 7fb0cab01640 1 -- 172.21.15.106:0/1603705474 wait complete. 2023-11-24T08:12:05.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.181+0000 7fb0cab01640 1 Processor -- start 2023-11-24T08:12:05.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.181+0000 7fb0cab01640 1 -- start start 2023-11-24T08:12:05.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0cab01640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c406e160 0x7fb0c40b26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0cab01640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c409a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0cab01640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c415ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0cab01640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40b3090 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0cab01640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415dce0 con 0x7fb0c406e160 2023-11-24T08:12:05.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0cab01640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415dfc0 con 0x7fb0c4146eb0 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0c3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c409a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0c889d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c406e160 0x7fb0c40b26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0c3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c409a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47820/0 (socket says 172.21.15.106:47820) 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2407098353 learned_addr learned my addr 172.21.15.106:0/2407098353 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.182+0000 7fb0c909e640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c415ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 msgr2=0x7fb0c415ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c415ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c406e160 msgr2=0x7fb0c40b26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c406e160 0x7fb0c40b26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c3fff640 1 -- 172.21.15.106:0/2407098353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0bc009b30 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c3fff640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c409a650 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c520 tx=0x7fb0b000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:05.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b00035f0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b0003770 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c415e2a0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.183+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b0003b20 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.184+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c40b2c20 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.185+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c415e7c0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.186+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fb0b0003c80 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.186+0000 7fb0c17fa640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0a806b230 0x7fb0a806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.187+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb0b0088f30 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.187+0000 7fb0c889d640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0a806b230 0x7fb0a806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.187+0000 7fb0c889d640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0a806b230 0x7fb0a806d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009aa0 tx=0x7fb0b8009190 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:05.192 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.191+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb0c415e7c0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.388+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb0c415e7c0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.466+0000 7fb0c17fa640 1 -- 172.21.15.106:0/2407098353 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v69) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb0c415e7c0 con 0x7fb0c40cd2e0 2023-11-24T08:12:05.468 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0a806b230 msgr2=0x7fb0a806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0a806b230 0x7fb0a806d6f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fb0b8009aa0 tx=0x7fb0b8009190 comp rx=0 tx=0).stop 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 msgr2=0x7fb0c409a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c409a650 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000c520 tx=0x7fb0b000c9e0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 shutdown_connections 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb0a806b230 0x7fb0a806d6f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb0c4146eb0 0x7fb0c415ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb0c40cd2e0 0x7fb0c409a650 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 --2- 172.21.15.106:0/2407098353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c406e160 0x7fb0c40b26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 >> 172.21.15.106:0/2407098353 conn(0x7fb0c40b0c50 msgr2=0x7fb0c40cebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:05.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 shutdown_connections 2023-11-24T08:12:05.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.469+0000 7fb0cab01640 1 -- 172.21.15.106:0/2407098353 wait complete. 2023-11-24T08:12:05.490 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:12:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.684+0000 7f614e86d640 1 Processor -- start 2023-11-24T08:12:05.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.684+0000 7f614e86d640 1 -- start start 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614e86d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f614806d0b0 0x7f6148070370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614e86d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614815b7a0 con 0x7f614813e0e0 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614e86d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614815ba60 con 0x7f61480708b0 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614e86d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614815bd40 con 0x7f614806d0b0 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614d86b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f614813e0e0 0x7f614806c9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48608/0 (socket says 172.21.15.106:48608) 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614d86b640 1 -- 172.21.15.106:0/1810071815 learned_addr learned my addr 172.21.15.106:0/1810071815 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:05.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.686+0000 7f614d06a640 1 --2- 172.21.15.106:0/1810071815 >> v2:172.21.15.106:3300/0 conn(0x7f614806d0b0 0x7f6148070370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.687+0000 7f614d06a640 1 -- 172.21.15.106:0/1810071815 >> v1:172.21.15.156:6789/0 conn(0x7f61480708b0 legacy=0x7f614813e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.687+0000 7f614d06a640 1 -- 172.21.15.106:0/1810071815 >> v1:172.21.15.106:6789/0 conn(0x7f614813e0e0 legacy=0x7f614806c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.687+0000 7f614d06a640 1 -- 172.21.15.106:0/1810071815 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614815c020 con 0x7f614806d0b0 2023-11-24T08:12:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.688+0000 7f614d06a640 1 --2- 172.21.15.106:0/1810071815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614806d0b0 0x7f6148070370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6144009d90 tx=0x7f614402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa667a33b257ce server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.688+0000 7f614c869640 1 -- 172.21.15.106:0/1810071815 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614402cc60 con 0x7f614806d0b0 2023-11-24T08:12:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.688+0000 7f614c869640 1 -- 172.21.15.106:0/1810071815 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614402cdc0 con 0x7f614806d0b0 2023-11-24T08:12:05.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.688+0000 7f614c869640 1 -- 172.21.15.106:0/1810071815 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144034660 con 0x7f614806d0b0 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 -- 172.21.15.106:0/1810071815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614806d0b0 msgr2=0x7f6148070370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 --2- 172.21.15.106:0/1810071815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614806d0b0 0x7f6148070370 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6144009d90 tx=0x7f614402beb0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 -- 172.21.15.106:0/1810071815 shutdown_connections 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 --2- 172.21.15.106:0/1810071815 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614806d0b0 0x7f6148070370 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 -- 172.21.15.106:0/1810071815 >> 172.21.15.106:0/1810071815 conn(0x7f61480b4630 msgr2=0x7f61480b4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 -- 172.21.15.106:0/1810071815 shutdown_connections 2023-11-24T08:12:05.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.689+0000 7f614e86d640 1 -- 172.21.15.106:0/1810071815 wait complete. 2023-11-24T08:12:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.690+0000 7f614e86d640 1 Processor -- start 2023-11-24T08:12:05.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.691+0000 7f614e86d640 1 -- start start 2023-11-24T08:12:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.691+0000 7f614e86d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614806d0b0 0x7f6148154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.691+0000 7f614e86d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61480708b0 0x7f61480aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e86d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 0x7f61481392c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e86d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148138220 con 0x7f614813e0e0 2023-11-24T08:12:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e86d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6148138500 con 0x7f614806d0b0 2023-11-24T08:12:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e86d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61481387e0 con 0x7f61480708b0 2023-11-24T08:12:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 0x7f61481392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 0x7f61481392c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47840/0 (socket says 172.21.15.106:47840) 2023-11-24T08:12:05.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 -- 172.21.15.106:0/935425944 learned_addr learned my addr 172.21.15.106:0/935425944 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 -- 172.21.15.106:0/935425944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61480708b0 msgr2=0x7f61480aeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61480708b0 0x7f61480aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 -- 172.21.15.106:0/935425944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614806d0b0 msgr2=0x7f6148154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614806d0b0 0x7f6148154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.692+0000 7f614e06c640 1 -- 172.21.15.106:0/935425944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144009a70 con 0x7f614813e0e0 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.693+0000 7f614e06c640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 0x7f61481392c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f614000c390 tx=0x7f614000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:05.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.693+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400035f0 con 0x7f614813e0e0 2023-11-24T08:12:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.693+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6140003750 con 0x7f614813e0e0 2023-11-24T08:12:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.693+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6140003b00 con 0x7f614813e0e0 2023-11-24T08:12:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.693+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6148138ac0 con 0x7f614813e0e0 2023-11-24T08:12:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.693+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6148154ff0 con 0x7f614813e0e0 2023-11-24T08:12:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.694+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6148138fe0 con 0x7f614813e0e0 2023-11-24T08:12:05.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.695+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6140003c60 con 0x7f614813e0e0 2023-11-24T08:12:05.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.696+0000 7f613e7fc640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f613406b230 0x7f613406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:05.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.696+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6140088cd0 con 0x7f614813e0e0 2023-11-24T08:12:05.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.697+0000 7f614d86b640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f613406b230 0x7f613406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:05.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.697+0000 7f614d86b640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f613406b230 0x7f613406d6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6138001550 tx=0x7f6138007a40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:05.702 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.701+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6148138fe0 con 0x7f614813e0e0 2023-11-24T08:12:05.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.952+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6148138fe0 con 0x7f614813e0e0 2023-11-24T08:12:05.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.953+0000 7f613e7fc640 1 -- 172.21.15.106:0/935425944 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6148138fe0 con 0x7f614813e0e0 2023-11-24T08:12:05.955 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:05.955 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:12:05.955 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2023-11-24T08:12:05.955 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.955 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2023-11-24T08:12:05.955 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A==", 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.956 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg==", 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:05.957 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:05.958 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.958 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.958 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.958 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.958 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2023-11-24T08:12:05.958 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q==", 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.959 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.960 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.960 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2023-11-24T08:12:05.960 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw==", 2023-11-24T08:12:05.960 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.960 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.961 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:05.961 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:05.961 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.961 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.961 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.961 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA==", 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.962 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.963 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.963 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.963 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2023-11-24T08:12:05.963 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA==", 2023-11-24T08:12:05.963 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.963 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.964 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.964 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.964 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.964 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.964 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA==", 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.965 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw==", 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.966 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.967 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.967 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.967 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.967 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2023-11-24T08:12:05.967 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw==", 2023-11-24T08:12:05.967 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.968 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw==", 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.969 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA==", 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.970 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug==", 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.971 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2023-11-24T08:12:05.972 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ==", 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.973 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg==", 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:12:05.974 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg==", 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.975 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2023-11-24T08:12:05.976 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w==", 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.977 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBludbKAhAARxKqsuG6+yx4M/G/4IloBw==", 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:05.978 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ==", 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:12:05.979 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA==", 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:05.980 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2023-11-24T08:12:05.981 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:12:05.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f613406b230 msgr2=0x7f613406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f613406b230 0x7f613406d6f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f6138001550 tx=0x7f6138007a40 comp rx=0 tx=0).stop 2023-11-24T08:12:05.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 msgr2=0x7f61481392c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:05.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 0x7f61481392c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f614000c390 tx=0x7f614000c850 comp rx=0 tx=0).stop 2023-11-24T08:12:05.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 shutdown_connections 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f614813e0e0 0x7f61481392c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f613406b230 0x7f613406d6f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f61480708b0 0x7f61480aeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 --2- 172.21.15.106:0/935425944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f614806d0b0 0x7f6148154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.955+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 >> 172.21.15.106:0/935425944 conn(0x7f61480b4630 msgr2=0x7f61480ccd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.956+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 shutdown_connections 2023-11-24T08:12:05.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:05.956+0000 7f614e86d640 1 -- 172.21.15.106:0/935425944 wait complete. 2023-11-24T08:12:05.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.194+0000 7f80e54ad640 1 Processor -- start 2023-11-24T08:12:06.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.194+0000 7f80e54ad640 1 -- start start 2023-11-24T08:12:06.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.195+0000 7f80e54ad640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80e00703f0 0x7f80e0162970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.195+0000 7f80e54ad640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e00cd160 0x7f80e00cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.195+0000 7f80e54ad640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80e0162eb0 con 0x7f80e014b100 2023-11-24T08:12:06.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.195+0000 7f80e54ad640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80e01654a0 con 0x7f80e00703f0 2023-11-24T08:12:06.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.195+0000 7f80e54ad640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00ccf00 con 0x7f80e00cd160 2023-11-24T08:12:06.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.196+0000 7f80d7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80e00703f0 0x7f80e0162970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.196+0000 7f80d7fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80e00703f0 0x7f80e0162970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:56038/0 (socket says 172.21.15.106:56038) 2023-11-24T08:12:06.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.196+0000 7f80d7fff640 1 -- 172.21.15.106:0/329105084 learned_addr learned my addr 172.21.15.106:0/329105084 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:06.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.196+0000 7f80e4cac640 1 --2- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e00cd160 0x7f80e00cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.197+0000 7f80d7fff640 1 -- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e00cd160 msgr2=0x7f80e00cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.197+0000 7f80d7fff640 1 --2- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e00cd160 0x7f80e00cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.197+0000 7f80d7fff640 1 -- 172.21.15.106:0/329105084 >> v1:172.21.15.156:6789/0 conn(0x7f80e014b100 legacy=0x7f80e0165c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.197+0000 7f80d7fff640 1 -- 172.21.15.106:0/329105084 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e0166320 con 0x7f80e00703f0 2023-11-24T08:12:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.199+0000 7f80d7fff640 1 --2- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00703f0 0x7f80e0162970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009870 tx=0x7f80c802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35bfcf68b0987da server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:06.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.199+0000 7f80df7fe640 1 -- 172.21.15.106:0/329105084 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8031640 con 0x7f80e00703f0 2023-11-24T08:12:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.199+0000 7f80df7fe640 1 -- 172.21.15.106:0/329105084 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80c80317a0 con 0x7f80e00703f0 2023-11-24T08:12:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.199+0000 7f80df7fe640 1 -- 172.21.15.106:0/329105084 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80c8031b30 con 0x7f80e00703f0 2023-11-24T08:12:06.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 -- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00703f0 msgr2=0x7f80e0162970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 --2- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00703f0 0x7f80e0162970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f80c8009870 tx=0x7f80c802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 -- 172.21.15.106:0/329105084 shutdown_connections 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 --2- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e00cd160 0x7f80e00cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 --2- 172.21.15.106:0/329105084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00703f0 0x7f80e0162970 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 -- 172.21.15.106:0/329105084 >> 172.21.15.106:0/329105084 conn(0x7f80e00cb830 msgr2=0x7f80e00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 -- 172.21.15.106:0/329105084 shutdown_connections 2023-11-24T08:12:06.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.200+0000 7f80e54ad640 1 -- 172.21.15.106:0/329105084 wait complete. 2023-11-24T08:12:06.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.202+0000 7f80e54ad640 1 Processor -- start 2023-11-24T08:12:06.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.202+0000 7f80e54ad640 1 -- start start 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.203+0000 7f80e54ad640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 0x7f80e00aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80e54ad640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00cd160 0x7f80e0160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80e54ad640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e014b100 0x7f80e01ed310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80e54ad640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00aa700 con 0x7f80e00703f0 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80e54ad640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00a9120 con 0x7f80e00cd160 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80e54ad640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e00a9400 con 0x7f80e014b100 2023-11-24T08:12:06.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 0x7f80e00aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 0x7f80e00aa1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47846/0 (socket says 172.21.15.106:47846) 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 -- 172.21.15.106:0/1069901144 learned_addr learned my addr 172.21.15.106:0/1069901144 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80d7fff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00cd160 0x7f80e0160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 -- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e014b100 msgr2=0x7f80e01ed310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e014b100 0x7f80e01ed310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 -- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00cd160 msgr2=0x7f80e0160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00cd160 0x7f80e0160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80dffff640 1 -- 172.21.15.106:0/1069901144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c80094d0 con 0x7f80e00703f0 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.204+0000 7f80d7fff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00cd160 0x7f80e0160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:06.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.205+0000 7f80dffff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 0x7f80e00aa1c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c3c0 tx=0x7f80d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.205+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d00052d0 con 0x7f80e00703f0 2023-11-24T08:12:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.205+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80d0005450 con 0x7f80e00703f0 2023-11-24T08:12:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.205+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80d0007260 con 0x7f80e00703f0 2023-11-24T08:12:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.205+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80e00a96e0 con 0x7f80e00703f0 2023-11-24T08:12:06.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.205+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e00a9ca0 con 0x7f80e00703f0 2023-11-24T08:12:06.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.206+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e004e840 con 0x7f80e00703f0 2023-11-24T08:12:06.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.207+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f80d00073c0 con 0x7f80e00703f0 2023-11-24T08:12:06.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.208+0000 7f80dd7fa640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80c006b230 0x7f80c006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.208+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f80d0088ba0 con 0x7f80e00703f0 2023-11-24T08:12:06.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.208+0000 7f80d7fff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80c006b230 0x7f80c006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.210 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.209+0000 7f80d7fff640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80c006b230 0x7f80c006d6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f80c8036000 tx=0x7f80c803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:06.214 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.213+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80d00539c0 con 0x7f80e00703f0 2023-11-24T08:12:06.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.391+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f80e006d7c0 con 0x7f80e00703f0 2023-11-24T08:12:06.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.393+0000 7f80dd7fa640 1 -- 172.21.15.106:0/1069901144 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+12905 (secure 0 0 0) 0x7f80d0056fe0 con 0x7f80e00703f0 2023-11-24T08:12:06.394 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:06.395 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":71,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:05.479320+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":60,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:06.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.395+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80c006b230 msgr2=0x7f80c006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.395+0000 7f80e54ad640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80c006b230 0x7f80c006d6f0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f80c8036000 tx=0x7f80c803c040 comp rx=0 tx=0).stop 2023-11-24T08:12:06.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.395+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 msgr2=0x7f80e00aa1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.395+0000 7f80e54ad640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 0x7f80e00aa1c0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f80d000c3c0 tx=0x7f80d000c880 comp rx=0 tx=0).stop 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.395+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 shutdown_connections 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80c006b230 0x7f80c006d6f0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80e014b100 0x7f80e01ed310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80e00cd160 0x7f80e0160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 --2- 172.21.15.106:0/1069901144 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80e00703f0 0x7f80e00aa1c0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 >> 172.21.15.106:0/1069901144 conn(0x7f80e00cb830 msgr2=0x7f80e0164c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 shutdown_connections 2023-11-24T08:12:06.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.396+0000 7f80e54ad640 1 -- 172.21.15.106:0/1069901144 wait complete. 2023-11-24T08:12:06.414 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:06.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:06.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.624+0000 7f8ac53d5640 1 Processor -- start 2023-11-24T08:12:06.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.624+0000 7f8ac53d5640 1 -- start start 2023-11-24T08:12:06.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8ac53d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ac0072160 0x7f8ac015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8ac53d5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac00c49f0 0x7f8ac006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8ac53d5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8ac006dc40 0x7f8ac00cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8ac53d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac006e0f0 con 0x7f8ac0072160 2023-11-24T08:12:06.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8ac53d5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac015ec90 con 0x7f8ac006dc40 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8ac53d5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00cd530 con 0x7f8ac00c49f0 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8abeffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ac0072160 0x7f8ac015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8abeffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8ac0072160 0x7f8ac015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47860/0 (socket says 172.21.15.106:47860) 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.626+0000 7f8abeffd640 1 -- 172.21.15.106:0/1589740335 learned_addr learned my addr 172.21.15.106:0/1589740335 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.627+0000 7f8abeffd640 1 -- 172.21.15.106:0/1589740335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac00c49f0 msgr2=0x7f8ac006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.627+0000 7f8abeffd640 1 --2- 172.21.15.106:0/1589740335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac00c49f0 0x7f8ac006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.627+0000 7f8abeffd640 1 -- 172.21.15.106:0/1589740335 >> v2:172.21.15.156:3300/0 conn(0x7f8ac006dc40 msgr2=0x7f8ac00cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.627+0000 7f8abeffd640 1 --2- 172.21.15.106:0/1589740335 >> v2:172.21.15.156:3300/0 conn(0x7f8ac006dc40 0x7f8ac00cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.627+0000 7f8abeffd640 1 -- 172.21.15.106:0/1589740335 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00c4ed0 con 0x7f8ac0072160 2023-11-24T08:12:06.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.628+0000 7f8abeffd640 1 --2- 172.21.15.106:0/1589740335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0072160 0x7f8ac015e750 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0072a30 tx=0x7f8ab002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a04677806390f7f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.628+0000 7f8abdffb640 1 -- 172.21.15.106:0/1589740335 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab002ea80 con 0x7f8ac0072160 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.628+0000 7f8abdffb640 1 -- 172.21.15.106:0/1589740335 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab002ebe0 con 0x7f8ac0072160 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.628+0000 7f8abdffb640 1 -- 172.21.15.106:0/1589740335 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0009600 con 0x7f8ac0072160 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1589740335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0072160 msgr2=0x7f8ac015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1589740335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0072160 0x7f8ac015e750 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0072a30 tx=0x7f8ab002e0e0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1589740335 shutdown_connections 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1589740335 >> v2:172.21.15.156:3300/0 conn(0x7f8ac006dc40 0x7f8ac00cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1589740335 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac00c49f0 0x7f8ac006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1589740335 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac0072160 0x7f8ac015e750 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1589740335 >> 172.21.15.106:0/1589740335 conn(0x7f8ac00b0c50 msgr2=0x7f8ac00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:06.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1589740335 shutdown_connections 2023-11-24T08:12:06.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.629+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1589740335 wait complete. 2023-11-24T08:12:06.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.631+0000 7f8ac53d5640 1 Processor -- start 2023-11-24T08:12:06.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.631+0000 7f8ac53d5640 1 -- start start 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8ac53d5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac006dc40 0x7f8ac009c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8ac53d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ac0072160 0x7f8ac00b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8ac53d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 0x7f8ac015cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8ac53d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00b3ea0 con 0x7f8ac00c49f0 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8ac53d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac015c2a0 con 0x7f8ac0072160 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8ac53d5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00b4140 con 0x7f8ac006dc40 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8abf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 0x7f8ac015cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8abf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 0x7f8ac015cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47864/0 (socket says 172.21.15.106:47864) 2023-11-24T08:12:06.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.632+0000 7f8abf7fe640 1 -- 172.21.15.106:0/1122928686 learned_addr learned my addr 172.21.15.106:0/1122928686 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abf7fe640 1 -- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac006dc40 msgr2=0x7f8ac009c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abf7fe640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac006dc40 0x7f8ac009c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abf7fe640 1 -- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ac0072160 msgr2=0x7f8ac00b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abe7fc640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ac0072160 0x7f8ac00b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abf7fe640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ac0072160 0x7f8ac00b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abf7fe640 1 -- 172.21.15.106:0/1122928686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab000bd40 con 0x7f8ac00c49f0 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abe7fc640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ac0072160 0x7f8ac00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:06.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8abf7fe640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 0x7f8ac015cbd0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009fd0 tx=0x7f8ab800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:06.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8011830 con 0x7f8ac00c49f0 2023-11-24T08:12:06.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8011990 con 0x7f8ac00c49f0 2023-11-24T08:12:06.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8011d20 con 0x7f8ac00c49f0 2023-11-24T08:12:06.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac00b4420 con 0x7f8ac00c49f0 2023-11-24T08:12:06.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.633+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac015a7f0 con 0x7f8ac00c49f0 2023-11-24T08:12:06.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.635+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac00ccb90 con 0x7f8ac00c49f0 2023-11-24T08:12:06.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.635+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8ab801b460 con 0x7f8ac00c49f0 2023-11-24T08:12:06.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.636+0000 7f8ab77fe640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8aa006b230 0x7f8aa006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:06.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.636+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8ab8089890 con 0x7f8ac00c49f0 2023-11-24T08:12:06.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.637+0000 7f8abeffd640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8aa006b230 0x7f8aa006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:06.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.637+0000 7f8abeffd640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8aa006b230 0x7f8aa006d6f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0037000 tx=0x7f8ab0030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:06.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.641+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8ab80580b0 con 0x7f8ac00c49f0 2023-11-24T08:12:06.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.820+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ac004e8b0 con 0x7f8ac00c49f0 2023-11-24T08:12:06.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.821+0000 7f8ab77fe640 1 -- 172.21.15.106:0/1122928686 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8ab8057c40 con 0x7f8ac00c49f0 2023-11-24T08:12:06.822 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:06.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8aa006b230 msgr2=0x7f8aa006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8aa006b230 0x7f8aa006d6f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0037000 tx=0x7f8ab0030000 comp rx=0 tx=0).stop 2023-11-24T08:12:06.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 msgr2=0x7f8ac015cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:06.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 0x7f8ac015cbd0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8009fd0 tx=0x7f8ab800b8e0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 shutdown_connections 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8ac00c49f0 0x7f8ac015cbd0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ac0072160 0x7f8ac00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8aa006b230 0x7f8aa006d6f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 --2- 172.21.15.106:0/1122928686 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8ac006dc40 0x7f8ac009c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.823+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 >> 172.21.15.106:0/1122928686 conn(0x7f8ac00b0c50 msgr2=0x7f8ac00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.824+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 shutdown_connections 2023-11-24T08:12:06.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:06.824+0000 7f8ac53d5640 1 -- 172.21.15.106:0/1122928686 wait complete. 2023-11-24T08:12:06.843 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:12:07.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.044+0000 7fcb67631640 1 Processor -- start 2023-11-24T08:12:07.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.045+0000 7fcb67631640 1 -- start start 2023-11-24T08:12:07.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb67631640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcb6015c7f0 0x7fcb6006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb67631640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb600cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb67631640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb600cdfb0 0x7fcb600ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb67631640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb60156010 con 0x7fcb600cdfb0 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb67631640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6006ce30 con 0x7fcb6015c7f0 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb67631640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600cfe10 con 0x7fcb600cf450 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.046+0000 7fcb65bce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb600cdfb0 0x7fcb600ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb600cdfb0 0x7fcb600ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47874/0 (socket says 172.21.15.106:47874) 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 -- 172.21.15.106:0/962975205 learned_addr learned my addr 172.21.15.106:0/962975205 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb653cd640 1 --2- 172.21.15.106:0/962975205 >> v2:172.21.15.156:3300/0 conn(0x7fcb6015c7f0 0x7fcb6006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb64bcc640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb600cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 -- 172.21.15.106:0/962975205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 msgr2=0x7fcb600cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb600cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 -- 172.21.15.106:0/962975205 >> v2:172.21.15.156:3300/0 conn(0x7fcb6015c7f0 msgr2=0x7fcb6006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 --2- 172.21.15.106:0/962975205 >> v2:172.21.15.156:3300/0 conn(0x7fcb6015c7f0 0x7fcb6006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb65bce640 1 -- 172.21.15.106:0/962975205 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600cece0 con 0x7fcb600cdfb0 2023-11-24T08:12:07.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.047+0000 7fcb64bcc640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb600cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.048+0000 7fcb65bce640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb600cdfb0 0x7fcb600ce440 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcb600d0130 tx=0x7fcb5802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=684a527cadb340b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb57fff640 1 -- 172.21.15.106:0/962975205 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5802ca70 con 0x7fcb600cdfb0 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb57fff640 1 -- 172.21.15.106:0/962975205 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5802cbd0 con 0x7fcb600cdfb0 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb57fff640 1 -- 172.21.15.106:0/962975205 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb580343f0 con 0x7fcb600cdfb0 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 -- 172.21.15.106:0/962975205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb600cdfb0 msgr2=0x7fcb600ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb600cdfb0 0x7fcb600ce440 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcb600d0130 tx=0x7fcb5802bed0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 -- 172.21.15.106:0/962975205 shutdown_connections 2023-11-24T08:12:07.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb600cdfb0 0x7fcb600ce440 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 --2- 172.21.15.106:0/962975205 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb600cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 --2- 172.21.15.106:0/962975205 >> v2:172.21.15.156:3300/0 conn(0x7fcb6015c7f0 0x7fcb6006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.049+0000 7fcb67631640 1 -- 172.21.15.106:0/962975205 >> 172.21.15.106:0/962975205 conn(0x7fcb6006a920 msgr2=0x7fcb6006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.050+0000 7fcb67631640 1 -- 172.21.15.106:0/962975205 shutdown_connections 2023-11-24T08:12:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.050+0000 7fcb67631640 1 -- 172.21.15.106:0/962975205 wait complete. 2023-11-24T08:12:07.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.051+0000 7fcb67631640 1 Processor -- start 2023-11-24T08:12:07.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.051+0000 7fcb67631640 1 -- start start 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.052+0000 7fcb67631640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb600cdfb0 0x7fcb60148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.052+0000 7fcb67631640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb6009c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.052+0000 7fcb67631640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 0x7fcb6009c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.052+0000 7fcb67631640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6009a9a0 con 0x7fcb6015c7f0 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.052+0000 7fcb67631640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6009ac80 con 0x7fcb600cdfb0 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.052+0000 7fcb67631640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6009af60 con 0x7fcb600cf450 2023-11-24T08:12:07.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 0x7fcb6009c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 0x7fcb6009c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47888/0 (socket says 172.21.15.106:47888) 2023-11-24T08:12:07.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 -- 172.21.15.106:0/3282734833 learned_addr learned my addr 172.21.15.106:0/3282734833 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:07.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 -- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 msgr2=0x7fcb6009c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb6009c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb653cd640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb600cdfb0 0x7fcb60148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 -- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb600cdfb0 msgr2=0x7fcb60148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb600cdfb0 0x7fcb60148560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 -- 172.21.15.106:0/3282734833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb58009b30 con 0x7fcb6015c7f0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.053+0000 7fcb65bce640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 0x7fcb6009c930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcb58036040 tx=0x7fcb58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.054+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb58009510 con 0x7fcb6015c7f0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.054+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5803e450 con 0x7fcb6015c7f0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.054+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6009b240 con 0x7fcb6015c7f0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.054+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5803e7e0 con 0x7fcb6015c7f0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.054+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6009ba40 con 0x7fcb6015c7f0 2023-11-24T08:12:07.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.055+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6015bd30 con 0x7fcb6015c7f0 2023-11-24T08:12:07.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.056+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcb5803e940 con 0x7fcb6015c7f0 2023-11-24T08:12:07.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.057+0000 7fcb55ffb640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb4406b230 0x7fcb4406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.057+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcb580ac830 con 0x7fcb6015c7f0 2023-11-24T08:12:07.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.057+0000 7fcb653cd640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb4406b230 0x7fcb4406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.058+0000 7fcb653cd640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb4406b230 0x7fcb4406d6f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c006fd0 tx=0x7fcb5c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:07.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.061+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcb58077580 con 0x7fcb6015c7f0 2023-11-24T08:12:07.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.237+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcb6009b700 con 0x7fcb6015c7f0 2023-11-24T08:12:07.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.237+0000 7fcb55ffb640 1 -- 172.21.15.106:0/3282734833 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcb58043060 con 0x7fcb6015c7f0 2023-11-24T08:12:07.239 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:07.239 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:12:07.239 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:07.239 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2023-11-24T08:12:07.239 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2023-11-24T08:12:07.240 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:07.240 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:12:07.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb4406b230 msgr2=0x7fcb4406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb4406b230 0x7fcb4406d6f0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c006fd0 tx=0x7fcb5c008040 comp rx=0 tx=0).stop 2023-11-24T08:12:07.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 msgr2=0x7fcb6009c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 0x7fcb6009c930 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcb58036040 tx=0x7fcb58004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:07.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 shutdown_connections 2023-11-24T08:12:07.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb6015c7f0 0x7fcb6009c930 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb4406b230 0x7fcb4406d6f0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb600cf450 0x7fcb6009c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 --2- 172.21.15.106:0/3282734833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb600cdfb0 0x7fcb60148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 >> 172.21.15.106:0/3282734833 conn(0x7fcb6006a920 msgr2=0x7fcb600cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:07.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 shutdown_connections 2023-11-24T08:12:07.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.240+0000 7fcb67631640 1 -- 172.21.15.106:0/3282734833 wait complete. 2023-11-24T08:12:07.260 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:12:07.260 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:12:07.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.464+0000 7f825d8d0640 1 Processor -- start 2023-11-24T08:12:07.466 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.465+0000 7f825d8d0640 1 -- start start 2023-11-24T08:12:07.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.466+0000 7f825d8d0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f825815d920 0x7f82580acfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.466+0000 7f825d8d0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f825806ac10 0x7f82580ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.466+0000 7f825d8d0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82580cfed0 con 0x7f82580b2400 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.466+0000 7f825d8d0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82580caae0 con 0x7f825815d920 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.466+0000 7f825d8d0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82580cadc0 con 0x7f825806ac10 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f825815d920 0x7f82580acfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f825815d920 0x7f82580acfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47894/0 (socket says 172.21.15.106:47894) 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 -- 172.21.15.106:0/3210973401 learned_addr learned my addr 172.21.15.106:0/3210973401 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f82577fe640 1 --2- 172.21.15.106:0/3210973401 >> v2:172.21.15.156:3300/0 conn(0x7f825806ac10 0x7f82580ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 -- 172.21.15.106:0/3210973401 >> v2:172.21.15.156:3300/0 conn(0x7f825806ac10 msgr2=0x7f82580ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:07.470 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 --2- 172.21.15.106:0/3210973401 >> v2:172.21.15.156:3300/0 conn(0x7f825806ac10 0x7f82580ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 -- 172.21.15.106:0/3210973401 >> v1:172.21.15.156:6789/0 conn(0x7f82580b2400 legacy=0x7f825806a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:07.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.467+0000 7f8256ffd640 1 -- 172.21.15.106:0/3210973401 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82580cb0a0 con 0x7f825815d920 2023-11-24T08:12:07.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.468+0000 7f8256ffd640 1 --2- 172.21.15.106:0/3210973401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f82580acfd0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f825806fea0 tx=0x7f824802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c608c8403d05fe1f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:07.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f8255ffb640 1 -- 172.21.15.106:0/3210973401 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f824802cb20 con 0x7f825815d920 2023-11-24T08:12:07.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f8255ffb640 1 -- 172.21.15.106:0/3210973401 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f824802cc80 con 0x7f825815d920 2023-11-24T08:12:07.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f8255ffb640 1 -- 172.21.15.106:0/3210973401 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82480343f0 con 0x7f825815d920 2023-11-24T08:12:07.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f825d8d0640 1 -- 172.21.15.106:0/3210973401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 msgr2=0x7f82580acfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f825d8d0640 1 --2- 172.21.15.106:0/3210973401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f82580acfd0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f825806fea0 tx=0x7f824802bf80 comp rx=0 tx=0).stop 2023-11-24T08:12:07.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f825d8d0640 1 -- 172.21.15.106:0/3210973401 shutdown_connections 2023-11-24T08:12:07.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f825d8d0640 1 --2- 172.21.15.106:0/3210973401 >> v2:172.21.15.156:3300/0 conn(0x7f825806ac10 0x7f82580ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f825d8d0640 1 --2- 172.21.15.106:0/3210973401 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f82580acfd0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.469+0000 7f825d8d0640 1 -- 172.21.15.106:0/3210973401 >> 172.21.15.106:0/3210973401 conn(0x7f82580999d0 msgr2=0x7f82580b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:07.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.470+0000 7f825d8d0640 1 -- 172.21.15.106:0/3210973401 shutdown_connections 2023-11-24T08:12:07.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.470+0000 7f825d8d0640 1 -- 172.21.15.106:0/3210973401 wait complete. 2023-11-24T08:12:07.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.471+0000 7f825d8d0640 1 Processor -- start 2023-11-24T08:12:07.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.472+0000 7f825d8d0640 1 -- start start 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.472+0000 7f825d8d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f825806ac10 0x7f825814c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.472+0000 7f825d8d0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580b2400 0x7f82580c5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f825d8d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f825806ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f825d8d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82580c4910 con 0x7f825815d920 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f825d8d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82580c4d30 con 0x7f825806ac10 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f825d8d0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f825814cce0 con 0x7f82580b2400 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f825806ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f825806ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47910/0 (socket says 172.21.15.106:47910) 2023-11-24T08:12:07.476 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 -- 172.21.15.106:0/429207559 learned_addr learned my addr 172.21.15.106:0/429207559 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82567fc640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580b2400 0x7f82580c5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 -- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580b2400 msgr2=0x7f82580c5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580b2400 0x7f82580c5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f8256ffd640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f825806ac10 0x7f825814c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 -- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f825806ac10 msgr2=0x7f825814c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f825806ac10 0x7f825814c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82577fe640 1 -- 172.21.15.106:0/429207559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248009b30 con 0x7f825815d920 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.473+0000 7f82567fc640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580b2400 0x7f82580c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.474+0000 7f82577fe640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f825806ea40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f825000d5d0 tx=0x7f825000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.474+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82500125f0 con 0x7f825815d920 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.474+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8250012750 con 0x7f825815d920 2023-11-24T08:12:07.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.474+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8250012b00 con 0x7f825815d920 2023-11-24T08:12:07.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.474+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82580c3cd0 con 0x7f825815d920 2023-11-24T08:12:07.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.474+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82580c4250 con 0x7f825815d920 2023-11-24T08:12:07.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.475+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82580c4a50 con 0x7f825815d920 2023-11-24T08:12:07.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.476+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8250012c60 con 0x7f825815d920 2023-11-24T08:12:07.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.477+0000 7f824ffff640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806b260 0x7f823806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:07.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.477+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8250089f30 con 0x7f825815d920 2023-11-24T08:12:07.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.477+0000 7f8256ffd640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806b260 0x7f823806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:07.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.478+0000 7f8256ffd640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806b260 0x7f823806d720 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8248035000 tx=0x7f8248030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:07.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.483+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f82580c4a50 con 0x7f825815d920 2023-11-24T08:12:07.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:07.671+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f82580c4a50 con 0x7f825815d920 2023-11-24T08:12:08.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.487+0000 7f824ffff640 1 -- 172.21.15.106:0/429207559 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) v1 ==== 138+0+0 (secure 0 0 0) 0x7f82580c4a50 con 0x7f825815d920 2023-11-24T08:12:08.489 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2023-11-24T08:12:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806b260 msgr2=0x7f823806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806b260 0x7f823806d720 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8248035000 tx=0x7f8248030000 comp rx=0 tx=0).stop 2023-11-24T08:12:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 msgr2=0x7f825806ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f825806ea40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f825000d5d0 tx=0x7f825000da90 comp rx=0 tx=0).stop 2023-11-24T08:12:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 shutdown_connections 2023-11-24T08:12:08.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f825815d920 0x7f825806ea40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f823806b260 0x7f823806d720 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82580b2400 0x7f82580c5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 --2- 172.21.15.106:0/429207559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f825806ac10 0x7f825814c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 >> 172.21.15.106:0/429207559 conn(0x7f82580999d0 msgr2=0x7f82580ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:08.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 shutdown_connections 2023-11-24T08:12:08.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.492+0000 7f825d8d0640 1 -- 172.21.15.106:0/429207559 wait complete. 2023-11-24T08:12:08.519 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:12:08.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.721+0000 7fae59cbb640 1 Processor -- start 2023-11-24T08:12:08.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.721+0000 7fae59cbb640 1 -- start start 2023-11-24T08:12:08.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.723+0000 7fae59cbb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae540d3a40 0x7fae5406ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:08.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.723+0000 7fae59cbb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae5415db00 con 0x7fae540b1ed0 2023-11-24T08:12:08.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.723+0000 7fae59cbb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae540d0ce0 con 0x7fae5415d720 2023-11-24T08:12:08.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.723+0000 7fae59cbb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae5406d400 con 0x7fae540d3a40 2023-11-24T08:12:08.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.723+0000 7fae53fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fae540b1ed0 0x7fae5406bfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48610/0 (socket says 172.21.15.106:48610) 2023-11-24T08:12:08.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.723+0000 7fae53fff640 1 -- 172.21.15.106:0/1420593364 learned_addr learned my addr 172.21.15.106:0/1420593364 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:08.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae52ffd640 1 --2- 172.21.15.106:0/1420593364 >> v2:172.21.15.156:3300/0 conn(0x7fae540d3a40 0x7fae5406ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:08.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae527fc640 1 -- 172.21.15.106:0/1420593364 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3247278983 0 0) 0x7fae5415db00 con 0x7fae540b1ed0 2023-11-24T08:12:08.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae527fc640 1 -- 172.21.15.106:0/1420593364 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae40003520 con 0x7fae540b1ed0 2023-11-24T08:12:08.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae52ffd640 1 -- 172.21.15.106:0/1420593364 >> v1:172.21.15.156:6789/0 conn(0x7fae5415d720 legacy=0x7fae540d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae52ffd640 1 -- 172.21.15.106:0/1420593364 >> v1:172.21.15.106:6789/0 conn(0x7fae540b1ed0 legacy=0x7fae5406bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae527fc640 1 -- 172.21.15.106:0/1420593364 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893911916 0 0) 0x7fae40003520 con 0x7fae540b1ed0 2023-11-24T08:12:08.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.724+0000 7fae52ffd640 1 -- 172.21.15.106:0/1420593364 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae540b0100 con 0x7fae540d3a40 2023-11-24T08:12:08.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.726+0000 7fae52ffd640 1 --2- 172.21.15.106:0/1420593364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae540d3a40 0x7fae5406ba80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0097a0 tx=0x7fae3c02b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ecc1e06da5faadc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:08.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.726+0000 7fae527fc640 1 -- 172.21.15.106:0/1420593364 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4c001670 con 0x7fae540d3a40 2023-11-24T08:12:08.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.727+0000 7fae527fc640 1 -- 172.21.15.106:0/1420593364 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c02e050 con 0x7fae540d3a40 2023-11-24T08:12:08.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.727+0000 7fae527fc640 1 -- 172.21.15.106:0/1420593364 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c030730 con 0x7fae540d3a40 2023-11-24T08:12:08.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.727+0000 7fae59cbb640 1 -- 172.21.15.106:0/1420593364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae540d3a40 msgr2=0x7fae5406ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.727+0000 7fae59cbb640 1 --2- 172.21.15.106:0/1420593364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae540d3a40 0x7fae5406ba80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0097a0 tx=0x7fae3c02b790 comp rx=0 tx=0).stop 2023-11-24T08:12:08.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.728+0000 7fae59cbb640 1 -- 172.21.15.106:0/1420593364 shutdown_connections 2023-11-24T08:12:08.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.728+0000 7fae59cbb640 1 --2- 172.21.15.106:0/1420593364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae540d3a40 0x7fae5406ba80 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.728+0000 7fae59cbb640 1 -- 172.21.15.106:0/1420593364 >> 172.21.15.106:0/1420593364 conn(0x7fae5408af60 msgr2=0x7fae5409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:08.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.728+0000 7fae59cbb640 1 -- 172.21.15.106:0/1420593364 shutdown_connections 2023-11-24T08:12:08.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.728+0000 7fae59cbb640 1 -- 172.21.15.106:0/1420593364 wait complete. 2023-11-24T08:12:08.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.729+0000 7fae59cbb640 1 Processor -- start 2023-11-24T08:12:08.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.730+0000 7fae59cbb640 1 -- start start 2023-11-24T08:12:08.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.730+0000 7fae59cbb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 0x7fae5414eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:08.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae59cbb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae540d3a40 0x7fae5406bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:08.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae59cbb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae5415d720 0x7fae5406c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:08.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae59cbb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae541f6760 con 0x7fae540b1ed0 2023-11-24T08:12:08.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae59cbb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae541f6940 con 0x7fae5415d720 2023-11-24T08:12:08.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae59cbb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae541f6c20 con 0x7fae540d3a40 2023-11-24T08:12:08.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 0x7fae5414eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:08.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 0x7fae5414eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47922/0 (socket says 172.21.15.106:47922) 2023-11-24T08:12:08.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 -- 172.21.15.106:0/433120768 learned_addr learned my addr 172.21.15.106:0/433120768 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae52ffd640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae540d3a40 0x7fae5406bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae53fff640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae5415d720 0x7fae5406c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 -- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae540d3a40 msgr2=0x7fae5406bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae540d3a40 0x7fae5406bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 -- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae5415d720 msgr2=0x7fae5406c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:08.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae5415d720 0x7fae5406c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:08.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae537fe640 1 -- 172.21.15.106:0/433120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c009400 con 0x7fae540b1ed0 2023-11-24T08:12:08.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.731+0000 7fae53fff640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae5415d720 0x7fae5406c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.732+0000 7fae537fe640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 0x7fae5414eb80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fae4400c730 tx=0x7fae4400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.732+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae44003b70 con 0x7fae540b1ed0 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.732+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae44003cf0 con 0x7fae540b1ed0 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.732+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae541f6f00 con 0x7fae540b1ed0 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.732+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae4400aa90 con 0x7fae540b1ed0 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.732+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae541f74c0 con 0x7fae540b1ed0 2023-11-24T08:12:08.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.733+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae541f7650 con 0x7fae540b1ed0 2023-11-24T08:12:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.734+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fae4400abf0 con 0x7fae540b1ed0 2023-11-24T08:12:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.735+0000 7fae4bfff640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae3406b230 0x7fae3406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.735+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fae4408a410 con 0x7fae540b1ed0 2023-11-24T08:12:08.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.735+0000 7fae52ffd640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae3406b230 0x7fae3406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:08.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.736+0000 7fae52ffd640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae3406b230 0x7fae3406d6f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae3c034000 tx=0x7fae3c02f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:08.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.739+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fae44055150 con 0x7fae540b1ed0 2023-11-24T08:12:08.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:08.934+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fae540d1f30 con 0x7fae540b1ed0 2023-11-24T08:12:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.493+0000 7fae4bfff640 1 -- 172.21.15.106:0/433120768 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) v1 ==== 170+0+0 (secure 0 0 0) 0x7fae44003e60 con 0x7fae540b1ed0 2023-11-24T08:12:09.495 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2023-11-24T08:12:09.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae3406b230 msgr2=0x7fae3406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae3406b230 0x7fae3406d6f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fae3c034000 tx=0x7fae3c02f000 comp rx=0 tx=0).stop 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 msgr2=0x7fae5414eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 0x7fae5414eb80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fae4400c730 tx=0x7fae4400cbf0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 shutdown_connections 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae5415d720 0x7fae5406c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae3406b230 0x7fae3406d6f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae540d3a40 0x7fae5406bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 --2- 172.21.15.106:0/433120768 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae540b1ed0 0x7fae5414eb80 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 >> 172.21.15.106:0/433120768 conn(0x7fae5408af60 msgr2=0x7fae540b4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 shutdown_connections 2023-11-24T08:12:09.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.497+0000 7fae59cbb640 1 -- 172.21.15.106:0/433120768 wait complete. 2023-11-24T08:12:09.532 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:12:09.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.718+0000 7fcaf431d640 1 Processor -- start 2023-11-24T08:12:09.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.718+0000 7fcaf431d640 1 -- start start 2023-11-24T08:12:09.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.719+0000 7fcaf431d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec15b440 0x7fcaec0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:09.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.719+0000 7fcaf431d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcaec0cbf20 0x7fcaec0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:09.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.719+0000 7fcaf431d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaec0cc3a0 con 0x7fcaec0c57e0 2023-11-24T08:12:09.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.719+0000 7fcaf431d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaec15b820 con 0x7fcaec0cbf20 2023-11-24T08:12:09.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.719+0000 7fcaf431d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec15eea0 con 0x7fcaec15b440 2023-11-24T08:12:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf18b8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcaec0cbf20 0x7fcaec0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf28ba640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcaec0c57e0 0x7fcaec0c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48616/0 (socket says 172.21.15.106:48616) 2023-11-24T08:12:09.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf28ba640 1 -- 172.21.15.106:0/1544911895 learned_addr learned my addr 172.21.15.106:0/1544911895 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf20b9640 1 --2- 172.21.15.106:0/1544911895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec15b440 0x7fcaec0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf18b8640 1 -- 172.21.15.106:0/1544911895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec15b440 msgr2=0x7fcaec0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:09.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf18b8640 1 --2- 172.21.15.106:0/1544911895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec15b440 0x7fcaec0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf18b8640 1 -- 172.21.15.106:0/1544911895 >> v1:172.21.15.106:6789/0 conn(0x7fcaec0c57e0 legacy=0x7fcaec0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf18b8640 1 -- 172.21.15.106:0/1544911895 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaec06d600 con 0x7fcaec0cbf20 2023-11-24T08:12:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.720+0000 7fcaf10b7640 1 -- 172.21.15.106:0/1544911895 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391505125 0 0) 0x7fcae4001490 con 0x7fcaec0c57e0 2023-11-24T08:12:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.721+0000 7fcaf18b8640 1 --2- 172.21.15.106:0/1544911895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec0cbf20 0x7fcaec0c52a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097a0 tx=0x7fcadc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ecd2d65e59098b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:09.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.722+0000 7fcaf10b7640 1 -- 172.21.15.106:0/1544911895 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc02e070 con 0x7fcaec0cbf20 2023-11-24T08:12:09.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.722+0000 7fcaf10b7640 1 -- 172.21.15.106:0/1544911895 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcadc0315a0 con 0x7fcaec0cbf20 2023-11-24T08:12:09.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.722+0000 7fcaf10b7640 1 -- 172.21.15.106:0/1544911895 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc031930 con 0x7fcaec0cbf20 2023-11-24T08:12:09.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.722+0000 7fcaf431d640 1 -- 172.21.15.106:0/1544911895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec0cbf20 msgr2=0x7fcaec0c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:09.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.722+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1544911895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec0cbf20 0x7fcaec0c52a0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fcadc0097a0 tx=0x7fcadc02b840 comp rx=0 tx=0).stop 2023-11-24T08:12:09.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.722+0000 7fcaf431d640 1 -- 172.21.15.106:0/1544911895 shutdown_connections 2023-11-24T08:12:09.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.723+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1544911895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec0cbf20 0x7fcaec0c52a0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.723+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1544911895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec15b440 0x7fcaec0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.723+0000 7fcaf431d640 1 -- 172.21.15.106:0/1544911895 >> 172.21.15.106:0/1544911895 conn(0x7fcaec0b0c50 msgr2=0x7fcaec0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:09.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.723+0000 7fcaf431d640 1 -- 172.21.15.106:0/1544911895 shutdown_connections 2023-11-24T08:12:09.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.723+0000 7fcaf431d640 1 -- 172.21.15.106:0/1544911895 wait complete. 2023-11-24T08:12:09.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.724+0000 7fcaf431d640 1 Processor -- start 2023-11-24T08:12:09.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.724+0000 7fcaf431d640 1 -- start start 2023-11-24T08:12:09.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.725+0000 7fcaf431d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec0c57e0 0x7fcaec15a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:09.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.725+0000 7fcaf431d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcaec0cbf20 0x7fcaec158a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:09.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.725+0000 7fcaf431d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 0x7fcaec154be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.725+0000 7fcaf431d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec15b960 con 0x7fcaec15b440 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.725+0000 7fcaf431d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec158f70 con 0x7fcaec0cbf20 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.725+0000 7fcaf431d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaec155120 con 0x7fcaec0c57e0 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 0x7fcaec154be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 0x7fcaec154be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47942/0 (socket says 172.21.15.106:47942) 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 -- 172.21.15.106:0/1301245894 learned_addr learned my addr 172.21.15.106:0/1301245894 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:09.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf20b9640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec0c57e0 0x7fcaec15a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:09.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 -- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec0c57e0 msgr2=0x7fcaec15a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:09.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf18b8640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcaec0cbf20 0x7fcaec158a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec0c57e0 0x7fcaec15a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 -- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcaec0cbf20 msgr2=0x7fcaec158a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcaec0cbf20 0x7fcaec158a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.726+0000 7fcaf28ba640 1 -- 172.21.15.106:0/1301245894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcadc009400 con 0x7fcaec15b440 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.727+0000 7fcaf28ba640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 0x7fcaec154be0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcae400c7f0 tx=0x7fcae400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.727+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae400a5e0 con 0x7fcaec15b440 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.727+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcae400a760 con 0x7fcaec15b440 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.727+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcae400ab50 con 0x7fcaec15b440 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.727+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaec1591d0 con 0x7fcaec15b440 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.727+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaec159790 con 0x7fcaec15b440 2023-11-24T08:12:09.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.728+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaec15f0d0 con 0x7fcaec15b440 2023-11-24T08:12:09.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.729+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fcae400acb0 con 0x7fcaec15b440 2023-11-24T08:12:09.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.730+0000 7fcae2ffd640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcad006b280 0x7fcad006d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:09.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.730+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fcae4089350 con 0x7fcaec15b440 2023-11-24T08:12:09.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.730+0000 7fcaf20b9640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcad006b280 0x7fcad006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:09.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.731+0000 7fcaf20b9640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcad006b280 0x7fcad006d740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcae8006fd0 tx=0x7fcae8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:09.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.735+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcaec15f0d0 con 0x7fcaec15b440 2023-11-24T08:12:09.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:09.953+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fcaec159980 con 0x7fcaec15b440 2023-11-24T08:12:10.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.539+0000 7fcae2ffd640 1 -- 172.21.15.106:0/1301245894 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v39) v1 ==== 174+0+0 (secure 0 0 0) 0x7fcaec159980 con 0x7fcaec15b440 2023-11-24T08:12:10.542 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 10 and data pool 11 2023-11-24T08:12:10.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.552+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcad006b280 msgr2=0x7fcad006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.552+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcad006b280 0x7fcad006d740 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcae8006fd0 tx=0x7fcae8008040 comp rx=0 tx=0).stop 2023-11-24T08:12:10.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.552+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 msgr2=0x7fcaec154be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.552+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 0x7fcaec154be0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fcae400c7f0 tx=0x7fcae400ccb0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.553+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 shutdown_connections 2023-11-24T08:12:10.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.553+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcaec15b440 0x7fcaec154be0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.553+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcaec0cbf20 0x7fcaec158a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.553+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcad006b280 0x7fcad006d740 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.553+0000 7fcaf431d640 1 --2- 172.21.15.106:0/1301245894 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcaec0c57e0 0x7fcaec15a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.553+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 >> 172.21.15.106:0/1301245894 conn(0x7fcaec0b0c50 msgr2=0x7fcaec0ac150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:10.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.554+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 shutdown_connections 2023-11-24T08:12:10.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.555+0000 7fcaf431d640 1 -- 172.21.15.106:0/1301245894 wait complete. 2023-11-24T08:12:10.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:10.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.762+0000 7fc3bf483640 1 Processor -- start 2023-11-24T08:12:10.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.762+0000 7fc3bf483640 1 -- start start 2023-11-24T08:12:10.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.764+0000 7fc3bf483640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 0x7fc3b80cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.764+0000 7fc3bf483640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b80b2d90 con 0x7fc3b80d22b0 2023-11-24T08:12:10.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.764+0000 7fc3bf483640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b80b45f0 con 0x7fc3b80d1ed0 2023-11-24T08:12:10.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.764+0000 7fc3bf483640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80ced90 con 0x7fc3b80ce8e0 2023-11-24T08:12:10.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.764+0000 7fc3bca1e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc3b80d22b0 0x7fc3b80ce1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48622/0 (socket says 172.21.15.106:48622) 2023-11-24T08:12:10.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.764+0000 7fc3bca1e640 1 -- 172.21.15.106:0/1091776883 learned_addr learned my addr 172.21.15.106:0/1091776883 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:10.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571952347 0 0) 0x7fc3b80b2d90 con 0x7fc3b80d22b0 2023-11-24T08:12:10.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3bda20640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 0x7fc3b80cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.768 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3a4003520 con 0x7fc3b80d22b0 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 555675017 0 0) 0x7fc3a4003520 con 0x7fc3b80d22b0 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 msgr2=0x7fc3b80cfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 0x7fc3b80cfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 >> v1:172.21.15.156:6789/0 conn(0x7fc3b80d1ed0 legacy=0x7fc3b80d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b80cf070 con 0x7fc3b80d22b0 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc3a8001670 con 0x7fc3b80d22b0 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 >> v1:172.21.15.106:6789/0 conn(0x7fc3b80d22b0 legacy=0x7fc3b80ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3a40039a0 0x7fc3a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40047c0 0x7fc3a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 0x7fc3a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80ced90 con 0x7fc3a40039a0 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a40054e0 con 0x7fc3a40047c0 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.765+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a400bec0 con 0x7fc3b80ce8e0 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3a40039a0 0x7fc3a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bca1e640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40047c0 0x7fc3a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bd21f640 1 -- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 msgr2=0x7fc3a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80ce8e0 0x7fc3a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bd21f640 1 -- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40047c0 msgr2=0x7fc3a4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40047c0 0x7fc3a4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bd21f640 1 -- 172.21.15.106:0/1091776883 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a4003520 con 0x7fc3a40039a0 2023-11-24T08:12:10.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.766+0000 7fc3bca1e640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40047c0 0x7fc3a4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:10.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.767+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3a40039a0 0x7fc3a4003dc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4001750 tx=0x7fc3a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7628e03d131f9a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:10.773 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.767+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40052d0 con 0x7fc3a40039a0 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.767+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b402e050 con 0x7fc3a40039a0 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.767+0000 7fc3affff640 1 -- 172.21.15.106:0/1091776883 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b40035f0 con 0x7fc3a40039a0 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bf483640 1 -- 172.21.15.106:0/1091776883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3a40039a0 msgr2=0x7fc3a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bf483640 1 --2- 172.21.15.106:0/1091776883 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3a40039a0 0x7fc3a4003dc0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4001750 tx=0x7fc3a40057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bda20640 1 -- 172.21.15.106:0/1091776883 reap_dead start 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bf483640 1 -- 172.21.15.106:0/1091776883 shutdown_connections 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bf483640 1 -- 172.21.15.106:0/1091776883 >> 172.21.15.106:0/1091776883 conn(0x7fc3b808af60 msgr2=0x7fc3b80d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bf483640 1 -- 172.21.15.106:0/1091776883 shutdown_connections 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.768+0000 7fc3bf483640 1 -- 172.21.15.106:0/1091776883 wait complete. 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.770+0000 7fc3bf483640 1 Processor -- start 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.770+0000 7fc3bf483640 1 -- start start 2023-11-24T08:12:10.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bf483640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40039a0 0x7fc3b80b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bf483640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80b7810 0x7fc3b80c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bf483640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 0x7fc3b80c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bf483640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80b8cd0 con 0x7fc3b80c7320 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bf483640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80b80c0 con 0x7fc3a40039a0 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bf483640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b80b83a0 con 0x7fc3b80b7810 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bda20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 0x7fc3b80c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.771+0000 7fc3bda20640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 0x7fc3b80c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47966/0 (socket says 172.21.15.106:47966) 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 -- 172.21.15.106:0/4054987886 learned_addr learned my addr 172.21.15.106:0/4054987886 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40039a0 0x7fc3b80b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bca1e640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80b7810 0x7fc3b80c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 -- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80b7810 msgr2=0x7fc3b80c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80b7810 0x7fc3b80c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 -- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40039a0 msgr2=0x7fc3b80b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40039a0 0x7fc3b80b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 -- 172.21.15.106:0/4054987886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b80b8680 con 0x7fc3b80c7320 2023-11-24T08:12:10.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bca1e640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80b7810 0x7fc3b80c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:10.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.772+0000 7fc3bda20640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 0x7fc3b80c77a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80d2bd0 tx=0x7fc3b000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.773+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0018070 con 0x7fc3b80c7320 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.773+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b000eb90 con 0x7fc3b80c7320 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.773+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b0003990 con 0x7fc3b80c7320 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.773+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b80b8960 con 0x7fc3b80c7320 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.773+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b80b7e70 con 0x7fc3b80c7320 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.774+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3b8067470 con 0x7fc3b80c7320 2023-11-24T08:12:10.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.775+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fc3b0014420 con 0x7fc3b80c7320 2023-11-24T08:12:10.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.776+0000 7fc3adffb640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc39c06b230 0x7fc39c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:10.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.776+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc3b0089b90 con 0x7fc3b80c7320 2023-11-24T08:12:10.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.776+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc39c06b230 0x7fc39c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:10.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.777+0000 7fc3bd21f640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc39c06b230 0x7fc39c06d6f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4033000 tx=0x7fc3b4009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:10.782 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.781+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3b0054660 con 0x7fc3b80c7320 2023-11-24T08:12:10.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.970+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3b80b2b50 con 0x7fc3b80c7320 2023-11-24T08:12:10.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.974+0000 7fc3adffb640 1 -- 172.21.15.106:0/4054987886 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16629 (secure 0 0 0) 0x7fc3b0057c80 con 0x7fc3b80c7320 2023-11-24T08:12:10.975 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:10.976 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":74,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:09.954870+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:10.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.976+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc39c06b230 msgr2=0x7fc39c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.976+0000 7fc3bf483640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc39c06b230 0x7fc39c06d6f0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4033000 tx=0x7fc3b4009030 comp rx=0 tx=0).stop 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.976+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 msgr2=0x7fc3b80c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.976+0000 7fc3bf483640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 0x7fc3b80c77a0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80d2bd0 tx=0x7fc3b000e4b0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 shutdown_connections 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3b80c7320 0x7fc3b80c77a0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc39c06b230 0x7fc39c06d6f0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3b80b7810 0x7fc3b80c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 --2- 172.21.15.106:0/4054987886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3a40039a0 0x7fc3b80b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 >> 172.21.15.106:0/4054987886 conn(0x7fc3b808af60 msgr2=0x7fc3b806ced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 shutdown_connections 2023-11-24T08:12:10.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:10.977+0000 7fc3bf483640 1 -- 172.21.15.106:0/4054987886 wait complete. 2023-11-24T08:12:10.996 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:11.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.195+0000 7f220ffff640 1 Processor -- start 2023-11-24T08:12:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.195+0000 7f220ffff640 1 -- start start 2023-11-24T08:12:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2210144b30 0x7f221015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220ffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f221015c410 0x7f22100b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220ffff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f221015bf80 con 0x7f22100b1de0 2023-11-24T08:12:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220ffff640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22100b1560 con 0x7f2210144b30 2023-11-24T08:12:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220ffff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22100b0fb0 con 0x7f221015c410 2023-11-24T08:12:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2210144b30 0x7f221015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2210144b30 0x7f221015f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47974/0 (socket says 172.21.15.106:47974) 2023-11-24T08:12:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220effd640 1 -- 172.21.15.106:0/1324356268 learned_addr learned my addr 172.21.15.106:0/1324356268 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220e7fc640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f221015c410 0x7f22100b0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.197+0000 7f220effd640 1 -- 172.21.15.106:0/1324356268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f221015c410 msgr2=0x7f22100b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.198+0000 7f220effd640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f221015c410 0x7f22100b0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.198+0000 7f220effd640 1 -- 172.21.15.106:0/1324356268 >> v1:172.21.15.106:6789/0 conn(0x7f22100b1de0 legacy=0x7f221006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.198+0000 7f220effd640 1 -- 172.21.15.106:0/1324356268 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22100b1280 con 0x7f2210144b30 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.198+0000 7f220dffb640 1 -- 172.21.15.106:0/1324356268 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808643955 0 0) 0x7f221015bf80 con 0x7f22100b1de0 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.198+0000 7f220e7fc640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f221015c410 0x7f22100b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.199+0000 7f220effd640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2210144b30 0x7f221015f920 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2200009870 tx=0x7f220002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573c9a35ba6f95b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:11.201 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.199+0000 7f220dffb640 1 -- 172.21.15.106:0/1324356268 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200030840 con 0x7f2210144b30 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.199+0000 7f220dffb640 1 -- 172.21.15.106:0/1324356268 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22000309a0 con 0x7f2210144b30 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.199+0000 7f220dffb640 1 -- 172.21.15.106:0/1324356268 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200030d30 con 0x7f2210144b30 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.199+0000 7f220ffff640 1 -- 172.21.15.106:0/1324356268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2210144b30 msgr2=0x7f221015f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.199+0000 7f220ffff640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2210144b30 0x7f221015f920 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f2200009870 tx=0x7f220002b980 comp rx=0 tx=0).stop 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.200+0000 7f220ffff640 1 -- 172.21.15.106:0/1324356268 shutdown_connections 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.200+0000 7f220ffff640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f221015c410 0x7f22100b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.200+0000 7f220ffff640 1 --2- 172.21.15.106:0/1324356268 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2210144b30 0x7f221015f920 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.200+0000 7f220ffff640 1 -- 172.21.15.106:0/1324356268 >> 172.21.15.106:0/1324356268 conn(0x7f22100cb830 msgr2=0x7f22100cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.200+0000 7f220ffff640 1 -- 172.21.15.106:0/1324356268 shutdown_connections 2023-11-24T08:12:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.200+0000 7f220ffff640 1 -- 172.21.15.106:0/1324356268 wait complete. 2023-11-24T08:12:11.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.201+0000 7f220ffff640 1 Processor -- start 2023-11-24T08:12:11.203 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.201+0000 7f220ffff640 1 -- start start 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 0x7f2210142900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220ffff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2210144b30 0x7f2210142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220ffff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221015c410 0x7f22101f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220ffff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2210144190 con 0x7f22100b1de0 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220ffff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2210071220 con 0x7f221015c410 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220ffff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f221015ac50 con 0x7f2210144b30 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.202+0000 7f220effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 0x7f2210142900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 0x7f2210142900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:47984/0 (socket says 172.21.15.106:47984) 2023-11-24T08:12:11.204 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 -- 172.21.15.106:0/1728993134 learned_addr learned my addr 172.21.15.106:0/1728993134 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:11.205 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 -- 172.21.15.106:0/1728993134 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2210144b30 msgr2=0x7f2210142e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2210144b30 0x7f2210142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 -- 172.21.15.106:0/1728993134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221015c410 msgr2=0x7f22101f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221015c410 0x7f22101f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 -- 172.21.15.106:0/1728993134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22000094d0 con 0x7f22100b1de0 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f220effd640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 0x7f2210142900 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f22000090b0 tx=0x7f2200004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.203+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220003e040 con 0x7f22100b1de0 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.204+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2200039e20 con 0x7f22100b1de0 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.204+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2200004270 con 0x7f22100b1de0 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.204+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2210143510 con 0x7f22100b1de0 2023-11-24T08:12:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.204+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2210143ad0 con 0x7f22100b1de0 2023-11-24T08:12:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.205+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f221004e840 con 0x7f22100b1de0 2023-11-24T08:12:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.206+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f22000043d0 con 0x7f22100b1de0 2023-11-24T08:12:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.207+0000 7f21f77fe640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f21fc06b230 0x7f21fc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.207+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22000ac8e0 con 0x7f22100b1de0 2023-11-24T08:12:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.207+0000 7f220e7fc640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f21fc06b230 0x7f21fc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.208+0000 7f220e7fc640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f21fc06b230 0x7f21fc06d6f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2204006fd0 tx=0x7f2204008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:11.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.211+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2200077450 con 0x7f22100b1de0 2023-11-24T08:12:11.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.390+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f221006d860 con 0x7f22100b1de0 2023-11-24T08:12:11.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.393+0000 7f21f77fe640 1 -- 172.21.15.106:0/1728993134 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16629 (secure 0 0 0) 0x7f220007aa70 con 0x7f22100b1de0 2023-11-24T08:12:11.395 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:11.395 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":74,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:09.954870+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f21fc06b230 msgr2=0x7f21fc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f21fc06b230 0x7f21fc06d6f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2204006fd0 tx=0x7f2204008040 comp rx=0 tx=0).stop 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 msgr2=0x7f2210142900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 0x7f2210142900 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f22000090b0 tx=0x7f2200004730 comp rx=0 tx=0).stop 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 shutdown_connections 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221015c410 0x7f22101f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f21fc06b230 0x7f21fc06d6f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2210144b30 0x7f2210142e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 --2- 172.21.15.106:0/1728993134 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f22100b1de0 0x7f2210142900 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 >> 172.21.15.106:0/1728993134 conn(0x7f22100cb830 msgr2=0x7f221015dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:11.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 shutdown_connections 2023-11-24T08:12:11.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.396+0000 7f220ffff640 1 -- 172.21.15.106:0/1728993134 wait complete. 2023-11-24T08:12:11.415 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:12:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.632+0000 7f0845a31640 1 Processor -- start 2023-11-24T08:12:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.632+0000 7f0845a31640 1 -- start start 2023-11-24T08:12:11.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f0845a31640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08400c49f0 0x7f084015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f0845a31640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f084015ec90 con 0x7f0840072160 2023-11-24T08:12:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f0845a31640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08400cd530 con 0x7f084006de10 2023-11-24T08:12:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f0845a31640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08400c4ed0 con 0x7f08400c49f0 2023-11-24T08:12:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f083effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f0840072160 0x7f084006d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48638/0 (socket says 172.21.15.106:48638) 2023-11-24T08:12:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f083effd640 1 -- 172.21.15.106:0/2647157713 learned_addr learned my addr 172.21.15.106:0/2647157713 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:11.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.634+0000 7f083f7fe640 1 --2- 172.21.15.106:0/2647157713 >> v2:172.21.15.106:3300/0 conn(0x7f08400c49f0 0x7f084015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.635+0000 7f083dffb640 1 -- 172.21.15.106:0/2647157713 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1825196847 0 0) 0x7f0830001110 con 0x7f0840072160 2023-11-24T08:12:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.635+0000 7f083dffb640 1 -- 172.21.15.106:0/2647157713 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0828005730 con 0x7f0840072160 2023-11-24T08:12:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.635+0000 7f083f7fe640 1 -- 172.21.15.106:0/2647157713 >> v1:172.21.15.156:6789/0 conn(0x7f084006de10 legacy=0x7f08400705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.635+0000 7f083f7fe640 1 -- 172.21.15.106:0/2647157713 >> v1:172.21.15.106:6789/0 conn(0x7f0840072160 legacy=0x7f084006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.635+0000 7f083f7fe640 1 -- 172.21.15.106:0/2647157713 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400c51b0 con 0x7f08400c49f0 2023-11-24T08:12:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.636+0000 7f083f7fe640 1 --2- 172.21.15.106:0/2647157713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f084015e750 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0838009d90 tx=0x7f083802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1d99f50a571f892 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.636+0000 7f083dffb640 1 -- 172.21.15.106:0/2647157713 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083802cc60 con 0x7f08400c49f0 2023-11-24T08:12:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.636+0000 7f083dffb640 1 -- 172.21.15.106:0/2647157713 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083802cdc0 con 0x7f08400c49f0 2023-11-24T08:12:11.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.636+0000 7f083dffb640 1 -- 172.21.15.106:0/2647157713 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0838034660 con 0x7f08400c49f0 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 -- 172.21.15.106:0/2647157713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 msgr2=0x7f084015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 --2- 172.21.15.106:0/2647157713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f084015e750 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f0838009d90 tx=0x7f083802beb0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 -- 172.21.15.106:0/2647157713 shutdown_connections 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 --2- 172.21.15.106:0/2647157713 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f084015e750 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 -- 172.21.15.106:0/2647157713 >> 172.21.15.106:0/2647157713 conn(0x7f08400b0c50 msgr2=0x7f08400b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 -- 172.21.15.106:0/2647157713 shutdown_connections 2023-11-24T08:12:11.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.637+0000 7f0845a31640 1 -- 172.21.15.106:0/2647157713 wait complete. 2023-11-24T08:12:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.639+0000 7f0845a31640 1 Processor -- start 2023-11-24T08:12:11.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.639+0000 7f0845a31640 1 -- start start 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f0845a31640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f084006de10 0x7f0840152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f0845a31640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0840072160 0x7f08400b3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f0845a31640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f08400b3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f0845a31640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400af080 con 0x7f08400c49f0 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f0845a31640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400b4520 con 0x7f0840072160 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f0845a31640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08400ae4a0 con 0x7f084006de10 2023-11-24T08:12:11.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f08400b3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f08400b3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48004/0 (socket says 172.21.15.106:48004) 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083f7fe640 1 -- 172.21.15.106:0/2263357118 learned_addr learned my addr 172.21.15.106:0/2263357118 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083effd640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f084006de10 0x7f0840152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083e7fc640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0840072160 0x7f08400b3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083f7fe640 1 -- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f084006de10 msgr2=0x7f0840152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.640+0000 7f083f7fe640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f084006de10 0x7f0840152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f083f7fe640 1 -- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0840072160 msgr2=0x7f08400b3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f083f7fe640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0840072160 0x7f08400b3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f083f7fe640 1 -- 172.21.15.106:0/2263357118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838009a70 con 0x7f08400c49f0 2023-11-24T08:12:11.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f083f7fe640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f08400b3fe0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0838037040 tx=0x7f0838004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0838040650 con 0x7f08400c49f0 2023-11-24T08:12:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08380407b0 con 0x7f08400c49f0 2023-11-24T08:12:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0838040b40 con 0x7f08400c49f0 2023-11-24T08:12:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08400ae750 con 0x7f08400c49f0 2023-11-24T08:12:11.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.641+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08400aed10 con 0x7f08400c49f0 2023-11-24T08:12:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.642+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08400af230 con 0x7f08400c49f0 2023-11-24T08:12:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.643+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f0838040ca0 con 0x7f08400c49f0 2023-11-24T08:12:11.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.644+0000 7f0827fff640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f082c06b230 0x7f082c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.644+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08380acb10 con 0x7f08400c49f0 2023-11-24T08:12:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.645+0000 7f083effd640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f082c06b230 0x7f082c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:11.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.645+0000 7f083effd640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f082c06b230 0x7f082c06d6f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0830007fd0 tx=0x7f0830009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:11.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.649+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08400af230 con 0x7f08400c49f0 2023-11-24T08:12:11.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:11.880+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f08400aeea0 con 0x7f08400c49f0 2023-11-24T08:12:12.549 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.547+0000 7f0827fff640 1 -- 172.21.15.106:0/2263357118 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v42) v1 ==== 118+0+0 (secure 0 0 0) 0x7f08380390b0 con 0x7f08400c49f0 2023-11-24T08:12:12.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f082c06b230 msgr2=0x7f082c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f082c06b230 0x7f082c06d6f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f0830007fd0 tx=0x7f0830009040 comp rx=0 tx=0).stop 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 msgr2=0x7f08400b3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f08400b3fe0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f0838037040 tx=0x7f0838004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 shutdown_connections 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08400c49f0 0x7f08400b3fe0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0840072160 0x7f08400b3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f082c06b230 0x7f082c06d6f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 --2- 172.21.15.106:0/2263357118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f084006de10 0x7f0840152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 >> 172.21.15.106:0/2263357118 conn(0x7f08400b0c50 msgr2=0x7f08400b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:12.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.551+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 shutdown_connections 2023-11-24T08:12:12.554 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.552+0000 7f0845a31640 1 -- 172.21.15.106:0/2263357118 wait complete. 2023-11-24T08:12:12.574 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:12:12.574 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:12:12.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.783+0000 7fe3463fa640 1 Processor -- start 2023-11-24T08:12:12.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.784+0000 7fe3463fa640 1 -- start start 2023-11-24T08:12:12.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.785+0000 7fe3463fa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe3400b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.785+0000 7fe3463fa640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34006d430 con 0x7fe34015ded0 2023-11-24T08:12:12.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.785+0000 7fe3463fa640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3400b14e0 con 0x7fe34015a1c0 2023-11-24T08:12:12.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.785+0000 7fe3463fa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015ea70 con 0x7fe3400b0b40 2023-11-24T08:12:12.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.785+0000 7fe345bf9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe34015ded0 0x7fe34015e360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48640/0 (socket says 172.21.15.106:48640) 2023-11-24T08:12:12.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.785+0000 7fe345bf9640 1 -- 172.21.15.106:0/1674219098 learned_addr learned my addr 172.21.15.106:0/1674219098 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe344bf7640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe3400b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865603853 0 0) 0x7fe34006d430 con 0x7fe34015ded0 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c003520 con 0x7fe34015ded0 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3885163142 0 0) 0x7fe32c003520 con 0x7fe34015ded0 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 msgr2=0x7fe3400b0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe3400b0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 >> v1:172.21.15.156:6789/0 conn(0x7fe34015a1c0 legacy=0x7fe34006d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34015ec50 con 0x7fe34015ded0 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fe338001670 con 0x7fe34015ded0 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe344bf7640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe3400b0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 >> v1:172.21.15.106:6789/0 conn(0x7fe34015ded0 legacy=0x7fe34015e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe32c0039a0 0x7fe32c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.786+0000 7fe337fff640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe32c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe337fff640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32c004d00 0x7fe32c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34015ea70 con 0x7fe32c004d00 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c00ba90 con 0x7fe32c0039a0 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c00de60 con 0x7fe3400b0b40 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe345bf9640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32c004d00 0x7fe32c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe3453f8640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe32c0039a0 0x7fe32c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe345bf9640 1 -- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 msgr2=0x7fe32c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:12.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe344bf7640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe32c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe345bf9640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe3400b0b40 0x7fe32c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe345bf9640 1 -- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe32c0039a0 msgr2=0x7fe32c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:12.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe345bf9640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe32c0039a0 0x7fe32c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.787+0000 7fe345bf9640 1 -- 172.21.15.106:0/1674219098 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c003520 con 0x7fe32c004d00 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.788+0000 7fe345bf9640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32c004d00 0x7fe32c005180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c210 tx=0x7fe32c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e451e1e9e64f2aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338005ad0 con 0x7fe32c004d00 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe338005c30 con 0x7fe32c004d00 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe337fff640 1 -- 172.21.15.106:0/1674219098 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33802c890 con 0x7fe32c004d00 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe3463fa640 1 -- 172.21.15.106:0/1674219098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32c004d00 msgr2=0x7fe32c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe3463fa640 1 --2- 172.21.15.106:0/1674219098 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32c004d00 0x7fe32c005180 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c210 tx=0x7fe32c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe345bf9640 1 -- 172.21.15.106:0/1674219098 reap_dead start 2023-11-24T08:12:12.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe3463fa640 1 -- 172.21.15.106:0/1674219098 shutdown_connections 2023-11-24T08:12:12.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe3463fa640 1 -- 172.21.15.106:0/1674219098 >> 172.21.15.106:0/1674219098 conn(0x7fe3400cbb40 msgr2=0x7fe3400cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:12.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe3463fa640 1 -- 172.21.15.106:0/1674219098 shutdown_connections 2023-11-24T08:12:12.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.789+0000 7fe3463fa640 1 -- 172.21.15.106:0/1674219098 wait complete. 2023-11-24T08:12:12.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.791+0000 7fe3463fa640 1 Processor -- start 2023-11-24T08:12:12.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.792+0000 7fe3463fa640 1 -- start start 2023-11-24T08:12:12.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.792+0000 7fe3463fa640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32c004d00 0x7fe34014f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe3463fa640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3400c81e0 0x7fe34009b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe3463fa640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 0x7fe34009b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe3463fa640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c3540 con 0x7fe3400c8640 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe3463fa640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3400c6b50 con 0x7fe3400c81e0 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe3463fa640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34014faa0 con 0x7fe32c004d00 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 0x7fe34009b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 0x7fe34009b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48032/0 (socket says 172.21.15.106:48032) 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 -- 172.21.15.106:0/3511754411 learned_addr learned my addr 172.21.15.106:0/3511754411 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe344bf7640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3400c81e0 0x7fe34009b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 -- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32c004d00 msgr2=0x7fe34014f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32c004d00 0x7fe34014f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 -- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3400c81e0 msgr2=0x7fe34009b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3400c81e0 0x7fe34009b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:12.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.793+0000 7fe345bf9640 1 -- 172.21.15.106:0/3511754411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3400c6db0 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.794+0000 7fe345bf9640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 0x7fe34009b610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c330 tx=0x7fe33802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.794+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33803c040 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.794+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33802cca0 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.794+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe338007270 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.794+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3400c7090 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.794+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3400c7650 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.795+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34004e840 con 0x7fe3400c8640 2023-11-24T08:12:12.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.796+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe338032070 con 0x7fe3400c8640 2023-11-24T08:12:12.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.797+0000 7fe335ffb640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32406b230 0x7fe32406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:12.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.797+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3380aa8f0 con 0x7fe3400c8640 2023-11-24T08:12:12.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.797+0000 7fe3453f8640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32406b230 0x7fe32406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:12.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.798+0000 7fe3453f8640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32406b230 0x7fe32406d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe33c007fd0 tx=0x7fe33c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:12.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:12.801+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe3380752f0 con 0x7fe3400c8640 2023-11-24T08:12:13.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.027+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe340159a20 con 0x7fe3400c8640 2023-11-24T08:12:13.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.550+0000 7fe335ffb640 1 -- 172.21.15.106:0/3511754411 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v43) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe338078910 con 0x7fe3400c8640 2023-11-24T08:12:13.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32406b230 msgr2=0x7fe32406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32406b230 0x7fe32406d6f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fe33c007fd0 tx=0x7fe33c009040 comp rx=0 tx=0).stop 2023-11-24T08:12:13.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 msgr2=0x7fe34009b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 0x7fe34009b610 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fe33800c330 tx=0x7fe33802e070 comp rx=0 tx=0).stop 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 shutdown_connections 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe3400c8640 0x7fe34009b610 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3400c81e0 0x7fe34009b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe32406b230 0x7fe32406d6f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 --2- 172.21.15.106:0/3511754411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32c004d00 0x7fe34014f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 >> 172.21.15.106:0/3511754411 conn(0x7fe3400cbb40 msgr2=0x7fe3400ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 shutdown_connections 2023-11-24T08:12:13.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.554+0000 7fe3463fa640 1 -- 172.21.15.106:0/3511754411 wait complete. 2023-11-24T08:12:13.578 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:12:13.578 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:13.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.791+0000 7fda0221e640 1 Processor -- start 2023-11-24T08:12:13.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.791+0000 7fda0221e640 1 -- start start 2023-11-24T08:12:13.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.793+0000 7fda0221e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd9fc0cfb60 0x7fd9fc161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.793+0000 7fda0221e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc0ce0b0 con 0x7fd9fc157c70 2023-11-24T08:12:13.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.793+0000 7fda0221e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9fc161960 con 0x7fd9fc15dc80 2023-11-24T08:12:13.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.793+0000 7fda0221e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9fc15e8d0 con 0x7fd9fc0cfb60 2023-11-24T08:12:13.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.793+0000 7fd9fb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd9fc157c70 0x7fd9fc0cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48654/0 (socket says 172.21.15.106:48654) 2023-11-24T08:12:13.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.793+0000 7fd9fb7fe640 1 -- 172.21.15.106:0/4121837342 learned_addr learned my addr 172.21.15.106:0/4121837342 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3633550665 0 0) 0x7fd9fc0ce0b0 con 0x7fd9fc157c70 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9e4003520 con 0x7fd9fc157c70 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9faffd640 1 --2- 172.21.15.106:0/4121837342 >> v2:172.21.15.156:3300/0 conn(0x7fd9fc0cfb60 0x7fd9fc161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2777250278 0 0) 0x7fd9e4003520 con 0x7fd9fc157c70 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 >> v2:172.21.15.156:3300/0 conn(0x7fd9fc0cfb60 msgr2=0x7fd9fc161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 --2- 172.21.15.106:0/4121837342 >> v2:172.21.15.156:3300/0 conn(0x7fd9fc0cfb60 0x7fd9fc161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 >> v1:172.21.15.156:6789/0 conn(0x7fd9fc15dc80 legacy=0x7fd9fc0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc15ebb0 con 0x7fd9fc157c70 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fd9ec001670 con 0x7fd9fc157c70 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 >> v1:172.21.15.106:6789/0 conn(0x7fd9fc157c70 legacy=0x7fd9fc0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9e4003970 0x7fd9e4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9e40047c0 0x7fd9e4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9e400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc15e8d0 con 0x7fd9e40052a0 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e400c170 con 0x7fd9e40047c0 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e400c330 con 0x7fd9e4003970 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.794+0000 7fd9fbfff640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9e400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9faffd640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9e40047c0 0x7fd9e4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9fb7fe640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9e4003970 0x7fd9e4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9fbfff640 1 -- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9e4003970 msgr2=0x7fd9e4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9fbfff640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9e4003970 0x7fd9e4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9fbfff640 1 -- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9e40047c0 msgr2=0x7fd9e4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9fbfff640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9e40047c0 0x7fd9e4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9fbfff640 1 -- 172.21.15.106:0/4121837342 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4003520 con 0x7fd9e40052a0 2023-11-24T08:12:13.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.795+0000 7fd9faffd640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9e40047c0 0x7fd9e4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.796+0000 7fd9fbfff640 1 -- 172.21.15.106:0/4121837342 reap_dead start 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.796+0000 7fd9fbfff640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9e400b980 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4001e20 tx=0x7fd9e40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efc24785b8d8bf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.796+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4003b70 con 0x7fd9e40052a0 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.796+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9f4003cd0 con 0x7fd9e40052a0 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.796+0000 7fd9fa7fc640 1 -- 172.21.15.106:0/4121837342 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9f40052f0 con 0x7fd9e40052a0 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 -- 172.21.15.106:0/4121837342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 msgr2=0x7fd9e400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9e400b980 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4001e20 tx=0x7fd9e40057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 -- 172.21.15.106:0/4121837342 shutdown_connections 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 --2- 172.21.15.106:0/4121837342 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9e400b980 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 -- 172.21.15.106:0/4121837342 >> 172.21.15.106:0/4121837342 conn(0x7fd9fc06a920 msgr2=0x7fd9fc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:13.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 -- 172.21.15.106:0/4121837342 shutdown_connections 2023-11-24T08:12:13.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.797+0000 7fda0221e640 1 -- 172.21.15.106:0/4121837342 wait complete. 2023-11-24T08:12:13.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.799+0000 7fda0221e640 1 Processor -- start 2023-11-24T08:12:13.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.800+0000 7fda0221e640 1 -- start start 2023-11-24T08:12:13.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fda0221e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9fc09a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fda0221e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9fc0ae1d0 0x7fd9fc0ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fda0221e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9fc0ae650 0x7fd9fc0cb480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fda0221e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c8360 con 0x7fd9e40052a0 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fda0221e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c85b0 con 0x7fd9fc0ae1d0 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fda0221e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9fc0c8890 con 0x7fd9fc0ae650 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9fc09a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9fc09a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:48048/0 (socket says 172.21.15.106:48048) 2023-11-24T08:12:13.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 -- 172.21.15.106:0/3753252112 learned_addr learned my addr 172.21.15.106:0/3753252112 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:13.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9faffd640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9fc0ae1d0 0x7fd9fc0ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 -- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9fc0ae650 msgr2=0x7fd9fc0cb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9fc0ae650 0x7fd9fc0cb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 -- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9fc0ae1d0 msgr2=0x7fd9fc0ad280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.801+0000 7fd9fb7fe640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9fc0ae1d0 0x7fd9fc0ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fd9fb7fe640 1 -- 172.21.15.106:0/3753252112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9fc0c8b70 con 0x7fd9e40052a0 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fd9fb7fe640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9fc09a050 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec001050 tx=0x7fd9ec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00eed0 con 0x7fd9e40052a0 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ec00c870 con 0x7fd9e40052a0 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00cc00 con 0x7fd9e40052a0 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9fc0c8e50 con 0x7fd9e40052a0 2023-11-24T08:12:13.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.802+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9fc156d70 con 0x7fd9e40052a0 2023-11-24T08:12:13.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.803+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9fc067470 con 0x7fd9e40052a0 2023-11-24T08:12:13.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.804+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd9ec01d460 con 0x7fd9e40052a0 2023-11-24T08:12:13.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.806+0000 7fd9e3fff640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd9e806b230 0x7fd9e806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:13.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.806+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9ec089c90 con 0x7fd9e40052a0 2023-11-24T08:12:13.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.806+0000 7fd9faffd640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd9e806b230 0x7fd9e806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:13.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.807+0000 7fd9faffd640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd9e806b230 0x7fd9e806d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0ca8f0 tx=0x7fd9f0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:13.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:13.811+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd9ec054720 con 0x7fd9e40052a0 2023-11-24T08:12:14.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.039+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9fc156f00 con 0x7fd9e40052a0 2023-11-24T08:12:14.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.040+0000 7fd9e3fff640 1 -- 172.21.15.106:0/3753252112 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fd9ec057d40 con 0x7fd9e40052a0 2023-11-24T08:12:14.042 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:14.042 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:09.955346+0000","modified":"2023-11-24T08:12:13.029327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/4067546353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4067546353},{"type":"v1","addr":"172.21.15.106:6837","nonce":4067546353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:12:14.042 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 43 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd9e806b230 msgr2=0x7fd9e806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd9e806b230 0x7fd9e806d6f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc0ca8f0 tx=0x7fd9f0008040 comp rx=0 tx=0).stop 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 msgr2=0x7fd9fc09a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9fc09a050 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec001050 tx=0x7fd9ec00ec80 comp rx=0 tx=0).stop 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 shutdown_connections 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd9e806b230 0x7fd9e806d6f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd9fc0ae650 0x7fd9fc0cb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9fc0ae1d0 0x7fd9fc0ad280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 --2- 172.21.15.106:0/3753252112 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd9e40052a0 0x7fd9fc09a050 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 >> 172.21.15.106:0/3753252112 conn(0x7fd9fc06a920 msgr2=0x7fd9fc06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 shutdown_connections 2023-11-24T08:12:14.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.043+0000 7fda0221e640 1 -- 172.21.15.106:0/3753252112 wait complete. 2023-11-24T08:12:14.065 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.281+0000 7f6315eda640 1 Processor -- start 2023-11-24T08:12:14.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.281+0000 7f6315eda640 1 -- start start 2023-11-24T08:12:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f6315eda640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f63100b2250 0x7f631015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f6315eda640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2630 0x7f63100ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f6315eda640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f631015ab80 con 0x7f63100cd470 2023-11-24T08:12:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f6315eda640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631015a990 con 0x7f63100b2250 2023-11-24T08:12:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f6315eda640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631015e680 con 0x7f63100b2630 2023-11-24T08:12:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f630f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f63100b2250 0x7f631015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f630f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f63100b2250 0x7f631015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43586/0 (socket says 172.21.15.106:43586) 2023-11-24T08:12:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.283+0000 7f630f7fe640 1 -- 172.21.15.106:0/1733275069 learned_addr learned my addr 172.21.15.106:0/1733275069 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:14.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.284+0000 7f630f7fe640 1 -- 172.21.15.106:0/1733275069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2630 msgr2=0x7f63100ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.284+0000 7f630f7fe640 1 --2- 172.21.15.106:0/1733275069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2630 0x7f63100ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.284+0000 7f630f7fe640 1 -- 172.21.15.106:0/1733275069 >> v1:172.21.15.106:6789/0 conn(0x7f63100cd470 legacy=0x7f631015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.284+0000 7f630f7fe640 1 -- 172.21.15.106:0/1733275069 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631015e960 con 0x7f63100b2250 2023-11-24T08:12:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.285+0000 7f630f7fe640 1 --2- 172.21.15.106:0/1733275069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100b2250 0x7f631015df80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6300009870 tx=0x7f630002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f583075404a3e840 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.285+0000 7f630effd640 1 -- 172.21.15.106:0/1733275069 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300030840 con 0x7f63100b2250 2023-11-24T08:12:14.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.285+0000 7f630effd640 1 -- 172.21.15.106:0/1733275069 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63000309a0 con 0x7f63100b2250 2023-11-24T08:12:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.285+0000 7f630effd640 1 -- 172.21.15.106:0/1733275069 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300030d30 con 0x7f63100b2250 2023-11-24T08:12:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 -- 172.21.15.106:0/1733275069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100b2250 msgr2=0x7f631015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 --2- 172.21.15.106:0/1733275069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100b2250 0x7f631015df80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6300009870 tx=0x7f630002b980 comp rx=0 tx=0).stop 2023-11-24T08:12:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 -- 172.21.15.106:0/1733275069 shutdown_connections 2023-11-24T08:12:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 --2- 172.21.15.106:0/1733275069 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2630 0x7f63100ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 --2- 172.21.15.106:0/1733275069 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100b2250 0x7f631015df80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 -- 172.21.15.106:0/1733275069 >> 172.21.15.106:0/1733275069 conn(0x7f631006a920 msgr2=0x7f631006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 -- 172.21.15.106:0/1733275069 shutdown_connections 2023-11-24T08:12:14.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.286+0000 7f6315eda640 1 -- 172.21.15.106:0/1733275069 wait complete. 2023-11-24T08:12:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.288+0000 7f6315eda640 1 Processor -- start 2023-11-24T08:12:14.289 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.288+0000 7f6315eda640 1 -- start start 2023-11-24T08:12:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6315eda640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2250 0x7f63101575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6315eda640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63100b2630 0x7f6310149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6315eda640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 0x7f63100c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6315eda640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63100ca1d0 con 0x7f63100cd470 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6315eda640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6310149bd0 con 0x7f63100b2630 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6315eda640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63100c9550 con 0x7f63100b2250 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 0x7f63100c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 0x7f63100c8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43598/0 (socket says 172.21.15.106:43598) 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 -- 172.21.15.106:0/3024323192 learned_addr learned my addr 172.21.15.106:0/3024323192 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630f7fe640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2250 0x7f63101575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f6306ffd640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63100b2630 0x7f6310149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 -- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2250 msgr2=0x7f63101575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2250 0x7f63101575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 -- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63100b2630 msgr2=0x7f6310149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63100b2630 0x7f6310149690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630ffff640 1 -- 172.21.15.106:0/3024323192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63000094d0 con 0x7f63100cd470 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.289+0000 7f630f7fe640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2250 0x7f63101575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.290+0000 7f630ffff640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 0x7f63100c8d30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f630800c730 tx=0x7f630800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.290+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308003b70 con 0x7f63100cd470 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.290+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6308003cf0 con 0x7f63100cd470 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.290+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6308003ea0 con 0x7f63100cd470 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.290+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63100c9830 con 0x7f63100cd470 2023-11-24T08:12:14.293 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.290+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63100c9df0 con 0x7f63100cd470 2023-11-24T08:12:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.291+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63100ca310 con 0x7f63100cd470 2023-11-24T08:12:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.292+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6308005310 con 0x7f63100cd470 2023-11-24T08:12:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.293+0000 7f630cff9640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f62f006b260 0x7f62f006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.294 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.293+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6308089b10 con 0x7f63100cd470 2023-11-24T08:12:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.293+0000 7f630f7fe640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f62f006b260 0x7f62f006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.295 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.294+0000 7f630f7fe640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f62f006b260 0x7f62f006d720 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6300036000 tx=0x7f630003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:14.299 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.298+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63080545f0 con 0x7f63100cd470 2023-11-24T08:12:14.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.489+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6310067470 con 0x7f63100cd470 2023-11-24T08:12:14.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.492+0000 7f630cff9640 1 -- 172.21.15.106:0/3024323192 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16616 (secure 0 0 0) 0x7f6308057c10 con 0x7f63100cd470 2023-11-24T08:12:14.494 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:14.494 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":75,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:11.536568+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:14.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.494+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f62f006b260 msgr2=0x7f62f006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.494+0000 7f6315eda640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f62f006b260 0x7f62f006d720 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f6300036000 tx=0x7f630003b040 comp rx=0 tx=0).stop 2023-11-24T08:12:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.494+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 msgr2=0x7f63100c8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.494+0000 7f6315eda640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 0x7f63100c8d30 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f630800c730 tx=0x7f630800cbf0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 shutdown_connections 2023-11-24T08:12:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63100cd470 0x7f63100c8d30 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63100b2630 0x7f6310149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f62f006b260 0x7f62f006d720 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 --2- 172.21.15.106:0/3024323192 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63100b2250 0x7f63101575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 >> 172.21.15.106:0/3024323192 conn(0x7f631006a920 msgr2=0x7f63100adfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 shutdown_connections 2023-11-24T08:12:14.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.495+0000 7f6315eda640 1 -- 172.21.15.106:0/3024323192 wait complete. 2023-11-24T08:12:14.516 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:12:14.516 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:12:14.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.727+0000 7f8c34aa6640 1 Processor -- start 2023-11-24T08:12:14.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.727+0000 7f8c34aa6640 1 -- start start 2023-11-24T08:12:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.728+0000 7f8c34aa6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 0x7f8c3015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.728+0000 7f8c34aa6640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8c300b1de0 0x7f8c300b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c34aa6640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c3015bf80 con 0x7f8c3015c410 2023-11-24T08:12:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c34aa6640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c300b27b0 con 0x7f8c300b1de0 2023-11-24T08:12:14.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c34aa6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b1560 con 0x7f8c30144b30 2023-11-24T08:12:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c27fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8c3015c410 0x7f8c300b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51834/0 (socket says 172.21.15.106:51834) 2023-11-24T08:12:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c27fff640 1 -- 172.21.15.106:0/1192201751 learned_addr learned my addr 172.21.15.106:0/1192201751 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c2ffff640 1 --2- 172.21.15.106:0/1192201751 >> v2:172.21.15.156:3300/0 conn(0x7f8c300b1de0 0x7f8c300b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 0x7f8c3015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522093208 0 0) 0x7f8c3015bf80 con 0x7f8c3015c410 2023-11-24T08:12:14.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.729+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c1c003520 con 0x7f8c3015c410 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3783664678 0 0) 0x7f8c1c003520 con 0x7f8c3015c410 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 msgr2=0x7f8c3015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 0x7f8c3015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 >> v2:172.21.15.156:3300/0 conn(0x7f8c300b1de0 msgr2=0x7f8c300b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 --2- 172.21.15.106:0/1192201751 >> v2:172.21.15.156:3300/0 conn(0x7f8c300b1de0 0x7f8c300b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c300b1280 con 0x7f8c3015c410 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f8c18001670 con 0x7f8c3015c410 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 >> v1:172.21.15.106:6789/0 conn(0x7f8c3015c410 legacy=0x7f8c300b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1c003970 0x7f8c1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 0x7f8c1c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c004d00 0x7f8c1c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b27b0 con 0x7f8c1c003970 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c300b1560 con 0x7f8c1c004d00 2023-11-24T08:12:14.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c00b980 con 0x7f8c30144b30 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1c003970 0x7f8c1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2ffff640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c004d00 0x7f8c1c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c27fff640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 0x7f8c1c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.730+0000 7f8c2f7fe640 1 -- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 msgr2=0x7f8c1c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.731+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c30144b30 0x7f8c1c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.731+0000 7f8c2f7fe640 1 -- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c004d00 msgr2=0x7f8c1c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.731+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c004d00 0x7f8c1c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.731+0000 7f8c2f7fe640 1 -- 172.21.15.106:0/1192201751 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c003520 con 0x7f8c1c003970 2023-11-24T08:12:14.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.731+0000 7f8c2ffff640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c004d00 0x7f8c1c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:14.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1c003970 0x7f8c1c003d90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000ba60 tx=0x7f8c1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cc297e0e412d30 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:14.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20032630 con 0x7f8c1c003970 2023-11-24T08:12:14.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c20032790 con 0x7f8c1c003970 2023-11-24T08:12:14.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c2effd640 1 -- 172.21.15.106:0/1192201751 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20032b20 con 0x7f8c1c003970 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1192201751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1c003970 msgr2=0x7f8c1c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1192201751 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c1c003970 0x7f8c1c003d90 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000ba60 tx=0x7f8c1c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.732+0000 7f8c2ffff640 1 -- 172.21.15.106:0/1192201751 reap_dead start 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.733+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1192201751 shutdown_connections 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.733+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1192201751 >> 172.21.15.106:0/1192201751 conn(0x7f8c300cb830 msgr2=0x7f8c300cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.733+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1192201751 shutdown_connections 2023-11-24T08:12:14.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.733+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1192201751 wait complete. 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.734+0000 7f8c34aa6640 1 Processor -- start 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 -- start start 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c003970 0x7f8c30154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c3015af10 0x7f8c301f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 0x7f8c300ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301568f0 con 0x7f8c3013e9d0 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c301f75a0 con 0x7f8c1c003970 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.735+0000 7f8c34aa6640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c30143400 con 0x7f8c3015af10 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 0x7f8c300ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 0x7f8c300ccc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43618/0 (socket says 172.21.15.106:43618) 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 -- 172.21.15.106:0/1973584320 learned_addr learned my addr 172.21.15.106:0/1973584320 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c003970 0x7f8c30154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 -- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c3015af10 msgr2=0x7f8c301f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c27fff640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c3015af10 0x7f8c301f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c3015af10 0x7f8c301f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 -- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c003970 msgr2=0x7f8c30154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c003970 0x7f8c30154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2ffff640 1 -- 172.21.15.106:0/1973584320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30142550 con 0x7f8c3013e9d0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.736+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c003970 0x7f8c30154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.737+0000 7f8c2ffff640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 0x7f8c300ccc90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8c300cc860 tx=0x7f8c2800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.737+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28017400 con 0x7f8c3013e9d0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.737+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c28011040 con 0x7f8c3013e9d0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.737+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28017780 con 0x7f8c3013e9d0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.737+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c301427f0 con 0x7f8c3013e9d0 2023-11-24T08:12:14.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.737+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c30142db0 con 0x7f8c3013e9d0 2023-11-24T08:12:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.738+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c30143590 con 0x7f8c3013e9d0 2023-11-24T08:12:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.739+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f8c280203b0 con 0x7f8c3013e9d0 2023-11-24T08:12:14.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.740+0000 7f8c2cff9640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c1006b260 0x7f8c1006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.740+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c2808a0c0 con 0x7f8c3013e9d0 2023-11-24T08:12:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.740+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c1006b260 0x7f8c1006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:14.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.741+0000 7f8c2f7fe640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c1006b260 0x7f8c1006d720 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000ba30 tx=0x7f8c2003d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:14.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.745+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c28054ad0 con 0x7f8c3013e9d0 2023-11-24T08:12:14.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.931+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8c3015bc10 con 0x7f8c1006b260 2023-11-24T08:12:14.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.955+0000 7f8c2cff9640 1 -- 172.21.15.106:0/1973584320 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282238 (secure 0 0 0) 0x7f8c20032ad0 con 0x7f8c1006b260 2023-11-24T08:12:14.960 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:14.961 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c1006b260 msgr2=0x7f8c1006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c1006b260 0x7f8c1006d720 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8c2000ba30 tx=0x7f8c2003d040 comp rx=0 tx=0).stop 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 msgr2=0x7f8c300ccc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 0x7f8c300ccc90 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f8c300cc860 tx=0x7f8c2800f4a0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 shutdown_connections 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c3013e9d0 0x7f8c300ccc90 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c1006b260 0x7f8c1006d720 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c3015af10 0x7f8c301f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 --2- 172.21.15.106:0/1973584320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c1c003970 0x7f8c30154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 >> 172.21.15.106:0/1973584320 conn(0x7f8c300cb830 msgr2=0x7f8c3013fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 shutdown_connections 2023-11-24T08:12:14.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:14.960+0000 7f8c34aa6640 1 -- 172.21.15.106:0/1973584320 wait complete. 2023-11-24T08:12:14.980 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2023-11-24T08:12:13.335995+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249184,"kb_used_data":8060,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725536,"statfs":{"total":773094113280,"available":772838948864,"internally_reserved":0,"allocated":8253440,"data_stored":4150285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.845910"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524784+0000","last_change":"2023-11-24T08:12:09.524784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524784+0000","last_undegraded":"2023-11-24T08:12:09.524784+0000","last_fullsized":"2023-11-24T08:12:09.524784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517595+0000","last_change":"2023-11-24T08:12:09.517595+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517595+0000","last_undegraded":"2023-11-24T08:12:09.517595+0000","last_fullsized":"2023-11-24T08:12:09.517595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508940+0000","last_change":"2023-11-24T08:12:09.508940+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508940+0000","last_undegraded":"2023-11-24T08:12:09.508940+0000","last_fullsized":"2023-11-24T08:12:09.508940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508376+0000","last_change":"2023-11-24T08:12:09.508376+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508376+0000","last_undegraded":"2023-11-24T08:12:09.508376+0000","last_fullsized":"2023-11-24T08:12:09.508376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511217+0000","last_change":"2023-11-24T08:12:09.511217+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511217+0000","last_undegraded":"2023-11-24T08:12:09.511217+0000","last_fullsized":"2023-11-24T08:12:09.511217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.514317+0000","last_change":"2023-11-24T08:12:09.514317+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.514317+0000","last_undegraded":"2023-11-24T08:12:09.514317+0000","last_fullsized":"2023-11-24T08:12:09.514317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.532518+0000","last_change":"2023-11-24T08:12:10.532518+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.532518+0000","last_undegraded":"2023-11-24T08:12:10.532518+0000","last_fullsized":"2023-11-24T08:12:10.532518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:09.759327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511431+0000","last_change":"2023-11-24T08:12:09.511431+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511431+0000","last_undegraded":"2023-11-24T08:12:09.511431+0000","last_fullsized":"2023-11-24T08:12:09.511431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510124+0000","last_change":"2023-11-24T08:12:09.510124+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510124+0000","last_undegraded":"2023-11-24T08:12:09.510124+0000","last_fullsized":"2023-11-24T08:12:09.510124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505932+0000","last_change":"2023-11-24T08:12:09.505932+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505932+0000","last_undegraded":"2023-11-24T08:12:09.505932+0000","last_fullsized":"2023-11-24T08:12:09.505932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523117+0000","last_change":"2023-11-24T08:12:09.523117+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523117+0000","last_undegraded":"2023-11-24T08:12:09.523117+0000","last_fullsized":"2023-11-24T08:12:09.523117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507941+0000","last_change":"2023-11-24T08:12:09.507941+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507941+0000","last_undegraded":"2023-11-24T08:12:09.507941+0000","last_fullsized":"2023-11-24T08:12:09.507941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516971+0000","last_change":"2023-11-24T08:12:09.516971+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516971+0000","last_undegraded":"2023-11-24T08:12:09.516971+0000","last_fullsized":"2023-11-24T08:12:09.516971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511315+0000","last_change":"2023-11-24T08:12:09.511315+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511315+0000","last_undegraded":"2023-11-24T08:12:09.511315+0000","last_fullsized":"2023-11-24T08:12:09.511315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519786+0000","last_change":"2023-11-24T08:12:09.519786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519786+0000","last_undegraded":"2023-11-24T08:12:09.519786+0000","last_fullsized":"2023-11-24T08:12:09.519786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503356+0000","last_change":"2023-11-24T08:12:09.503356+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503356+0000","last_undegraded":"2023-11-24T08:12:09.503356+0000","last_fullsized":"2023-11-24T08:12:09.503356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509693+0000","last_change":"2023-11-24T08:12:09.509693+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509693+0000","last_undegraded":"2023-11-24T08:12:09.509693+0000","last_fullsized":"2023-11-24T08:12:09.509693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506814+0000","last_change":"2023-11-24T08:12:09.506814+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506814+0000","last_undegraded":"2023-11-24T08:12:09.506814+0000","last_fullsized":"2023-11-24T08:12:09.506814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524401+0000","last_change":"2023-11-24T08:12:09.524401+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524401+0000","last_undegraded":"2023-11-24T08:12:09.524401+0000","last_fullsized":"2023-11-24T08:12:09.524401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516989+0000","last_change":"2023-11-24T08:12:09.516989+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516989+0000","last_undegraded":"2023-11-24T08:12:09.516989+0000","last_fullsized":"2023-11-24T08:12:09.516989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506189+0000","last_change":"2023-11-24T08:12:09.506189+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506189+0000","last_undegraded":"2023-11-24T08:12:09.506189+0000","last_fullsized":"2023-11-24T08:12:09.506189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509465+0000","last_change":"2023-11-24T08:12:09.509465+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509465+0000","last_undegraded":"2023-11-24T08:12:09.509465+0000","last_fullsized":"2023-11-24T08:12:09.509465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520990+0000","last_change":"2023-11-24T08:12:09.520990+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520990+0000","last_undegraded":"2023-11-24T08:12:09.520990+0000","last_fullsized":"2023-11-24T08:12:09.520990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524248+0000","last_change":"2023-11-24T08:12:09.524248+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524248+0000","last_undegraded":"2023-11-24T08:12:09.524248+0000","last_fullsized":"2023-11-24T08:12:09.524248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510034+0000","last_change":"2023-11-24T08:12:09.510034+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510034+0000","last_undegraded":"2023-11-24T08:12:09.510034+0000","last_fullsized":"2023-11-24T08:12:09.510034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.531598+0000","last_change":"2023-11-24T08:12:10.531598+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.531598+0000","last_undegraded":"2023-11-24T08:12:10.531598+0000","last_fullsized":"2023-11-24T08:12:10.531598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:31.654975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508296+0000","last_change":"2023-11-24T08:12:09.508296+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508296+0000","last_undegraded":"2023-11-24T08:12:09.508296+0000","last_fullsized":"2023-11-24T08:12:09.508296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503786+0000","last_change":"2023-11-24T08:12:09.503786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503786+0000","last_undegraded":"2023-11-24T08:12:09.503786+0000","last_fullsized":"2023-11-24T08:12:09.503786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511659+0000","last_change":"2023-11-24T08:12:09.511659+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511659+0000","last_undegraded":"2023-11-24T08:12:09.511659+0000","last_fullsized":"2023-11-24T08:12:09.511659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.544012+0000","last_change":"2023-11-24T08:12:10.544012+0000","last_active":"2023-11-24T08:12:10.544012+0000","last_peered":"2023-11-24T08:12:10.544012+0000","last_clean":"2023-11-24T08:12:10.544012+0000","last_became_active":"2023-11-24T08:12:10.543658+0000","last_became_peered":"2023-11-24T08:12:10.543658+0000","last_unstale":"2023-11-24T08:12:10.544012+0000","last_undegraded":"2023-11-24T08:12:10.544012+0000","last_fullsized":"2023-11-24T08:12:10.544012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:34.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.524767+0000","last_change":"2023-11-24T08:12:10.524767+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.524767+0000","last_undegraded":"2023-11-24T08:12:10.524767+0000","last_fullsized":"2023-11-24T08:12:10.524767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:21:52.679838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507818+0000","last_change":"2023-11-24T08:12:09.507818+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507818+0000","last_undegraded":"2023-11-24T08:12:09.507818+0000","last_fullsized":"2023-11-24T08:12:09.507818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520772+0000","last_change":"2023-11-24T08:12:09.520772+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520772+0000","last_undegraded":"2023-11-24T08:12:09.520772+0000","last_fullsized":"2023-11-24T08:12:09.520772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537413+0000","last_change":"2023-11-24T08:12:09.535765+0000","last_active":"2023-11-24T08:12:09.537413+0000","last_peered":"2023-11-24T08:12:09.537413+0000","last_clean":"2023-11-24T08:12:09.537413+0000","last_became_active":"2023-11-24T08:12:09.535385+0000","last_became_peered":"2023-11-24T08:12:09.535385+0000","last_unstale":"2023-11-24T08:12:09.537413+0000","last_undegraded":"2023-11-24T08:12:09.537413+0000","last_fullsized":"2023-11-24T08:12:09.537413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:18.985711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509116+0000","last_change":"2023-11-24T08:12:09.509116+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509116+0000","last_undegraded":"2023-11-24T08:12:09.509116+0000","last_fullsized":"2023-11-24T08:12:09.509116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530987+0000","last_change":"2023-11-24T08:12:09.524544+0000","last_active":"2023-11-24T08:12:09.530987+0000","last_peered":"2023-11-24T08:12:09.530987+0000","last_clean":"2023-11-24T08:12:09.530987+0000","last_became_active":"2023-11-24T08:12:09.524188+0000","last_became_peered":"2023-11-24T08:12:09.524188+0000","last_unstale":"2023-11-24T08:12:09.530987+0000","last_undegraded":"2023-11-24T08:12:09.530987+0000","last_fullsized":"2023-11-24T08:12:09.530987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:56.808696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524895+0000","last_change":"2023-11-24T08:12:09.524895+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524895+0000","last_undegraded":"2023-11-24T08:12:09.524895+0000","last_fullsized":"2023-11-24T08:12:09.524895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538199+0000","last_change":"2023-11-24T08:12:09.533833+0000","last_active":"2023-11-24T08:12:09.538199+0000","last_peered":"2023-11-24T08:12:09.538199+0000","last_clean":"2023-11-24T08:12:09.538199+0000","last_became_active":"2023-11-24T08:12:09.533263+0000","last_became_peered":"2023-11-24T08:12:09.533263+0000","last_unstale":"2023-11-24T08:12:09.538199+0000","last_undegraded":"2023-11-24T08:12:09.538199+0000","last_fullsized":"2023-11-24T08:12:09.538199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:50.328328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522446+0000","last_change":"2023-11-24T08:12:09.522446+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522446+0000","last_undegraded":"2023-11-24T08:12:09.522446+0000","last_fullsized":"2023-11-24T08:12:09.522446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538124+0000","last_change":"2023-11-24T08:12:09.529735+0000","last_active":"2023-11-24T08:12:09.538124+0000","last_peered":"2023-11-24T08:12:09.538124+0000","last_clean":"2023-11-24T08:12:09.538124+0000","last_became_active":"2023-11-24T08:12:09.529517+0000","last_became_peered":"2023-11-24T08:12:09.529517+0000","last_unstale":"2023-11-24T08:12:09.538124+0000","last_undegraded":"2023-11-24T08:12:09.538124+0000","last_fullsized":"2023-11-24T08:12:09.538124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:14.043315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.525674+0000","last_change":"2023-11-24T08:12:10.525674+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.525674+0000","last_undegraded":"2023-11-24T08:12:10.525674+0000","last_fullsized":"2023-11-24T08:12:10.525674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:27.094673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526557+0000","last_change":"2023-11-24T08:12:09.522576+0000","last_active":"2023-11-24T08:12:09.526557+0000","last_peered":"2023-11-24T08:12:09.526557+0000","last_clean":"2023-11-24T08:12:09.526557+0000","last_became_active":"2023-11-24T08:12:09.522114+0000","last_became_peered":"2023-11-24T08:12:09.522114+0000","last_unstale":"2023-11-24T08:12:09.526557+0000","last_undegraded":"2023-11-24T08:12:09.526557+0000","last_fullsized":"2023-11-24T08:12:09.526557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:27:40.788887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503392+0000","last_change":"2023-11-24T08:12:09.503392+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503392+0000","last_undegraded":"2023-11-24T08:12:09.503392+0000","last_fullsized":"2023-11-24T08:12:09.503392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537905+0000","last_change":"2023-11-24T08:12:09.531869+0000","last_active":"2023-11-24T08:12:09.537905+0000","last_peered":"2023-11-24T08:12:09.537905+0000","last_clean":"2023-11-24T08:12:09.537905+0000","last_became_active":"2023-11-24T08:12:09.531598+0000","last_became_peered":"2023-11-24T08:12:09.531598+0000","last_unstale":"2023-11-24T08:12:09.537905+0000","last_undegraded":"2023-11-24T08:12:09.537905+0000","last_fullsized":"2023-11-24T08:12:09.537905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:24.358046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510090+0000","last_change":"2023-11-24T08:12:09.510090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510090+0000","last_undegraded":"2023-11-24T08:12:09.510090+0000","last_fullsized":"2023-11-24T08:12:09.510090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518079+0000","last_change":"2023-11-24T08:12:09.513949+0000","last_active":"2023-11-24T08:12:09.518079+0000","last_peered":"2023-11-24T08:12:09.518079+0000","last_clean":"2023-11-24T08:12:09.518079+0000","last_became_active":"2023-11-24T08:12:09.513517+0000","last_became_peered":"2023-11-24T08:12:09.513517+0000","last_unstale":"2023-11-24T08:12:09.518079+0000","last_undegraded":"2023-11-24T08:12:09.518079+0000","last_fullsized":"2023-11-24T08:12:09.518079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:51.436372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517039+0000","last_change":"2023-11-24T08:12:09.517039+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517039+0000","last_undegraded":"2023-11-24T08:12:09.517039+0000","last_fullsized":"2023-11-24T08:12:09.517039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.524795+0000","last_change":"2023-11-24T08:12:09.519208+0000","last_active":"2023-11-24T08:12:09.524795+0000","last_peered":"2023-11-24T08:12:09.524795+0000","last_clean":"2023-11-24T08:12:09.524795+0000","last_became_active":"2023-11-24T08:12:09.518952+0000","last_became_peered":"2023-11-24T08:12:09.518952+0000","last_unstale":"2023-11-24T08:12:09.524795+0000","last_undegraded":"2023-11-24T08:12:09.524795+0000","last_fullsized":"2023-11-24T08:12:09.524795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:57:11.824082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.542676+0000","last_change":"2023-11-24T08:12:10.542676+0000","last_active":"2023-11-24T08:12:10.542676+0000","last_peered":"2023-11-24T08:12:10.542676+0000","last_clean":"2023-11-24T08:12:10.542676+0000","last_became_active":"2023-11-24T08:12:10.542269+0000","last_became_peered":"2023-11-24T08:12:10.542269+0000","last_unstale":"2023-11-24T08:12:10.542676+0000","last_undegraded":"2023-11-24T08:12:10.542676+0000","last_fullsized":"2023-11-24T08:12:10.542676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:38.761881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.525878+0000","last_change":"2023-11-24T08:12:09.519666+0000","last_active":"2023-11-24T08:12:09.525878+0000","last_peered":"2023-11-24T08:12:09.525878+0000","last_clean":"2023-11-24T08:12:09.525878+0000","last_became_active":"2023-11-24T08:12:09.519348+0000","last_became_peered":"2023-11-24T08:12:09.519348+0000","last_unstale":"2023-11-24T08:12:09.525878+0000","last_undegraded":"2023-11-24T08:12:09.525878+0000","last_fullsized":"2023-11-24T08:12:09.525878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:08.548384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524304+0000","last_change":"2023-11-24T08:12:09.524304+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524304+0000","last_undegraded":"2023-11-24T08:12:09.524304+0000","last_fullsized":"2023-11-24T08:12:09.524304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538991+0000","last_change":"2023-11-24T08:12:09.537826+0000","last_active":"2023-11-24T08:12:09.538991+0000","last_peered":"2023-11-24T08:12:09.538991+0000","last_clean":"2023-11-24T08:12:09.538991+0000","last_became_active":"2023-11-24T08:12:09.537555+0000","last_became_peered":"2023-11-24T08:12:09.537555+0000","last_unstale":"2023-11-24T08:12:09.538991+0000","last_undegraded":"2023-11-24T08:12:09.538991+0000","last_fullsized":"2023-11-24T08:12:09.538991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:54:04.166572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.515379+0000","last_change":"2023-11-24T08:12:09.515379+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.515379+0000","last_undegraded":"2023-11-24T08:12:09.515379+0000","last_fullsized":"2023-11-24T08:12:09.515379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531034+0000","last_change":"2023-11-24T08:12:09.524384+0000","last_active":"2023-11-24T08:12:09.531034+0000","last_peered":"2023-11-24T08:12:09.531034+0000","last_clean":"2023-11-24T08:12:09.531034+0000","last_became_active":"2023-11-24T08:12:09.524038+0000","last_became_peered":"2023-11-24T08:12:09.524038+0000","last_unstale":"2023-11-24T08:12:09.531034+0000","last_undegraded":"2023-11-24T08:12:09.531034+0000","last_fullsized":"2023-11-24T08:12:09.531034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:45.336925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.513394+0000","last_change":"2023-11-24T08:12:09.513394+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.513394+0000","last_undegraded":"2023-11-24T08:12:09.513394+0000","last_fullsized":"2023-11-24T08:12:09.513394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518054+0000","last_change":"2023-11-24T08:12:09.514045+0000","last_active":"2023-11-24T08:12:09.518054+0000","last_peered":"2023-11-24T08:12:09.518054+0000","last_clean":"2023-11-24T08:12:09.518054+0000","last_became_active":"2023-11-24T08:12:09.513337+0000","last_became_peered":"2023-11-24T08:12:09.513337+0000","last_unstale":"2023-11-24T08:12:09.518054+0000","last_undegraded":"2023-11-24T08:12:09.518054+0000","last_fullsized":"2023-11-24T08:12:09.518054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:17.346075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.518834+0000","last_change":"2023-11-24T08:12:09.518834+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.518834+0000","last_undegraded":"2023-11-24T08:12:09.518834+0000","last_fullsized":"2023-11-24T08:12:09.518834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530522+0000","last_change":"2023-11-24T08:12:09.528456+0000","last_active":"2023-11-24T08:12:09.530522+0000","last_peered":"2023-11-24T08:12:09.530522+0000","last_clean":"2023-11-24T08:12:09.530522+0000","last_became_active":"2023-11-24T08:12:09.528140+0000","last_became_peered":"2023-11-24T08:12:09.528140+0000","last_unstale":"2023-11-24T08:12:09.530522+0000","last_undegraded":"2023-11-24T08:12:09.530522+0000","last_fullsized":"2023-11-24T08:12:09.530522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:12.633925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.529872+0000","last_change":"2023-11-24T08:12:10.529872+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.529872+0000","last_undegraded":"2023-11-24T08:12:10.529872+0000","last_fullsized":"2023-11-24T08:12:10.529872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:09.099806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538979+0000","last_change":"2023-11-24T08:12:09.537450+0000","last_active":"2023-11-24T08:12:09.538979+0000","last_peered":"2023-11-24T08:12:09.538979+0000","last_clean":"2023-11-24T08:12:09.538979+0000","last_became_active":"2023-11-24T08:12:09.537196+0000","last_became_peered":"2023-11-24T08:12:09.537196+0000","last_unstale":"2023-11-24T08:12:09.538979+0000","last_undegraded":"2023-11-24T08:12:09.538979+0000","last_fullsized":"2023-11-24T08:12:09.538979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:19.005358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517657+0000","last_change":"2023-11-24T08:12:09.517657+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517657+0000","last_undegraded":"2023-11-24T08:12:09.517657+0000","last_fullsized":"2023-11-24T08:12:09.517657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533269+0000","last_change":"2023-11-24T08:12:09.530752+0000","last_active":"2023-11-24T08:12:10.533269+0000","last_peered":"2023-11-24T08:12:10.533269+0000","last_clean":"2023-11-24T08:12:10.533269+0000","last_became_active":"2023-11-24T08:12:09.529282+0000","last_became_peered":"2023-11-24T08:12:09.529282+0000","last_unstale":"2023-11-24T08:12:10.533269+0000","last_undegraded":"2023-11-24T08:12:10.533269+0000","last_fullsized":"2023-11-24T08:12:10.533269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:31.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510752+0000","last_change":"2023-11-24T08:12:09.510752+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510752+0000","last_undegraded":"2023-11-24T08:12:09.510752+0000","last_fullsized":"2023-11-24T08:12:09.510752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.534559+0000","last_change":"2023-11-24T08:12:09.530860+0000","last_active":"2023-11-24T08:12:09.534559+0000","last_peered":"2023-11-24T08:12:09.534559+0000","last_clean":"2023-11-24T08:12:09.534559+0000","last_became_active":"2023-11-24T08:12:09.530167+0000","last_became_peered":"2023-11-24T08:12:09.530167+0000","last_unstale":"2023-11-24T08:12:09.534559+0000","last_undegraded":"2023-11-24T08:12:09.534559+0000","last_fullsized":"2023-11-24T08:12:09.534559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:08.559029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.504790+0000","last_change":"2023-11-24T08:12:09.504790+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.504790+0000","last_undegraded":"2023-11-24T08:12:09.504790+0000","last_fullsized":"2023-11-24T08:12:09.504790+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.539017+0000","last_change":"2023-11-24T08:12:09.538063+0000","last_active":"2023-11-24T08:12:09.539017+0000","last_peered":"2023-11-24T08:12:09.539017+0000","last_clean":"2023-11-24T08:12:09.539017+0000","last_became_active":"2023-11-24T08:12:09.536932+0000","last_became_peered":"2023-11-24T08:12:09.536932+0000","last_unstale":"2023-11-24T08:12:09.539017+0000","last_undegraded":"2023-11-24T08:12:09.539017+0000","last_fullsized":"2023-11-24T08:12:09.539017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:31.591358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523784+0000","last_change":"2023-11-24T08:12:09.523784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523784+0000","last_undegraded":"2023-11-24T08:12:09.523784+0000","last_fullsized":"2023-11-24T08:12:09.523784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538170+0000","last_change":"2023-11-24T08:12:09.533762+0000","last_active":"2023-11-24T08:12:09.538170+0000","last_peered":"2023-11-24T08:12:09.538170+0000","last_clean":"2023-11-24T08:12:09.538170+0000","last_became_active":"2023-11-24T08:12:09.533465+0000","last_became_peered":"2023-11-24T08:12:09.533465+0000","last_unstale":"2023-11-24T08:12:09.538170+0000","last_undegraded":"2023-11-24T08:12:09.538170+0000","last_fullsized":"2023-11-24T08:12:09.538170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:44.590676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505924+0000","last_change":"2023-11-24T08:12:09.505924+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505924+0000","last_undegraded":"2023-11-24T08:12:09.505924+0000","last_fullsized":"2023-11-24T08:12:09.505924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537888+0000","last_change":"2023-11-24T08:12:09.530831+0000","last_active":"2023-11-24T08:12:09.537888+0000","last_peered":"2023-11-24T08:12:09.537888+0000","last_clean":"2023-11-24T08:12:09.537888+0000","last_became_active":"2023-11-24T08:12:09.530459+0000","last_became_peered":"2023-11-24T08:12:09.530459+0000","last_unstale":"2023-11-24T08:12:09.537888+0000","last_undegraded":"2023-11-24T08:12:09.537888+0000","last_fullsized":"2023-11-24T08:12:09.537888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:40.045785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509492+0000","last_change":"2023-11-24T08:12:09.509492+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509492+0000","last_undegraded":"2023-11-24T08:12:09.509492+0000","last_fullsized":"2023-11-24T08:12:09.509492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537541+0000","last_change":"2023-11-24T08:12:09.535763+0000","last_active":"2023-11-24T08:12:09.537541+0000","last_peered":"2023-11-24T08:12:09.537541+0000","last_clean":"2023-11-24T08:12:09.537541+0000","last_became_active":"2023-11-24T08:12:09.535048+0000","last_became_peered":"2023-11-24T08:12:09.535048+0000","last_unstale":"2023-11-24T08:12:09.537541+0000","last_undegraded":"2023-11-24T08:12:09.537541+0000","last_fullsized":"2023-11-24T08:12:09.537541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:42.819752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511207+0000","last_change":"2023-11-24T08:12:09.511207+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511207+0000","last_undegraded":"2023-11-24T08:12:09.511207+0000","last_fullsized":"2023-11-24T08:12:09.511207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538446+0000","last_change":"2023-11-24T08:12:09.536935+0000","last_active":"2023-11-24T08:12:09.538446+0000","last_peered":"2023-11-24T08:12:09.538446+0000","last_clean":"2023-11-24T08:12:09.538446+0000","last_became_active":"2023-11-24T08:12:09.535354+0000","last_became_peered":"2023-11-24T08:12:09.535354+0000","last_unstale":"2023-11-24T08:12:09.538446+0000","last_undegraded":"2023-11-24T08:12:09.538446+0000","last_fullsized":"2023-11-24T08:12:09.538446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:49.306476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507471+0000","last_change":"2023-11-24T08:12:09.507471+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507471+0000","last_undegraded":"2023-11-24T08:12:09.507471+0000","last_fullsized":"2023-11-24T08:12:09.507471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533487+0000","last_change":"2023-11-24T08:12:09.530273+0000","last_active":"2023-11-24T08:12:10.533487+0000","last_peered":"2023-11-24T08:12:10.533487+0000","last_clean":"2023-11-24T08:12:10.533487+0000","last_became_active":"2023-11-24T08:12:09.529967+0000","last_became_peered":"2023-11-24T08:12:09.529967+0000","last_unstale":"2023-11-24T08:12:10.533487+0000","last_undegraded":"2023-11-24T08:12:10.533487+0000","last_fullsized":"2023-11-24T08:12:10.533487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:37.818727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508506+0000","last_change":"2023-11-24T08:12:09.508506+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508506+0000","last_undegraded":"2023-11-24T08:12:09.508506+0000","last_fullsized":"2023-11-24T08:12:09.508506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.535270+0000","last_change":"2023-11-24T08:12:09.530296+0000","last_active":"2023-11-24T08:12:09.535270+0000","last_peered":"2023-11-24T08:12:09.535270+0000","last_clean":"2023-11-24T08:12:09.535270+0000","last_became_active":"2023-11-24T08:12:09.529840+0000","last_became_peered":"2023-11-24T08:12:09.529840+0000","last_unstale":"2023-11-24T08:12:09.535270+0000","last_undegraded":"2023-11-24T08:12:09.535270+0000","last_fullsized":"2023-11-24T08:12:09.535270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:42.718534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519744+0000","last_change":"2023-11-24T08:12:09.519744+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519744+0000","last_undegraded":"2023-11-24T08:12:09.519744+0000","last_fullsized":"2023-11-24T08:12:09.519744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538516+0000","last_change":"2023-11-24T08:12:09.536994+0000","last_active":"2023-11-24T08:12:09.538516+0000","last_peered":"2023-11-24T08:12:09.538516+0000","last_clean":"2023-11-24T08:12:09.538516+0000","last_became_active":"2023-11-24T08:12:09.536664+0000","last_became_peered":"2023-11-24T08:12:09.536664+0000","last_unstale":"2023-11-24T08:12:09.538516+0000","last_undegraded":"2023-11-24T08:12:09.538516+0000","last_fullsized":"2023-11-24T08:12:09.538516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:10.151500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.533267+0000","last_change":"2023-11-24T08:12:10.533267+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.533267+0000","last_undegraded":"2023-11-24T08:12:10.533267+0000","last_fullsized":"2023-11-24T08:12:10.533267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:10.427272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.534767+0000","last_change":"2023-11-24T08:12:09.534163+0000","last_active":"2023-11-24T08:12:10.534767+0000","last_peered":"2023-11-24T08:12:10.534767+0000","last_clean":"2023-11-24T08:12:10.534767+0000","last_became_active":"2023-11-24T08:12:09.533384+0000","last_became_peered":"2023-11-24T08:12:09.533384+0000","last_unstale":"2023-11-24T08:12:10.534767+0000","last_undegraded":"2023-11-24T08:12:10.534767+0000","last_fullsized":"2023-11-24T08:12:10.534767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:39.337647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522771+0000","last_change":"2023-11-24T08:12:09.522771+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522771+0000","last_undegraded":"2023-11-24T08:12:09.522771+0000","last_fullsized":"2023-11-24T08:12:09.522771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.529558+0000","last_change":"2023-11-24T08:12:09.524480+0000","last_active":"2023-11-24T08:12:09.529558+0000","last_peered":"2023-11-24T08:12:09.529558+0000","last_clean":"2023-11-24T08:12:09.529558+0000","last_became_active":"2023-11-24T08:12:09.524212+0000","last_became_peered":"2023-11-24T08:12:09.524212+0000","last_unstale":"2023-11-24T08:12:09.529558+0000","last_undegraded":"2023-11-24T08:12:09.529558+0000","last_fullsized":"2023-11-24T08:12:09.529558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:57.611814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520763+0000","last_change":"2023-11-24T08:12:09.520763+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520763+0000","last_undegraded":"2023-11-24T08:12:09.520763+0000","last_fullsized":"2023-11-24T08:12:09.520763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.532684+0000","last_change":"2023-11-24T08:12:09.530037+0000","last_active":"2023-11-24T08:12:09.532684+0000","last_peered":"2023-11-24T08:12:09.532684+0000","last_clean":"2023-11-24T08:12:09.532684+0000","last_became_active":"2023-11-24T08:12:09.529803+0000","last_became_peered":"2023-11-24T08:12:09.529803+0000","last_unstale":"2023-11-24T08:12:09.532684+0000","last_undegraded":"2023-11-24T08:12:09.532684+0000","last_fullsized":"2023-11-24T08:12:09.532684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:48.699065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510858+0000","last_change":"2023-11-24T08:12:09.510858+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510858+0000","last_undegraded":"2023-11-24T08:12:09.510858+0000","last_fullsized":"2023-11-24T08:12:09.510858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526913+0000","last_change":"2023-11-24T08:12:09.522451+0000","last_active":"2023-11-24T08:12:09.526913+0000","last_peered":"2023-11-24T08:12:09.526913+0000","last_clean":"2023-11-24T08:12:09.526913+0000","last_became_active":"2023-11-24T08:12:09.521531+0000","last_became_peered":"2023-11-24T08:12:09.521531+0000","last_unstale":"2023-11-24T08:12:09.526913+0000","last_undegraded":"2023-11-24T08:12:09.526913+0000","last_fullsized":"2023-11-24T08:12:09.526913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.342039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516090+0000","last_change":"2023-11-24T08:12:09.516090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516090+0000","last_undegraded":"2023-11-24T08:12:09.516090+0000","last_fullsized":"2023-11-24T08:12:09.516090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.527191+0000","last_change":"2023-11-24T08:12:09.521396+0000","last_active":"2023-11-24T08:12:09.527191+0000","last_peered":"2023-11-24T08:12:09.527191+0000","last_clean":"2023-11-24T08:12:09.527191+0000","last_became_active":"2023-11-24T08:12:09.520385+0000","last_became_peered":"2023-11-24T08:12:09.520385+0000","last_unstale":"2023-11-24T08:12:09.527191+0000","last_undegraded":"2023-11-24T08:12:09.527191+0000","last_fullsized":"2023-11-24T08:12:09.527191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:25.592586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506002+0000","last_change":"2023-11-24T08:12:09.506002+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506002+0000","last_undegraded":"2023-11-24T08:12:09.506002+0000","last_fullsized":"2023-11-24T08:12:09.506002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531702+0000","last_change":"2023-11-24T08:12:09.529444+0000","last_active":"2023-11-24T08:12:09.531702+0000","last_peered":"2023-11-24T08:12:09.531702+0000","last_clean":"2023-11-24T08:12:09.531702+0000","last_became_active":"2023-11-24T08:12:09.528128+0000","last_became_peered":"2023-11-24T08:12:09.528128+0000","last_unstale":"2023-11-24T08:12:09.531702+0000","last_undegraded":"2023-11-24T08:12:09.531702+0000","last_fullsized":"2023-11-24T08:12:09.531702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:47:06.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511825+0000","last_change":"2023-11-24T08:12:09.511825+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511825+0000","last_undegraded":"2023-11-24T08:12:09.511825+0000","last_fullsized":"2023-11-24T08:12:09.511825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538378+0000","last_change":"2023-11-24T08:12:09.535030+0000","last_active":"2023-11-24T08:12:09.538378+0000","last_peered":"2023-11-24T08:12:09.538378+0000","last_clean":"2023-11-24T08:12:09.538378+0000","last_became_active":"2023-11-24T08:12:09.530814+0000","last_became_peered":"2023-11-24T08:12:09.530814+0000","last_unstale":"2023-11-24T08:12:09.538378+0000","last_undegraded":"2023-11-24T08:12:09.538378+0000","last_fullsized":"2023-11-24T08:12:09.538378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:52.597123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508831+0000","last_change":"2023-11-24T08:12:09.508831+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508831+0000","last_undegraded":"2023-11-24T08:12:09.508831+0000","last_fullsized":"2023-11-24T08:12:09.508831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538420+0000","last_change":"2023-11-24T08:12:09.535433+0000","last_active":"2023-11-24T08:12:09.538420+0000","last_peered":"2023-11-24T08:12:09.538420+0000","last_clean":"2023-11-24T08:12:09.538420+0000","last_became_active":"2023-11-24T08:12:09.534941+0000","last_became_peered":"2023-11-24T08:12:09.534941+0000","last_unstale":"2023-11-24T08:12:09.538420+0000","last_undegraded":"2023-11-24T08:12:09.538420+0000","last_fullsized":"2023-11-24T08:12:09.538420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:06.870016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":205,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T08:12:08.502638+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:08.502638+0000","last_peered":"2023-11-24T08:12:08.502638+0000","last_clean":"2023-11-24T08:12:08.502638+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:08.502638+0000","last_undegraded":"2023-11-24T08:12:08.502638+0000","last_fullsized":"2023-11-24T08:12:08.502638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.014,"15min":1.012},"min":{"1min":0.64300000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.456,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.367},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.03,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3340000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.125},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5149999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.157},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1739999999999999,"15min":1.1659999999999999},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.454,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1120000000000001,"15min":1.0660000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.534,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.214,"15min":1.206},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.03}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.246,"15min":1.236},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.3959999999999999,"5min":3.387,"15min":3.387},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.246,"15min":1.22},"min":{"1min":0.90500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.583,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.3140000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.234,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.448,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.391,"15min":1.3759999999999999},"min":{"1min":0.96599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6259999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4039999999999999,"15min":1.427},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6399999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.25},{"interface":"front","average":{"1min":1.391,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.129,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.698,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5069999999999999,"15min":1.4830000000000001},"min":{"1min":1.3759999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.806,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.542},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.492,"15min":1.4790000000000001},"min":{"1min":1.1100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.728,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0780000000000001,"15min":1.095},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.472,"5min":2.085,"15min":2.085},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.214,"15min":1.2370000000000001},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4850000000000001,"5min":1.794,"15min":1.794},"last":0.93000000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.70999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6599999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.413},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.80400000000000005,"5min":0.749,"15min":0.749},"max":{"1min":1.7370000000000001,"5min":1.98,"15min":1.98},"last":0.88}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2250000000000001,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6950000000000001,"5min":1.778,"15min":1.778},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6899999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7849999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.446},"min":{"1min":0.98299999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.847,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3500000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.343,"15min":1.369},"min":{"1min":0.77600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.242,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4359999999999999,"5min":2.661,"15min":2.661},"last":1.5700000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.625,"15min":1.708},"min":{"1min":1.151,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.5900000000000001,"15min":1.651},"min":{"1min":1.0249999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.921,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.4970000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.726,"15min":1.78},"min":{"1min":1.2949999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.016,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.7250000000000001,"15min":1.7849999999999999},"min":{"1min":1.2050000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1499999999999999,"5min":2.657,"15min":2.657},"last":1.722}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0529999999999999,"15min":1.0249999999999999},"min":{"1min":0.72399999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6100000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.133,"15min":1.157},"min":{"1min":0.77700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":1.837},"last":1.454}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1890000000000001,"15min":1.198},"min":{"1min":0.82799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4850000000000001,"5min":1.722,"15min":1.722},"last":1.3700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2649999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4970000000000001,"5min":2.085,"15min":2.085},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3620000000000001},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.399,"15min":1.4339999999999999},"min":{"1min":0.95399999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.573,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.468,"15min":1.5169999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.7130000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.397}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":1.1259999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8500000000000001,"5min":3.577,"15min":3.577},"last":1.153},{"interface":"front","average":{"1min":1.49,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.0620000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.958,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5780000000000001,"15min":1.6060000000000001},"min":{"1min":1.0209999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.869,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.484},{"interface":"front","average":{"1min":1.554,"5min":1.5489999999999999,"15min":1.5409999999999999},"min":{"1min":1.0089999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0030000000000001,"5min":2.512,"15min":2.512},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.6839999999999999,"15min":1.716},"min":{"1min":1.21,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.8839999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.708,"15min":1.7370000000000001},"min":{"1min":1.2949999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.9390000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.581}]}]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.01,"15min":0.98699999999999999},"min":{"1min":0.64700000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6240000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.099,"15min":1.093},"min":{"1min":0.71999999999999997,"5min":0.626,"15min":0.626},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2310000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6140000000000001,"5min":1.859,"15min":1.859},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1919999999999999,"15min":1.1639999999999999},"min":{"1min":0.79400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1230000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.625}]},{"osd":2,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2749999999999999},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.746},{"interface":"front","average":{"1min":1.115,"5min":1.099,"15min":1.105},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8879999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78900000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.659,"5min":2.25,"15min":2.25},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.86599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8360000000000001,"5min":2.746,"15min":2.746},"last":1.8120000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.331},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.452,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8440000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4259999999999999,"15min":1.446},"min":{"1min":0.73799999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.4809999999999999,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.532},{"interface":"front","average":{"1min":1.47,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8500000000000001,"5min":2.641,"15min":2.641},"last":1.393}]},{"osd":7,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5149999999999999,"15min":1.5129999999999999},"min":{"1min":0.69599999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2269999999999999,"5min":2.548,"15min":2.548},"last":1.865},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.534,"15min":1.5489999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.4470000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0489999999999999,"15min":1.145},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3180000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0589999999999999,"15min":1.143},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.085,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.95899999999999996,"15min":0.995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4630000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.534,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.5740000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.1559999999999999,"15min":1.22},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5269999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.385,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.631,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.105},{"interface":"front","average":{"1min":1.141,"5min":1.337,"15min":1.373},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4670000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.125}]},{"osd":5,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.5069999999999999,"15min":1.5680000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.486,"15min":1.536},"min":{"1min":0.872,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8700000000000001,"5min":3.339,"15min":3.339},"last":1.4610000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.5609999999999999,"15min":1.6659999999999999},"min":{"1min":0.746,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.742,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.522,"15min":1.5580000000000001},"min":{"1min":1.0249999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.681,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.403}]},{"osd":7,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.605,"15min":1.675},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.603,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.8320000000000001,"15min":2.016},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7629999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":365703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.47899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.21,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.81699999999999995,"15min":0.77700000000000002},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6950000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.6679999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.244,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.92800000000000005},"min":{"1min":0.58999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.514,"5min":1.671,"15min":1.671},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98599999999999999,"15min":0.97399999999999998},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.387,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.254},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.232,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1699999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.421},{"interface":"front","average":{"1min":1.294,"5min":1.23,"15min":1.224},"min":{"1min":0.876,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.389}]},{"osd":5,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3620000000000001,"15min":1.431},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.629,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.294,"15min":1.286},"min":{"1min":0.872,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.516}]},{"osd":6,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3839999999999999,"15min":1.4159999999999999},"min":{"1min":0.94799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.895,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3260000000000001,"15min":1.337},"min":{"1min":0.93500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9490000000000001,"5min":1.958,"15min":1.958},"last":1.3149999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4770000000000001},"min":{"1min":0.91000000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.99,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.87,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0739999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5449999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.171},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1359999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.121},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.024,"15min":1.0109999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.391,"5min":1.621,"15min":1.621},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.143,"15min":1.1579999999999999},"min":{"1min":0.66100000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.534,"5min":1.907,"15min":1.907},"last":1.228},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.018,"15min":1.046},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3740000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.022}]},{"osd":4,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.161,"15min":1.163},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.425,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.874,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6919999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.83399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.653,"5min":2.661,"15min":2.661},"last":1.478},{"interface":"front","average":{"1min":1.258,"5min":1.3120000000000001,"15min":1.3360000000000001},"min":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.788,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.409},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.343,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.7070000000000001,"5min":2.105,"15min":2.105},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4099999999999999,"15min":1.4039999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8600000000000001,"5min":2.008,"15min":2.008},"last":1.6830000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1739999999999999,"5min":1.403,"15min":1.403},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95399999999999996},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2989999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.327},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5209999999999999,"5min":1.986,"15min":1.986},"last":1.081}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0489999999999999,"15min":1.0720000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3380000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.96099999999999997},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.23,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.171},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.641,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.137},{"interface":"front","average":{"1min":1.272,"5min":1.2869999999999999,"15min":1.29},"min":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.78,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.216}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.202,"15min":1.22},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4550000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8220000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.474}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.272},"min":{"1min":0.81299999999999994,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.546,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.284,"15min":1.286},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.742,"5min":2.149,"15min":2.149},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3979999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.865,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3839999999999999,"15min":1.4119999999999999},"min":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.603,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:12:14.985 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:12:15.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.236+0000 7f5048e9f640 1 Processor -- start 2023-11-24T08:12:15.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.236+0000 7f5048e9f640 1 -- start start 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.237+0000 7f5048e9f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504413d120 0x7f504415af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.237+0000 7f5048e9f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f504406d6c0 0x7f504406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.237+0000 7f5048e9f640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f504406e060 con 0x7f50440b0a70 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.237+0000 7f5048e9f640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f504413f9e0 con 0x7f504406d6c0 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.237+0000 7f5048e9f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504413fcc0 con 0x7f504413d120 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f504406d6c0 0x7f504406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f504406d6c0 0x7f504406db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43624/0 (socket says 172.21.15.106:43624) 2023-11-24T08:12:15.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 -- 172.21.15.106:0/755361643 learned_addr learned my addr 172.21.15.106:0/755361643 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:15.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f50437fe640 1 --2- 172.21.15.106:0/755361643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504413d120 0x7f504415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 -- 172.21.15.106:0/755361643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504413d120 msgr2=0x7f504415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:15.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 --2- 172.21.15.106:0/755361643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504413d120 0x7f504415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 -- 172.21.15.106:0/755361643 >> v1:172.21.15.156:6789/0 conn(0x7f50440b0a70 legacy=0x7f50440b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:15.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.238+0000 7f5042ffd640 1 -- 172.21.15.106:0/755361643 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50440cec00 con 0x7f504406d6c0 2023-11-24T08:12:15.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.239+0000 7f5042ffd640 1 --2- 172.21.15.106:0/755361643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f504406db20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d150 tx=0x7f502c02e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48d35c77359688f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:15.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.239+0000 7f50427fc640 1 -- 172.21.15.106:0/755361643 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c02e9e0 con 0x7f504406d6c0 2023-11-24T08:12:15.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.239+0000 7f50427fc640 1 -- 172.21.15.106:0/755361643 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502c02eb40 con 0x7f504406d6c0 2023-11-24T08:12:15.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.239+0000 7f50427fc640 1 -- 172.21.15.106:0/755361643 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502c009600 con 0x7f504406d6c0 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 -- 172.21.15.106:0/755361643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 msgr2=0x7f504406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 --2- 172.21.15.106:0/755361643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f504406db20 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f502c00d150 tx=0x7f502c02e040 comp rx=0 tx=0).stop 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 -- 172.21.15.106:0/755361643 shutdown_connections 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 --2- 172.21.15.106:0/755361643 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f504406db20 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 --2- 172.21.15.106:0/755361643 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f504413d120 0x7f504415af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 -- 172.21.15.106:0/755361643 >> 172.21.15.106:0/755361643 conn(0x7f50440cb830 msgr2=0x7f50440cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 -- 172.21.15.106:0/755361643 shutdown_connections 2023-11-24T08:12:15.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.240+0000 7f5048e9f640 1 -- 172.21.15.106:0/755361643 wait complete. 2023-11-24T08:12:15.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.242+0000 7f5048e9f640 1 Processor -- start 2023-11-24T08:12:15.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.242+0000 7f5048e9f640 1 -- start start 2023-11-24T08:12:15.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f5048e9f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f50440ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f5048e9f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50440b0a70 0x7f50441536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f5048e9f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504413d120 0x7f5044151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f5048e9f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5044070390 con 0x7f504406d6c0 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f5048e9f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5044070670 con 0x7f504413d120 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f5048e9f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5044070950 con 0x7f50440b0a70 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f50440ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f50440ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43628/0 (socket says 172.21.15.106:43628) 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 -- 172.21.15.106:0/1835578954 learned_addr learned my addr 172.21.15.106:0/1835578954 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 -- 172.21.15.106:0/1835578954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50440b0a70 msgr2=0x7f50441536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50440b0a70 0x7f50441536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 -- 172.21.15.106:0/1835578954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504413d120 msgr2=0x7f5044151870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504413d120 0x7f5044151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.243+0000 7f50437fe640 1 -- 172.21.15.106:0/1835578954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c00bd40 con 0x7f504406d6c0 2023-11-24T08:12:15.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.244+0000 7f50437fe640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f50440ad800 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f503400d5f0 tx=0x7f503400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.244+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50340125f0 con 0x7f504406d6c0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.244+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5034012750 con 0x7f504406d6c0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.244+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5034012b00 con 0x7f504406d6c0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.244+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5044070c30 con 0x7f504406d6c0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.244+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5044070f10 con 0x7f504406d6c0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.245+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f504404e840 con 0x7f504406d6c0 2023-11-24T08:12:15.247 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.246+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f5034012c60 con 0x7f504406d6c0 2023-11-24T08:12:15.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.247+0000 7f503bfff640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f502406b230 0x7f502406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.247+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f503408a1a0 con 0x7f504406d6c0 2023-11-24T08:12:15.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.247+0000 7f5042ffd640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f502406b230 0x7f502406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.248+0000 7f5042ffd640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f502406b230 0x7f502406d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f502c037000 tx=0x7f502c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:15.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.251+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5034054d00 con 0x7f504406d6c0 2023-11-24T08:12:15.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.427+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5044140560 con 0x7f502406b230 2023-11-24T08:12:15.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.449+0000 7f503bfff640 1 -- 172.21.15.106:0/1835578954 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282238 (secure 0 0 0) 0x7f502c00b360 con 0x7f502406b230 2023-11-24T08:12:15.454 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:15.455 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f502406b230 msgr2=0x7f502406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f502406b230 0x7f502406d6f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f502c037000 tx=0x7f502c030000 comp rx=0 tx=0).stop 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 msgr2=0x7f50440ad800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f50440ad800 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f503400d5f0 tx=0x7f503400dab0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 shutdown_connections 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f504413d120 0x7f5044151870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f502406b230 0x7f502406d6f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f50440b0a70 0x7f50441536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 --2- 172.21.15.106:0/1835578954 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f504406d6c0 0x7f50440ad800 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 >> 172.21.15.106:0/1835578954 conn(0x7f50440cb830 msgr2=0x7f50440ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:15.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 shutdown_connections 2023-11-24T08:12:15.458 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.454+0000 7f5048e9f640 1 -- 172.21.15.106:0/1835578954 wait complete. 2023-11-24T08:12:15.473 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2023-11-24T08:12:15.336671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249184,"kb_used_data":8060,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725536,"statfs":{"total":773094113280,"available":772838948864,"internally_reserved":0,"allocated":8253440,"data_stored":4150285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001906"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524784+0000","last_change":"2023-11-24T08:12:09.524784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524784+0000","last_undegraded":"2023-11-24T08:12:09.524784+0000","last_fullsized":"2023-11-24T08:12:09.524784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517595+0000","last_change":"2023-11-24T08:12:09.517595+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517595+0000","last_undegraded":"2023-11-24T08:12:09.517595+0000","last_fullsized":"2023-11-24T08:12:09.517595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508940+0000","last_change":"2023-11-24T08:12:09.508940+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508940+0000","last_undegraded":"2023-11-24T08:12:09.508940+0000","last_fullsized":"2023-11-24T08:12:09.508940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508376+0000","last_change":"2023-11-24T08:12:09.508376+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508376+0000","last_undegraded":"2023-11-24T08:12:09.508376+0000","last_fullsized":"2023-11-24T08:12:09.508376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511217+0000","last_change":"2023-11-24T08:12:09.511217+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511217+0000","last_undegraded":"2023-11-24T08:12:09.511217+0000","last_fullsized":"2023-11-24T08:12:09.511217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.514317+0000","last_change":"2023-11-24T08:12:09.514317+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.514317+0000","last_undegraded":"2023-11-24T08:12:09.514317+0000","last_fullsized":"2023-11-24T08:12:09.514317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.532518+0000","last_change":"2023-11-24T08:12:10.532518+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.532518+0000","last_undegraded":"2023-11-24T08:12:10.532518+0000","last_fullsized":"2023-11-24T08:12:10.532518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:09.759327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511431+0000","last_change":"2023-11-24T08:12:09.511431+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511431+0000","last_undegraded":"2023-11-24T08:12:09.511431+0000","last_fullsized":"2023-11-24T08:12:09.511431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510124+0000","last_change":"2023-11-24T08:12:09.510124+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510124+0000","last_undegraded":"2023-11-24T08:12:09.510124+0000","last_fullsized":"2023-11-24T08:12:09.510124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505932+0000","last_change":"2023-11-24T08:12:09.505932+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505932+0000","last_undegraded":"2023-11-24T08:12:09.505932+0000","last_fullsized":"2023-11-24T08:12:09.505932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523117+0000","last_change":"2023-11-24T08:12:09.523117+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523117+0000","last_undegraded":"2023-11-24T08:12:09.523117+0000","last_fullsized":"2023-11-24T08:12:09.523117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507941+0000","last_change":"2023-11-24T08:12:09.507941+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507941+0000","last_undegraded":"2023-11-24T08:12:09.507941+0000","last_fullsized":"2023-11-24T08:12:09.507941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516971+0000","last_change":"2023-11-24T08:12:09.516971+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516971+0000","last_undegraded":"2023-11-24T08:12:09.516971+0000","last_fullsized":"2023-11-24T08:12:09.516971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511315+0000","last_change":"2023-11-24T08:12:09.511315+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511315+0000","last_undegraded":"2023-11-24T08:12:09.511315+0000","last_fullsized":"2023-11-24T08:12:09.511315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519786+0000","last_change":"2023-11-24T08:12:09.519786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519786+0000","last_undegraded":"2023-11-24T08:12:09.519786+0000","last_fullsized":"2023-11-24T08:12:09.519786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503356+0000","last_change":"2023-11-24T08:12:09.503356+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503356+0000","last_undegraded":"2023-11-24T08:12:09.503356+0000","last_fullsized":"2023-11-24T08:12:09.503356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509693+0000","last_change":"2023-11-24T08:12:09.509693+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509693+0000","last_undegraded":"2023-11-24T08:12:09.509693+0000","last_fullsized":"2023-11-24T08:12:09.509693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506814+0000","last_change":"2023-11-24T08:12:09.506814+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506814+0000","last_undegraded":"2023-11-24T08:12:09.506814+0000","last_fullsized":"2023-11-24T08:12:09.506814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524401+0000","last_change":"2023-11-24T08:12:09.524401+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524401+0000","last_undegraded":"2023-11-24T08:12:09.524401+0000","last_fullsized":"2023-11-24T08:12:09.524401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516989+0000","last_change":"2023-11-24T08:12:09.516989+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516989+0000","last_undegraded":"2023-11-24T08:12:09.516989+0000","last_fullsized":"2023-11-24T08:12:09.516989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506189+0000","last_change":"2023-11-24T08:12:09.506189+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506189+0000","last_undegraded":"2023-11-24T08:12:09.506189+0000","last_fullsized":"2023-11-24T08:12:09.506189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509465+0000","last_change":"2023-11-24T08:12:09.509465+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509465+0000","last_undegraded":"2023-11-24T08:12:09.509465+0000","last_fullsized":"2023-11-24T08:12:09.509465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520990+0000","last_change":"2023-11-24T08:12:09.520990+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520990+0000","last_undegraded":"2023-11-24T08:12:09.520990+0000","last_fullsized":"2023-11-24T08:12:09.520990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524248+0000","last_change":"2023-11-24T08:12:09.524248+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524248+0000","last_undegraded":"2023-11-24T08:12:09.524248+0000","last_fullsized":"2023-11-24T08:12:09.524248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510034+0000","last_change":"2023-11-24T08:12:09.510034+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510034+0000","last_undegraded":"2023-11-24T08:12:09.510034+0000","last_fullsized":"2023-11-24T08:12:09.510034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.531598+0000","last_change":"2023-11-24T08:12:10.531598+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.531598+0000","last_undegraded":"2023-11-24T08:12:10.531598+0000","last_fullsized":"2023-11-24T08:12:10.531598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:31.654975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508296+0000","last_change":"2023-11-24T08:12:09.508296+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508296+0000","last_undegraded":"2023-11-24T08:12:09.508296+0000","last_fullsized":"2023-11-24T08:12:09.508296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503786+0000","last_change":"2023-11-24T08:12:09.503786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503786+0000","last_undegraded":"2023-11-24T08:12:09.503786+0000","last_fullsized":"2023-11-24T08:12:09.503786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511659+0000","last_change":"2023-11-24T08:12:09.511659+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511659+0000","last_undegraded":"2023-11-24T08:12:09.511659+0000","last_fullsized":"2023-11-24T08:12:09.511659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.544012+0000","last_change":"2023-11-24T08:12:10.544012+0000","last_active":"2023-11-24T08:12:10.544012+0000","last_peered":"2023-11-24T08:12:10.544012+0000","last_clean":"2023-11-24T08:12:10.544012+0000","last_became_active":"2023-11-24T08:12:10.543658+0000","last_became_peered":"2023-11-24T08:12:10.543658+0000","last_unstale":"2023-11-24T08:12:10.544012+0000","last_undegraded":"2023-11-24T08:12:10.544012+0000","last_fullsized":"2023-11-24T08:12:10.544012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:34.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.524767+0000","last_change":"2023-11-24T08:12:10.524767+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.524767+0000","last_undegraded":"2023-11-24T08:12:10.524767+0000","last_fullsized":"2023-11-24T08:12:10.524767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:21:52.679838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507818+0000","last_change":"2023-11-24T08:12:09.507818+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507818+0000","last_undegraded":"2023-11-24T08:12:09.507818+0000","last_fullsized":"2023-11-24T08:12:09.507818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520772+0000","last_change":"2023-11-24T08:12:09.520772+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520772+0000","last_undegraded":"2023-11-24T08:12:09.520772+0000","last_fullsized":"2023-11-24T08:12:09.520772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537413+0000","last_change":"2023-11-24T08:12:09.535765+0000","last_active":"2023-11-24T08:12:09.537413+0000","last_peered":"2023-11-24T08:12:09.537413+0000","last_clean":"2023-11-24T08:12:09.537413+0000","last_became_active":"2023-11-24T08:12:09.535385+0000","last_became_peered":"2023-11-24T08:12:09.535385+0000","last_unstale":"2023-11-24T08:12:09.537413+0000","last_undegraded":"2023-11-24T08:12:09.537413+0000","last_fullsized":"2023-11-24T08:12:09.537413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:18.985711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509116+0000","last_change":"2023-11-24T08:12:09.509116+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509116+0000","last_undegraded":"2023-11-24T08:12:09.509116+0000","last_fullsized":"2023-11-24T08:12:09.509116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530987+0000","last_change":"2023-11-24T08:12:09.524544+0000","last_active":"2023-11-24T08:12:09.530987+0000","last_peered":"2023-11-24T08:12:09.530987+0000","last_clean":"2023-11-24T08:12:09.530987+0000","last_became_active":"2023-11-24T08:12:09.524188+0000","last_became_peered":"2023-11-24T08:12:09.524188+0000","last_unstale":"2023-11-24T08:12:09.530987+0000","last_undegraded":"2023-11-24T08:12:09.530987+0000","last_fullsized":"2023-11-24T08:12:09.530987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:56.808696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524895+0000","last_change":"2023-11-24T08:12:09.524895+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524895+0000","last_undegraded":"2023-11-24T08:12:09.524895+0000","last_fullsized":"2023-11-24T08:12:09.524895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538199+0000","last_change":"2023-11-24T08:12:09.533833+0000","last_active":"2023-11-24T08:12:09.538199+0000","last_peered":"2023-11-24T08:12:09.538199+0000","last_clean":"2023-11-24T08:12:09.538199+0000","last_became_active":"2023-11-24T08:12:09.533263+0000","last_became_peered":"2023-11-24T08:12:09.533263+0000","last_unstale":"2023-11-24T08:12:09.538199+0000","last_undegraded":"2023-11-24T08:12:09.538199+0000","last_fullsized":"2023-11-24T08:12:09.538199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:50.328328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522446+0000","last_change":"2023-11-24T08:12:09.522446+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522446+0000","last_undegraded":"2023-11-24T08:12:09.522446+0000","last_fullsized":"2023-11-24T08:12:09.522446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538124+0000","last_change":"2023-11-24T08:12:09.529735+0000","last_active":"2023-11-24T08:12:09.538124+0000","last_peered":"2023-11-24T08:12:09.538124+0000","last_clean":"2023-11-24T08:12:09.538124+0000","last_became_active":"2023-11-24T08:12:09.529517+0000","last_became_peered":"2023-11-24T08:12:09.529517+0000","last_unstale":"2023-11-24T08:12:09.538124+0000","last_undegraded":"2023-11-24T08:12:09.538124+0000","last_fullsized":"2023-11-24T08:12:09.538124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:14.043315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.525674+0000","last_change":"2023-11-24T08:12:10.525674+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.525674+0000","last_undegraded":"2023-11-24T08:12:10.525674+0000","last_fullsized":"2023-11-24T08:12:10.525674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:27.094673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526557+0000","last_change":"2023-11-24T08:12:09.522576+0000","last_active":"2023-11-24T08:12:09.526557+0000","last_peered":"2023-11-24T08:12:09.526557+0000","last_clean":"2023-11-24T08:12:09.526557+0000","last_became_active":"2023-11-24T08:12:09.522114+0000","last_became_peered":"2023-11-24T08:12:09.522114+0000","last_unstale":"2023-11-24T08:12:09.526557+0000","last_undegraded":"2023-11-24T08:12:09.526557+0000","last_fullsized":"2023-11-24T08:12:09.526557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:27:40.788887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503392+0000","last_change":"2023-11-24T08:12:09.503392+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503392+0000","last_undegraded":"2023-11-24T08:12:09.503392+0000","last_fullsized":"2023-11-24T08:12:09.503392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537905+0000","last_change":"2023-11-24T08:12:09.531869+0000","last_active":"2023-11-24T08:12:09.537905+0000","last_peered":"2023-11-24T08:12:09.537905+0000","last_clean":"2023-11-24T08:12:09.537905+0000","last_became_active":"2023-11-24T08:12:09.531598+0000","last_became_peered":"2023-11-24T08:12:09.531598+0000","last_unstale":"2023-11-24T08:12:09.537905+0000","last_undegraded":"2023-11-24T08:12:09.537905+0000","last_fullsized":"2023-11-24T08:12:09.537905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:24.358046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510090+0000","last_change":"2023-11-24T08:12:09.510090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510090+0000","last_undegraded":"2023-11-24T08:12:09.510090+0000","last_fullsized":"2023-11-24T08:12:09.510090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518079+0000","last_change":"2023-11-24T08:12:09.513949+0000","last_active":"2023-11-24T08:12:09.518079+0000","last_peered":"2023-11-24T08:12:09.518079+0000","last_clean":"2023-11-24T08:12:09.518079+0000","last_became_active":"2023-11-24T08:12:09.513517+0000","last_became_peered":"2023-11-24T08:12:09.513517+0000","last_unstale":"2023-11-24T08:12:09.518079+0000","last_undegraded":"2023-11-24T08:12:09.518079+0000","last_fullsized":"2023-11-24T08:12:09.518079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:51.436372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517039+0000","last_change":"2023-11-24T08:12:09.517039+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517039+0000","last_undegraded":"2023-11-24T08:12:09.517039+0000","last_fullsized":"2023-11-24T08:12:09.517039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.524795+0000","last_change":"2023-11-24T08:12:09.519208+0000","last_active":"2023-11-24T08:12:09.524795+0000","last_peered":"2023-11-24T08:12:09.524795+0000","last_clean":"2023-11-24T08:12:09.524795+0000","last_became_active":"2023-11-24T08:12:09.518952+0000","last_became_peered":"2023-11-24T08:12:09.518952+0000","last_unstale":"2023-11-24T08:12:09.524795+0000","last_undegraded":"2023-11-24T08:12:09.524795+0000","last_fullsized":"2023-11-24T08:12:09.524795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:57:11.824082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.542676+0000","last_change":"2023-11-24T08:12:10.542676+0000","last_active":"2023-11-24T08:12:10.542676+0000","last_peered":"2023-11-24T08:12:10.542676+0000","last_clean":"2023-11-24T08:12:10.542676+0000","last_became_active":"2023-11-24T08:12:10.542269+0000","last_became_peered":"2023-11-24T08:12:10.542269+0000","last_unstale":"2023-11-24T08:12:10.542676+0000","last_undegraded":"2023-11-24T08:12:10.542676+0000","last_fullsized":"2023-11-24T08:12:10.542676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:38.761881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.525878+0000","last_change":"2023-11-24T08:12:09.519666+0000","last_active":"2023-11-24T08:12:09.525878+0000","last_peered":"2023-11-24T08:12:09.525878+0000","last_clean":"2023-11-24T08:12:09.525878+0000","last_became_active":"2023-11-24T08:12:09.519348+0000","last_became_peered":"2023-11-24T08:12:09.519348+0000","last_unstale":"2023-11-24T08:12:09.525878+0000","last_undegraded":"2023-11-24T08:12:09.525878+0000","last_fullsized":"2023-11-24T08:12:09.525878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:08.548384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524304+0000","last_change":"2023-11-24T08:12:09.524304+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524304+0000","last_undegraded":"2023-11-24T08:12:09.524304+0000","last_fullsized":"2023-11-24T08:12:09.524304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538991+0000","last_change":"2023-11-24T08:12:09.537826+0000","last_active":"2023-11-24T08:12:09.538991+0000","last_peered":"2023-11-24T08:12:09.538991+0000","last_clean":"2023-11-24T08:12:09.538991+0000","last_became_active":"2023-11-24T08:12:09.537555+0000","last_became_peered":"2023-11-24T08:12:09.537555+0000","last_unstale":"2023-11-24T08:12:09.538991+0000","last_undegraded":"2023-11-24T08:12:09.538991+0000","last_fullsized":"2023-11-24T08:12:09.538991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:54:04.166572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.515379+0000","last_change":"2023-11-24T08:12:09.515379+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.515379+0000","last_undegraded":"2023-11-24T08:12:09.515379+0000","last_fullsized":"2023-11-24T08:12:09.515379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531034+0000","last_change":"2023-11-24T08:12:09.524384+0000","last_active":"2023-11-24T08:12:09.531034+0000","last_peered":"2023-11-24T08:12:09.531034+0000","last_clean":"2023-11-24T08:12:09.531034+0000","last_became_active":"2023-11-24T08:12:09.524038+0000","last_became_peered":"2023-11-24T08:12:09.524038+0000","last_unstale":"2023-11-24T08:12:09.531034+0000","last_undegraded":"2023-11-24T08:12:09.531034+0000","last_fullsized":"2023-11-24T08:12:09.531034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:45.336925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.513394+0000","last_change":"2023-11-24T08:12:09.513394+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.513394+0000","last_undegraded":"2023-11-24T08:12:09.513394+0000","last_fullsized":"2023-11-24T08:12:09.513394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518054+0000","last_change":"2023-11-24T08:12:09.514045+0000","last_active":"2023-11-24T08:12:09.518054+0000","last_peered":"2023-11-24T08:12:09.518054+0000","last_clean":"2023-11-24T08:12:09.518054+0000","last_became_active":"2023-11-24T08:12:09.513337+0000","last_became_peered":"2023-11-24T08:12:09.513337+0000","last_unstale":"2023-11-24T08:12:09.518054+0000","last_undegraded":"2023-11-24T08:12:09.518054+0000","last_fullsized":"2023-11-24T08:12:09.518054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:17.346075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.518834+0000","last_change":"2023-11-24T08:12:09.518834+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.518834+0000","last_undegraded":"2023-11-24T08:12:09.518834+0000","last_fullsized":"2023-11-24T08:12:09.518834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530522+0000","last_change":"2023-11-24T08:12:09.528456+0000","last_active":"2023-11-24T08:12:09.530522+0000","last_peered":"2023-11-24T08:12:09.530522+0000","last_clean":"2023-11-24T08:12:09.530522+0000","last_became_active":"2023-11-24T08:12:09.528140+0000","last_became_peered":"2023-11-24T08:12:09.528140+0000","last_unstale":"2023-11-24T08:12:09.530522+0000","last_undegraded":"2023-11-24T08:12:09.530522+0000","last_fullsized":"2023-11-24T08:12:09.530522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:12.633925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.529872+0000","last_change":"2023-11-24T08:12:10.529872+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.529872+0000","last_undegraded":"2023-11-24T08:12:10.529872+0000","last_fullsized":"2023-11-24T08:12:10.529872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:09.099806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538979+0000","last_change":"2023-11-24T08:12:09.537450+0000","last_active":"2023-11-24T08:12:09.538979+0000","last_peered":"2023-11-24T08:12:09.538979+0000","last_clean":"2023-11-24T08:12:09.538979+0000","last_became_active":"2023-11-24T08:12:09.537196+0000","last_became_peered":"2023-11-24T08:12:09.537196+0000","last_unstale":"2023-11-24T08:12:09.538979+0000","last_undegraded":"2023-11-24T08:12:09.538979+0000","last_fullsized":"2023-11-24T08:12:09.538979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:19.005358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517657+0000","last_change":"2023-11-24T08:12:09.517657+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517657+0000","last_undegraded":"2023-11-24T08:12:09.517657+0000","last_fullsized":"2023-11-24T08:12:09.517657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533269+0000","last_change":"2023-11-24T08:12:09.530752+0000","last_active":"2023-11-24T08:12:10.533269+0000","last_peered":"2023-11-24T08:12:10.533269+0000","last_clean":"2023-11-24T08:12:10.533269+0000","last_became_active":"2023-11-24T08:12:09.529282+0000","last_became_peered":"2023-11-24T08:12:09.529282+0000","last_unstale":"2023-11-24T08:12:10.533269+0000","last_undegraded":"2023-11-24T08:12:10.533269+0000","last_fullsized":"2023-11-24T08:12:10.533269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:31.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510752+0000","last_change":"2023-11-24T08:12:09.510752+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510752+0000","last_undegraded":"2023-11-24T08:12:09.510752+0000","last_fullsized":"2023-11-24T08:12:09.510752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.534559+0000","last_change":"2023-11-24T08:12:09.530860+0000","last_active":"2023-11-24T08:12:09.534559+0000","last_peered":"2023-11-24T08:12:09.534559+0000","last_clean":"2023-11-24T08:12:09.534559+0000","last_became_active":"2023-11-24T08:12:09.530167+0000","last_became_peered":"2023-11-24T08:12:09.530167+0000","last_unstale":"2023-11-24T08:12:09.534559+0000","last_undegraded":"2023-11-24T08:12:09.534559+0000","last_fullsized":"2023-11-24T08:12:09.534559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:08.559029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.504790+0000","last_change":"2023-11-24T08:12:09.504790+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.504790+0000","last_undegraded":"2023-11-24T08:12:09.504790+0000","last_fullsized":"2023-11-24T08:12:09.504790+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.539017+0000","last_change":"2023-11-24T08:12:09.538063+0000","last_active":"2023-11-24T08:12:09.539017+0000","last_peered":"2023-11-24T08:12:09.539017+0000","last_clean":"2023-11-24T08:12:09.539017+0000","last_became_active":"2023-11-24T08:12:09.536932+0000","last_became_peered":"2023-11-24T08:12:09.536932+0000","last_unstale":"2023-11-24T08:12:09.539017+0000","last_undegraded":"2023-11-24T08:12:09.539017+0000","last_fullsized":"2023-11-24T08:12:09.539017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:31.591358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523784+0000","last_change":"2023-11-24T08:12:09.523784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523784+0000","last_undegraded":"2023-11-24T08:12:09.523784+0000","last_fullsized":"2023-11-24T08:12:09.523784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538170+0000","last_change":"2023-11-24T08:12:09.533762+0000","last_active":"2023-11-24T08:12:09.538170+0000","last_peered":"2023-11-24T08:12:09.538170+0000","last_clean":"2023-11-24T08:12:09.538170+0000","last_became_active":"2023-11-24T08:12:09.533465+0000","last_became_peered":"2023-11-24T08:12:09.533465+0000","last_unstale":"2023-11-24T08:12:09.538170+0000","last_undegraded":"2023-11-24T08:12:09.538170+0000","last_fullsized":"2023-11-24T08:12:09.538170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:44.590676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505924+0000","last_change":"2023-11-24T08:12:09.505924+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505924+0000","last_undegraded":"2023-11-24T08:12:09.505924+0000","last_fullsized":"2023-11-24T08:12:09.505924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537888+0000","last_change":"2023-11-24T08:12:09.530831+0000","last_active":"2023-11-24T08:12:09.537888+0000","last_peered":"2023-11-24T08:12:09.537888+0000","last_clean":"2023-11-24T08:12:09.537888+0000","last_became_active":"2023-11-24T08:12:09.530459+0000","last_became_peered":"2023-11-24T08:12:09.530459+0000","last_unstale":"2023-11-24T08:12:09.537888+0000","last_undegraded":"2023-11-24T08:12:09.537888+0000","last_fullsized":"2023-11-24T08:12:09.537888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:40.045785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509492+0000","last_change":"2023-11-24T08:12:09.509492+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509492+0000","last_undegraded":"2023-11-24T08:12:09.509492+0000","last_fullsized":"2023-11-24T08:12:09.509492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537541+0000","last_change":"2023-11-24T08:12:09.535763+0000","last_active":"2023-11-24T08:12:09.537541+0000","last_peered":"2023-11-24T08:12:09.537541+0000","last_clean":"2023-11-24T08:12:09.537541+0000","last_became_active":"2023-11-24T08:12:09.535048+0000","last_became_peered":"2023-11-24T08:12:09.535048+0000","last_unstale":"2023-11-24T08:12:09.537541+0000","last_undegraded":"2023-11-24T08:12:09.537541+0000","last_fullsized":"2023-11-24T08:12:09.537541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:42.819752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511207+0000","last_change":"2023-11-24T08:12:09.511207+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511207+0000","last_undegraded":"2023-11-24T08:12:09.511207+0000","last_fullsized":"2023-11-24T08:12:09.511207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538446+0000","last_change":"2023-11-24T08:12:09.536935+0000","last_active":"2023-11-24T08:12:09.538446+0000","last_peered":"2023-11-24T08:12:09.538446+0000","last_clean":"2023-11-24T08:12:09.538446+0000","last_became_active":"2023-11-24T08:12:09.535354+0000","last_became_peered":"2023-11-24T08:12:09.535354+0000","last_unstale":"2023-11-24T08:12:09.538446+0000","last_undegraded":"2023-11-24T08:12:09.538446+0000","last_fullsized":"2023-11-24T08:12:09.538446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:49.306476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507471+0000","last_change":"2023-11-24T08:12:09.507471+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507471+0000","last_undegraded":"2023-11-24T08:12:09.507471+0000","last_fullsized":"2023-11-24T08:12:09.507471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533487+0000","last_change":"2023-11-24T08:12:09.530273+0000","last_active":"2023-11-24T08:12:10.533487+0000","last_peered":"2023-11-24T08:12:10.533487+0000","last_clean":"2023-11-24T08:12:10.533487+0000","last_became_active":"2023-11-24T08:12:09.529967+0000","last_became_peered":"2023-11-24T08:12:09.529967+0000","last_unstale":"2023-11-24T08:12:10.533487+0000","last_undegraded":"2023-11-24T08:12:10.533487+0000","last_fullsized":"2023-11-24T08:12:10.533487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:37.818727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508506+0000","last_change":"2023-11-24T08:12:09.508506+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508506+0000","last_undegraded":"2023-11-24T08:12:09.508506+0000","last_fullsized":"2023-11-24T08:12:09.508506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.535270+0000","last_change":"2023-11-24T08:12:09.530296+0000","last_active":"2023-11-24T08:12:09.535270+0000","last_peered":"2023-11-24T08:12:09.535270+0000","last_clean":"2023-11-24T08:12:09.535270+0000","last_became_active":"2023-11-24T08:12:09.529840+0000","last_became_peered":"2023-11-24T08:12:09.529840+0000","last_unstale":"2023-11-24T08:12:09.535270+0000","last_undegraded":"2023-11-24T08:12:09.535270+0000","last_fullsized":"2023-11-24T08:12:09.535270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:42.718534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519744+0000","last_change":"2023-11-24T08:12:09.519744+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519744+0000","last_undegraded":"2023-11-24T08:12:09.519744+0000","last_fullsized":"2023-11-24T08:12:09.519744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538516+0000","last_change":"2023-11-24T08:12:09.536994+0000","last_active":"2023-11-24T08:12:09.538516+0000","last_peered":"2023-11-24T08:12:09.538516+0000","last_clean":"2023-11-24T08:12:09.538516+0000","last_became_active":"2023-11-24T08:12:09.536664+0000","last_became_peered":"2023-11-24T08:12:09.536664+0000","last_unstale":"2023-11-24T08:12:09.538516+0000","last_undegraded":"2023-11-24T08:12:09.538516+0000","last_fullsized":"2023-11-24T08:12:09.538516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:10.151500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.533267+0000","last_change":"2023-11-24T08:12:10.533267+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.533267+0000","last_undegraded":"2023-11-24T08:12:10.533267+0000","last_fullsized":"2023-11-24T08:12:10.533267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:10.427272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.534767+0000","last_change":"2023-11-24T08:12:09.534163+0000","last_active":"2023-11-24T08:12:10.534767+0000","last_peered":"2023-11-24T08:12:10.534767+0000","last_clean":"2023-11-24T08:12:10.534767+0000","last_became_active":"2023-11-24T08:12:09.533384+0000","last_became_peered":"2023-11-24T08:12:09.533384+0000","last_unstale":"2023-11-24T08:12:10.534767+0000","last_undegraded":"2023-11-24T08:12:10.534767+0000","last_fullsized":"2023-11-24T08:12:10.534767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:39.337647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522771+0000","last_change":"2023-11-24T08:12:09.522771+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522771+0000","last_undegraded":"2023-11-24T08:12:09.522771+0000","last_fullsized":"2023-11-24T08:12:09.522771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.529558+0000","last_change":"2023-11-24T08:12:09.524480+0000","last_active":"2023-11-24T08:12:09.529558+0000","last_peered":"2023-11-24T08:12:09.529558+0000","last_clean":"2023-11-24T08:12:09.529558+0000","last_became_active":"2023-11-24T08:12:09.524212+0000","last_became_peered":"2023-11-24T08:12:09.524212+0000","last_unstale":"2023-11-24T08:12:09.529558+0000","last_undegraded":"2023-11-24T08:12:09.529558+0000","last_fullsized":"2023-11-24T08:12:09.529558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:57.611814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520763+0000","last_change":"2023-11-24T08:12:09.520763+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520763+0000","last_undegraded":"2023-11-24T08:12:09.520763+0000","last_fullsized":"2023-11-24T08:12:09.520763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.532684+0000","last_change":"2023-11-24T08:12:09.530037+0000","last_active":"2023-11-24T08:12:09.532684+0000","last_peered":"2023-11-24T08:12:09.532684+0000","last_clean":"2023-11-24T08:12:09.532684+0000","last_became_active":"2023-11-24T08:12:09.529803+0000","last_became_peered":"2023-11-24T08:12:09.529803+0000","last_unstale":"2023-11-24T08:12:09.532684+0000","last_undegraded":"2023-11-24T08:12:09.532684+0000","last_fullsized":"2023-11-24T08:12:09.532684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:48.699065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510858+0000","last_change":"2023-11-24T08:12:09.510858+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510858+0000","last_undegraded":"2023-11-24T08:12:09.510858+0000","last_fullsized":"2023-11-24T08:12:09.510858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526913+0000","last_change":"2023-11-24T08:12:09.522451+0000","last_active":"2023-11-24T08:12:09.526913+0000","last_peered":"2023-11-24T08:12:09.526913+0000","last_clean":"2023-11-24T08:12:09.526913+0000","last_became_active":"2023-11-24T08:12:09.521531+0000","last_became_peered":"2023-11-24T08:12:09.521531+0000","last_unstale":"2023-11-24T08:12:09.526913+0000","last_undegraded":"2023-11-24T08:12:09.526913+0000","last_fullsized":"2023-11-24T08:12:09.526913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.342039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516090+0000","last_change":"2023-11-24T08:12:09.516090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516090+0000","last_undegraded":"2023-11-24T08:12:09.516090+0000","last_fullsized":"2023-11-24T08:12:09.516090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.527191+0000","last_change":"2023-11-24T08:12:09.521396+0000","last_active":"2023-11-24T08:12:09.527191+0000","last_peered":"2023-11-24T08:12:09.527191+0000","last_clean":"2023-11-24T08:12:09.527191+0000","last_became_active":"2023-11-24T08:12:09.520385+0000","last_became_peered":"2023-11-24T08:12:09.520385+0000","last_unstale":"2023-11-24T08:12:09.527191+0000","last_undegraded":"2023-11-24T08:12:09.527191+0000","last_fullsized":"2023-11-24T08:12:09.527191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:25.592586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506002+0000","last_change":"2023-11-24T08:12:09.506002+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506002+0000","last_undegraded":"2023-11-24T08:12:09.506002+0000","last_fullsized":"2023-11-24T08:12:09.506002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531702+0000","last_change":"2023-11-24T08:12:09.529444+0000","last_active":"2023-11-24T08:12:09.531702+0000","last_peered":"2023-11-24T08:12:09.531702+0000","last_clean":"2023-11-24T08:12:09.531702+0000","last_became_active":"2023-11-24T08:12:09.528128+0000","last_became_peered":"2023-11-24T08:12:09.528128+0000","last_unstale":"2023-11-24T08:12:09.531702+0000","last_undegraded":"2023-11-24T08:12:09.531702+0000","last_fullsized":"2023-11-24T08:12:09.531702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:47:06.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511825+0000","last_change":"2023-11-24T08:12:09.511825+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511825+0000","last_undegraded":"2023-11-24T08:12:09.511825+0000","last_fullsized":"2023-11-24T08:12:09.511825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538378+0000","last_change":"2023-11-24T08:12:09.535030+0000","last_active":"2023-11-24T08:12:09.538378+0000","last_peered":"2023-11-24T08:12:09.538378+0000","last_clean":"2023-11-24T08:12:09.538378+0000","last_became_active":"2023-11-24T08:12:09.530814+0000","last_became_peered":"2023-11-24T08:12:09.530814+0000","last_unstale":"2023-11-24T08:12:09.538378+0000","last_undegraded":"2023-11-24T08:12:09.538378+0000","last_fullsized":"2023-11-24T08:12:09.538378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:52.597123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508831+0000","last_change":"2023-11-24T08:12:09.508831+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508831+0000","last_undegraded":"2023-11-24T08:12:09.508831+0000","last_fullsized":"2023-11-24T08:12:09.508831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538420+0000","last_change":"2023-11-24T08:12:09.535433+0000","last_active":"2023-11-24T08:12:09.538420+0000","last_peered":"2023-11-24T08:12:09.538420+0000","last_clean":"2023-11-24T08:12:09.538420+0000","last_became_active":"2023-11-24T08:12:09.534941+0000","last_became_peered":"2023-11-24T08:12:09.534941+0000","last_unstale":"2023-11-24T08:12:09.538420+0000","last_undegraded":"2023-11-24T08:12:09.538420+0000","last_fullsized":"2023-11-24T08:12:09.538420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:06.870016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":205,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T08:12:08.502638+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:08.502638+0000","last_peered":"2023-11-24T08:12:08.502638+0000","last_clean":"2023-11-24T08:12:08.502638+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:08.502638+0000","last_undegraded":"2023-11-24T08:12:08.502638+0000","last_fullsized":"2023-11-24T08:12:08.502638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.014,"15min":1.012},"min":{"1min":0.64300000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.456,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.367},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.03,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3340000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.125},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5149999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.157},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1739999999999999,"15min":1.1659999999999999},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.454,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1120000000000001,"15min":1.0660000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.534,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.214,"15min":1.206},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.03}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.246,"15min":1.236},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.3959999999999999,"5min":3.387,"15min":3.387},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.246,"15min":1.22},"min":{"1min":0.90500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.583,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.3140000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.234,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.448,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.391,"15min":1.3759999999999999},"min":{"1min":0.96599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6259999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4039999999999999,"15min":1.427},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6399999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.25},{"interface":"front","average":{"1min":1.391,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.129,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.698,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5069999999999999,"15min":1.4830000000000001},"min":{"1min":1.3759999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.806,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.542},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.492,"15min":1.4790000000000001},"min":{"1min":1.1100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.728,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0780000000000001,"15min":1.095},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.472,"5min":2.085,"15min":2.085},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.214,"15min":1.2370000000000001},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4850000000000001,"5min":1.794,"15min":1.794},"last":0.93000000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.70999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6599999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.413},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.80400000000000005,"5min":0.749,"15min":0.749},"max":{"1min":1.7370000000000001,"5min":1.98,"15min":1.98},"last":0.88}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2250000000000001,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6950000000000001,"5min":1.778,"15min":1.778},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6899999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7849999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.446},"min":{"1min":0.98299999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.847,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3500000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.343,"15min":1.369},"min":{"1min":0.77600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.242,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4359999999999999,"5min":2.661,"15min":2.661},"last":1.5700000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.625,"15min":1.708},"min":{"1min":1.151,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.5900000000000001,"15min":1.651},"min":{"1min":1.0249999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.921,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.4970000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.726,"15min":1.78},"min":{"1min":1.2949999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.016,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.7250000000000001,"15min":1.7849999999999999},"min":{"1min":1.2050000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1499999999999999,"5min":2.657,"15min":2.657},"last":1.722}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0529999999999999,"15min":1.0249999999999999},"min":{"1min":0.72399999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6100000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.133,"15min":1.157},"min":{"1min":0.77700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":1.837},"last":1.454}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1890000000000001,"15min":1.198},"min":{"1min":0.82799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4850000000000001,"5min":1.722,"15min":1.722},"last":1.3700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2649999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4970000000000001,"5min":2.085,"15min":2.085},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3620000000000001},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.399,"15min":1.4339999999999999},"min":{"1min":0.95399999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.573,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.468,"15min":1.5169999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.7130000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.397}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":1.1259999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8500000000000001,"5min":3.577,"15min":3.577},"last":1.153},{"interface":"front","average":{"1min":1.49,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.0620000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.958,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5780000000000001,"15min":1.6060000000000001},"min":{"1min":1.0209999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.869,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.484},{"interface":"front","average":{"1min":1.554,"5min":1.5489999999999999,"15min":1.5409999999999999},"min":{"1min":1.0089999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0030000000000001,"5min":2.512,"15min":2.512},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.6839999999999999,"15min":1.716},"min":{"1min":1.21,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.8839999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.708,"15min":1.7370000000000001},"min":{"1min":1.2949999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.9390000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.581}]}]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.01,"15min":0.98699999999999999},"min":{"1min":0.64700000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6240000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.099,"15min":1.093},"min":{"1min":0.71999999999999997,"5min":0.626,"15min":0.626},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2310000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6140000000000001,"5min":1.859,"15min":1.859},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1919999999999999,"15min":1.1639999999999999},"min":{"1min":0.79400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1230000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.625}]},{"osd":2,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2749999999999999},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.746},{"interface":"front","average":{"1min":1.115,"5min":1.099,"15min":1.105},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8879999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78900000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.659,"5min":2.25,"15min":2.25},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.86599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8360000000000001,"5min":2.746,"15min":2.746},"last":1.8120000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.331},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.452,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8440000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4259999999999999,"15min":1.446},"min":{"1min":0.73799999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.4809999999999999,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.532},{"interface":"front","average":{"1min":1.47,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8500000000000001,"5min":2.641,"15min":2.641},"last":1.393}]},{"osd":7,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5149999999999999,"15min":1.5129999999999999},"min":{"1min":0.69599999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2269999999999999,"5min":2.548,"15min":2.548},"last":1.865},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.534,"15min":1.5489999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.4470000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0489999999999999,"15min":1.145},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3180000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0589999999999999,"15min":1.143},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.085,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.95899999999999996,"15min":0.995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4630000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.534,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.5740000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.1559999999999999,"15min":1.22},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5269999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.385,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.631,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.105},{"interface":"front","average":{"1min":1.141,"5min":1.337,"15min":1.373},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4670000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.125}]},{"osd":5,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.5069999999999999,"15min":1.5680000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.486,"15min":1.536},"min":{"1min":0.872,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8700000000000001,"5min":3.339,"15min":3.339},"last":1.4610000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.5609999999999999,"15min":1.6659999999999999},"min":{"1min":0.746,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.742,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.522,"15min":1.5580000000000001},"min":{"1min":1.0249999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.681,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.403}]},{"osd":7,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.605,"15min":1.675},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.603,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.8320000000000001,"15min":2.016},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7629999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":365703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.47899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.21,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.81699999999999995,"15min":0.77700000000000002},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6950000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.6679999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.244,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.92800000000000005},"min":{"1min":0.58999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.514,"5min":1.671,"15min":1.671},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98599999999999999,"15min":0.97399999999999998},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.387,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.254},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.232,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1699999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.421},{"interface":"front","average":{"1min":1.294,"5min":1.23,"15min":1.224},"min":{"1min":0.876,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.389}]},{"osd":5,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3620000000000001,"15min":1.431},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.629,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.294,"15min":1.286},"min":{"1min":0.872,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.516}]},{"osd":6,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3839999999999999,"15min":1.4159999999999999},"min":{"1min":0.94799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.895,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3260000000000001,"15min":1.337},"min":{"1min":0.93500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9490000000000001,"5min":1.958,"15min":1.958},"last":1.3149999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4770000000000001},"min":{"1min":0.91000000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.99,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.87,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0739999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5449999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.171},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1359999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.121},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.024,"15min":1.0109999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.391,"5min":1.621,"15min":1.621},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.143,"15min":1.1579999999999999},"min":{"1min":0.66100000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.534,"5min":1.907,"15min":1.907},"last":1.228},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.018,"15min":1.046},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3740000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.022}]},{"osd":4,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.161,"15min":1.163},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.425,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.874,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6919999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.83399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.653,"5min":2.661,"15min":2.661},"last":1.478},{"interface":"front","average":{"1min":1.258,"5min":1.3120000000000001,"15min":1.3360000000000001},"min":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.788,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.409},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.343,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.7070000000000001,"5min":2.105,"15min":2.105},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4099999999999999,"15min":1.4039999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8600000000000001,"5min":2.008,"15min":2.008},"last":1.6830000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1739999999999999,"5min":1.403,"15min":1.403},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95399999999999996},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2989999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.327},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5209999999999999,"5min":1.986,"15min":1.986},"last":1.081}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0489999999999999,"15min":1.0720000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3380000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.96099999999999997},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.23,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.171},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.641,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.137},{"interface":"front","average":{"1min":1.272,"5min":1.2869999999999999,"15min":1.29},"min":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.78,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.216}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.202,"15min":1.22},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4550000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8220000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.474}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.272},"min":{"1min":0.81299999999999994,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.546,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.284,"15min":1.286},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.742,"5min":2.149,"15min":2.149},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3979999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.865,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3839999999999999,"15min":1.4119999999999999},"min":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.603,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:12:15.477 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:12:15.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.724+0000 7f08caee5640 1 Processor -- start 2023-11-24T08:12:15.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.725+0000 7f08caee5640 1 -- start start 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08caee5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08c40d31a0 0x7f08c40d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08caee5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c40ceb40 con 0x7f08c40ce010 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08caee5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c40d1070 con 0x7f08c41609e0 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08caee5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c40d3b50 con 0x7f08c40d31a0 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08c3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08c40d31a0 0x7f08c40d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08c3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f08c40d31a0 0x7f08c40d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43632/0 (socket says 172.21.15.106:43632) 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08c3fff640 1 -- 172.21.15.106:0/1270435189 learned_addr learned my addr 172.21.15.106:0/1270435189 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:15.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.726+0000 7f08c9482640 1 --1- 172.21.15.106:0/1270435189 >> v1:172.21.15.106:6789/0 conn(0x7f08c40ce010 0x7f08c40ce430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51846/0 (socket says 172.21.15.106:51846) 2023-11-24T08:12:15.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.727+0000 7f08c3fff640 1 -- 172.21.15.106:0/1270435189 >> v1:172.21.15.156:6789/0 conn(0x7f08c41609e0 legacy=0x7f08c406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:15.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.727+0000 7f08c3fff640 1 -- 172.21.15.106:0/1270435189 >> v1:172.21.15.106:6789/0 conn(0x7f08c40ce010 legacy=0x7f08c40ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.727+0000 7f08c3fff640 1 -- 172.21.15.106:0/1270435189 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c40cedc0 con 0x7f08c40d31a0 2023-11-24T08:12:15.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.728+0000 7f08c3fff640 1 --2- 172.21.15.106:0/1270435189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c40d31a0 0x7f08c40d3610 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009e50 tx=0x7f08b802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99c3e5f2314df8de server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:15.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.728+0000 7f08c37fe640 1 -- 172.21.15.106:0/1270435189 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b802cb20 con 0x7f08c40d31a0 2023-11-24T08:12:15.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.728+0000 7f08c37fe640 1 -- 172.21.15.106:0/1270435189 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b802cc80 con 0x7f08c40d31a0 2023-11-24T08:12:15.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.728+0000 7f08c37fe640 1 -- 172.21.15.106:0/1270435189 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80343f0 con 0x7f08c40d31a0 2023-11-24T08:12:15.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 -- 172.21.15.106:0/1270435189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c40d31a0 msgr2=0x7f08c40d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 --2- 172.21.15.106:0/1270435189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c40d31a0 0x7f08c40d3610 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009e50 tx=0x7f08b802bf80 comp rx=0 tx=0).stop 2023-11-24T08:12:15.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 -- 172.21.15.106:0/1270435189 shutdown_connections 2023-11-24T08:12:15.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 --2- 172.21.15.106:0/1270435189 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c40d31a0 0x7f08c40d3610 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 -- 172.21.15.106:0/1270435189 >> 172.21.15.106:0/1270435189 conn(0x7f08c408af60 msgr2=0x7f08c409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:15.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 -- 172.21.15.106:0/1270435189 shutdown_connections 2023-11-24T08:12:15.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.729+0000 7f08caee5640 1 -- 172.21.15.106:0/1270435189 wait complete. 2023-11-24T08:12:15.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.731+0000 7f08caee5640 1 Processor -- start 2023-11-24T08:12:15.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.731+0000 7f08caee5640 1 -- start start 2023-11-24T08:12:15.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08caee5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08c40ce010 0x7f08c415f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08caee5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08c40d31a0 0x7f08c40c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08caee5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 0x7f08c40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08caee5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c40c8350 con 0x7f08c41609e0 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08caee5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c415ff10 con 0x7f08c40ce010 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08caee5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c40c7ab0 con 0x7f08c40d31a0 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 0x7f08c40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 0x7f08c40b5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43648/0 (socket says 172.21.15.106:43648) 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 -- 172.21.15.106:0/1623926515 learned_addr learned my addr 172.21.15.106:0/1623926515 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c3fff640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08c40d31a0 0x7f08c40c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c8c81640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08c40ce010 0x7f08c415f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 -- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08c40d31a0 msgr2=0x7f08c40c70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08c40d31a0 0x7f08c40c70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 -- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08c40ce010 msgr2=0x7f08c415f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08c40ce010 0x7f08c415f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c9482640 1 -- 172.21.15.106:0/1623926515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009b30 con 0x7f08c41609e0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.732+0000 7f08c8c81640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08c40ce010 0x7f08c415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.733+0000 7f08c9482640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 0x7f08c40b5030 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f08bc001eb0 tx=0x7f08bc00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.733+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0074e0 con 0x7f08c41609e0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.733+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc007640 con 0x7f08c41609e0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.733+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0049e0 con 0x7f08c41609e0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.733+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c40c7d90 con 0x7f08c41609e0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.733+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c409e350 con 0x7f08c41609e0 2023-11-24T08:12:15.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.734+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c409e190 con 0x7f08c41609e0 2023-11-24T08:12:15.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.735+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f08bc004b40 con 0x7f08c41609e0 2023-11-24T08:12:15.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.736+0000 7f08c17fa640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08b006b230 0x7f08b006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:15.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.736+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08bc089690 con 0x7f08c41609e0 2023-11-24T08:12:15.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.736+0000 7f08c8c81640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08b006b230 0x7f08b006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:15.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.737+0000 7f08c8c81640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08b006b230 0x7f08b006d6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f08c409de40 tx=0x7f08b4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:15.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.740+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08bc0541b0 con 0x7f08c41609e0 2023-11-24T08:12:15.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.925+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08c40cf380 con 0x7f08b006b230 2023-11-24T08:12:15.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.936+0000 7f08c17fa640 1 -- 172.21.15.106:0/1623926515 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282238 (secure 0 0 0) 0x7f08b4001cc0 con 0x7f08b006b230 2023-11-24T08:12:15.939 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:15.943 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:12:15.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08b006b230 msgr2=0x7f08b006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08b006b230 0x7f08b006d6f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f08c409de40 tx=0x7f08b4009040 comp rx=0 tx=0).stop 2023-11-24T08:12:15.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 msgr2=0x7f08c40b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:15.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 0x7f08c40b5030 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f08bc001eb0 tx=0x7f08bc00c820 comp rx=0 tx=0).stop 2023-11-24T08:12:15.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 shutdown_connections 2023-11-24T08:12:15.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08c41609e0 0x7f08c40b5030 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08b006b230 0x7f08b006d6f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08c40d31a0 0x7f08c40c70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 --2- 172.21.15.106:0/1623926515 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08c40ce010 0x7f08c415f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:15.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.942+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 >> 172.21.15.106:0/1623926515 conn(0x7f08c408af60 msgr2=0x7f08c40b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:15.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.943+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 shutdown_connections 2023-11-24T08:12:15.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:15.943+0000 7f08caee5640 1 -- 172.21.15.106:0/1623926515 wait complete. 2023-11-24T08:12:15.963 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2023-11-24T08:12:15.336671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249184,"kb_used_data":8060,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725536,"statfs":{"total":773094113280,"available":772838948864,"internally_reserved":0,"allocated":8253440,"data_stored":4150285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001906"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524784+0000","last_change":"2023-11-24T08:12:09.524784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524784+0000","last_undegraded":"2023-11-24T08:12:09.524784+0000","last_fullsized":"2023-11-24T08:12:09.524784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517595+0000","last_change":"2023-11-24T08:12:09.517595+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517595+0000","last_undegraded":"2023-11-24T08:12:09.517595+0000","last_fullsized":"2023-11-24T08:12:09.517595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508940+0000","last_change":"2023-11-24T08:12:09.508940+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508940+0000","last_undegraded":"2023-11-24T08:12:09.508940+0000","last_fullsized":"2023-11-24T08:12:09.508940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508376+0000","last_change":"2023-11-24T08:12:09.508376+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508376+0000","last_undegraded":"2023-11-24T08:12:09.508376+0000","last_fullsized":"2023-11-24T08:12:09.508376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511217+0000","last_change":"2023-11-24T08:12:09.511217+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511217+0000","last_undegraded":"2023-11-24T08:12:09.511217+0000","last_fullsized":"2023-11-24T08:12:09.511217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.514317+0000","last_change":"2023-11-24T08:12:09.514317+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.514317+0000","last_undegraded":"2023-11-24T08:12:09.514317+0000","last_fullsized":"2023-11-24T08:12:09.514317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.532518+0000","last_change":"2023-11-24T08:12:10.532518+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.532518+0000","last_undegraded":"2023-11-24T08:12:10.532518+0000","last_fullsized":"2023-11-24T08:12:10.532518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:09.759327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511431+0000","last_change":"2023-11-24T08:12:09.511431+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511431+0000","last_undegraded":"2023-11-24T08:12:09.511431+0000","last_fullsized":"2023-11-24T08:12:09.511431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510124+0000","last_change":"2023-11-24T08:12:09.510124+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510124+0000","last_undegraded":"2023-11-24T08:12:09.510124+0000","last_fullsized":"2023-11-24T08:12:09.510124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505932+0000","last_change":"2023-11-24T08:12:09.505932+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505932+0000","last_undegraded":"2023-11-24T08:12:09.505932+0000","last_fullsized":"2023-11-24T08:12:09.505932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523117+0000","last_change":"2023-11-24T08:12:09.523117+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523117+0000","last_undegraded":"2023-11-24T08:12:09.523117+0000","last_fullsized":"2023-11-24T08:12:09.523117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507941+0000","last_change":"2023-11-24T08:12:09.507941+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507941+0000","last_undegraded":"2023-11-24T08:12:09.507941+0000","last_fullsized":"2023-11-24T08:12:09.507941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516971+0000","last_change":"2023-11-24T08:12:09.516971+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516971+0000","last_undegraded":"2023-11-24T08:12:09.516971+0000","last_fullsized":"2023-11-24T08:12:09.516971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511315+0000","last_change":"2023-11-24T08:12:09.511315+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511315+0000","last_undegraded":"2023-11-24T08:12:09.511315+0000","last_fullsized":"2023-11-24T08:12:09.511315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519786+0000","last_change":"2023-11-24T08:12:09.519786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519786+0000","last_undegraded":"2023-11-24T08:12:09.519786+0000","last_fullsized":"2023-11-24T08:12:09.519786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503356+0000","last_change":"2023-11-24T08:12:09.503356+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503356+0000","last_undegraded":"2023-11-24T08:12:09.503356+0000","last_fullsized":"2023-11-24T08:12:09.503356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509693+0000","last_change":"2023-11-24T08:12:09.509693+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509693+0000","last_undegraded":"2023-11-24T08:12:09.509693+0000","last_fullsized":"2023-11-24T08:12:09.509693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506814+0000","last_change":"2023-11-24T08:12:09.506814+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506814+0000","last_undegraded":"2023-11-24T08:12:09.506814+0000","last_fullsized":"2023-11-24T08:12:09.506814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524401+0000","last_change":"2023-11-24T08:12:09.524401+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524401+0000","last_undegraded":"2023-11-24T08:12:09.524401+0000","last_fullsized":"2023-11-24T08:12:09.524401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516989+0000","last_change":"2023-11-24T08:12:09.516989+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516989+0000","last_undegraded":"2023-11-24T08:12:09.516989+0000","last_fullsized":"2023-11-24T08:12:09.516989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506189+0000","last_change":"2023-11-24T08:12:09.506189+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506189+0000","last_undegraded":"2023-11-24T08:12:09.506189+0000","last_fullsized":"2023-11-24T08:12:09.506189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509465+0000","last_change":"2023-11-24T08:12:09.509465+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509465+0000","last_undegraded":"2023-11-24T08:12:09.509465+0000","last_fullsized":"2023-11-24T08:12:09.509465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520990+0000","last_change":"2023-11-24T08:12:09.520990+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520990+0000","last_undegraded":"2023-11-24T08:12:09.520990+0000","last_fullsized":"2023-11-24T08:12:09.520990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524248+0000","last_change":"2023-11-24T08:12:09.524248+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524248+0000","last_undegraded":"2023-11-24T08:12:09.524248+0000","last_fullsized":"2023-11-24T08:12:09.524248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510034+0000","last_change":"2023-11-24T08:12:09.510034+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510034+0000","last_undegraded":"2023-11-24T08:12:09.510034+0000","last_fullsized":"2023-11-24T08:12:09.510034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.531598+0000","last_change":"2023-11-24T08:12:10.531598+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.531598+0000","last_undegraded":"2023-11-24T08:12:10.531598+0000","last_fullsized":"2023-11-24T08:12:10.531598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:31.654975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508296+0000","last_change":"2023-11-24T08:12:09.508296+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508296+0000","last_undegraded":"2023-11-24T08:12:09.508296+0000","last_fullsized":"2023-11-24T08:12:09.508296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503786+0000","last_change":"2023-11-24T08:12:09.503786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503786+0000","last_undegraded":"2023-11-24T08:12:09.503786+0000","last_fullsized":"2023-11-24T08:12:09.503786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511659+0000","last_change":"2023-11-24T08:12:09.511659+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511659+0000","last_undegraded":"2023-11-24T08:12:09.511659+0000","last_fullsized":"2023-11-24T08:12:09.511659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.544012+0000","last_change":"2023-11-24T08:12:10.544012+0000","last_active":"2023-11-24T08:12:10.544012+0000","last_peered":"2023-11-24T08:12:10.544012+0000","last_clean":"2023-11-24T08:12:10.544012+0000","last_became_active":"2023-11-24T08:12:10.543658+0000","last_became_peered":"2023-11-24T08:12:10.543658+0000","last_unstale":"2023-11-24T08:12:10.544012+0000","last_undegraded":"2023-11-24T08:12:10.544012+0000","last_fullsized":"2023-11-24T08:12:10.544012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:34.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.524767+0000","last_change":"2023-11-24T08:12:10.524767+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.524767+0000","last_undegraded":"2023-11-24T08:12:10.524767+0000","last_fullsized":"2023-11-24T08:12:10.524767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:21:52.679838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507818+0000","last_change":"2023-11-24T08:12:09.507818+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507818+0000","last_undegraded":"2023-11-24T08:12:09.507818+0000","last_fullsized":"2023-11-24T08:12:09.507818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520772+0000","last_change":"2023-11-24T08:12:09.520772+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520772+0000","last_undegraded":"2023-11-24T08:12:09.520772+0000","last_fullsized":"2023-11-24T08:12:09.520772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537413+0000","last_change":"2023-11-24T08:12:09.535765+0000","last_active":"2023-11-24T08:12:09.537413+0000","last_peered":"2023-11-24T08:12:09.537413+0000","last_clean":"2023-11-24T08:12:09.537413+0000","last_became_active":"2023-11-24T08:12:09.535385+0000","last_became_peered":"2023-11-24T08:12:09.535385+0000","last_unstale":"2023-11-24T08:12:09.537413+0000","last_undegraded":"2023-11-24T08:12:09.537413+0000","last_fullsized":"2023-11-24T08:12:09.537413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:18.985711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509116+0000","last_change":"2023-11-24T08:12:09.509116+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509116+0000","last_undegraded":"2023-11-24T08:12:09.509116+0000","last_fullsized":"2023-11-24T08:12:09.509116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530987+0000","last_change":"2023-11-24T08:12:09.524544+0000","last_active":"2023-11-24T08:12:09.530987+0000","last_peered":"2023-11-24T08:12:09.530987+0000","last_clean":"2023-11-24T08:12:09.530987+0000","last_became_active":"2023-11-24T08:12:09.524188+0000","last_became_peered":"2023-11-24T08:12:09.524188+0000","last_unstale":"2023-11-24T08:12:09.530987+0000","last_undegraded":"2023-11-24T08:12:09.530987+0000","last_fullsized":"2023-11-24T08:12:09.530987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:56.808696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524895+0000","last_change":"2023-11-24T08:12:09.524895+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524895+0000","last_undegraded":"2023-11-24T08:12:09.524895+0000","last_fullsized":"2023-11-24T08:12:09.524895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538199+0000","last_change":"2023-11-24T08:12:09.533833+0000","last_active":"2023-11-24T08:12:09.538199+0000","last_peered":"2023-11-24T08:12:09.538199+0000","last_clean":"2023-11-24T08:12:09.538199+0000","last_became_active":"2023-11-24T08:12:09.533263+0000","last_became_peered":"2023-11-24T08:12:09.533263+0000","last_unstale":"2023-11-24T08:12:09.538199+0000","last_undegraded":"2023-11-24T08:12:09.538199+0000","last_fullsized":"2023-11-24T08:12:09.538199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:50.328328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522446+0000","last_change":"2023-11-24T08:12:09.522446+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522446+0000","last_undegraded":"2023-11-24T08:12:09.522446+0000","last_fullsized":"2023-11-24T08:12:09.522446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538124+0000","last_change":"2023-11-24T08:12:09.529735+0000","last_active":"2023-11-24T08:12:09.538124+0000","last_peered":"2023-11-24T08:12:09.538124+0000","last_clean":"2023-11-24T08:12:09.538124+0000","last_became_active":"2023-11-24T08:12:09.529517+0000","last_became_peered":"2023-11-24T08:12:09.529517+0000","last_unstale":"2023-11-24T08:12:09.538124+0000","last_undegraded":"2023-11-24T08:12:09.538124+0000","last_fullsized":"2023-11-24T08:12:09.538124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:14.043315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.525674+0000","last_change":"2023-11-24T08:12:10.525674+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.525674+0000","last_undegraded":"2023-11-24T08:12:10.525674+0000","last_fullsized":"2023-11-24T08:12:10.525674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:27.094673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526557+0000","last_change":"2023-11-24T08:12:09.522576+0000","last_active":"2023-11-24T08:12:09.526557+0000","last_peered":"2023-11-24T08:12:09.526557+0000","last_clean":"2023-11-24T08:12:09.526557+0000","last_became_active":"2023-11-24T08:12:09.522114+0000","last_became_peered":"2023-11-24T08:12:09.522114+0000","last_unstale":"2023-11-24T08:12:09.526557+0000","last_undegraded":"2023-11-24T08:12:09.526557+0000","last_fullsized":"2023-11-24T08:12:09.526557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:27:40.788887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503392+0000","last_change":"2023-11-24T08:12:09.503392+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503392+0000","last_undegraded":"2023-11-24T08:12:09.503392+0000","last_fullsized":"2023-11-24T08:12:09.503392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537905+0000","last_change":"2023-11-24T08:12:09.531869+0000","last_active":"2023-11-24T08:12:09.537905+0000","last_peered":"2023-11-24T08:12:09.537905+0000","last_clean":"2023-11-24T08:12:09.537905+0000","last_became_active":"2023-11-24T08:12:09.531598+0000","last_became_peered":"2023-11-24T08:12:09.531598+0000","last_unstale":"2023-11-24T08:12:09.537905+0000","last_undegraded":"2023-11-24T08:12:09.537905+0000","last_fullsized":"2023-11-24T08:12:09.537905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:24.358046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510090+0000","last_change":"2023-11-24T08:12:09.510090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510090+0000","last_undegraded":"2023-11-24T08:12:09.510090+0000","last_fullsized":"2023-11-24T08:12:09.510090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518079+0000","last_change":"2023-11-24T08:12:09.513949+0000","last_active":"2023-11-24T08:12:09.518079+0000","last_peered":"2023-11-24T08:12:09.518079+0000","last_clean":"2023-11-24T08:12:09.518079+0000","last_became_active":"2023-11-24T08:12:09.513517+0000","last_became_peered":"2023-11-24T08:12:09.513517+0000","last_unstale":"2023-11-24T08:12:09.518079+0000","last_undegraded":"2023-11-24T08:12:09.518079+0000","last_fullsized":"2023-11-24T08:12:09.518079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:51.436372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517039+0000","last_change":"2023-11-24T08:12:09.517039+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517039+0000","last_undegraded":"2023-11-24T08:12:09.517039+0000","last_fullsized":"2023-11-24T08:12:09.517039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.524795+0000","last_change":"2023-11-24T08:12:09.519208+0000","last_active":"2023-11-24T08:12:09.524795+0000","last_peered":"2023-11-24T08:12:09.524795+0000","last_clean":"2023-11-24T08:12:09.524795+0000","last_became_active":"2023-11-24T08:12:09.518952+0000","last_became_peered":"2023-11-24T08:12:09.518952+0000","last_unstale":"2023-11-24T08:12:09.524795+0000","last_undegraded":"2023-11-24T08:12:09.524795+0000","last_fullsized":"2023-11-24T08:12:09.524795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:57:11.824082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.542676+0000","last_change":"2023-11-24T08:12:10.542676+0000","last_active":"2023-11-24T08:12:10.542676+0000","last_peered":"2023-11-24T08:12:10.542676+0000","last_clean":"2023-11-24T08:12:10.542676+0000","last_became_active":"2023-11-24T08:12:10.542269+0000","last_became_peered":"2023-11-24T08:12:10.542269+0000","last_unstale":"2023-11-24T08:12:10.542676+0000","last_undegraded":"2023-11-24T08:12:10.542676+0000","last_fullsized":"2023-11-24T08:12:10.542676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:38.761881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.525878+0000","last_change":"2023-11-24T08:12:09.519666+0000","last_active":"2023-11-24T08:12:09.525878+0000","last_peered":"2023-11-24T08:12:09.525878+0000","last_clean":"2023-11-24T08:12:09.525878+0000","last_became_active":"2023-11-24T08:12:09.519348+0000","last_became_peered":"2023-11-24T08:12:09.519348+0000","last_unstale":"2023-11-24T08:12:09.525878+0000","last_undegraded":"2023-11-24T08:12:09.525878+0000","last_fullsized":"2023-11-24T08:12:09.525878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:08.548384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524304+0000","last_change":"2023-11-24T08:12:09.524304+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524304+0000","last_undegraded":"2023-11-24T08:12:09.524304+0000","last_fullsized":"2023-11-24T08:12:09.524304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538991+0000","last_change":"2023-11-24T08:12:09.537826+0000","last_active":"2023-11-24T08:12:09.538991+0000","last_peered":"2023-11-24T08:12:09.538991+0000","last_clean":"2023-11-24T08:12:09.538991+0000","last_became_active":"2023-11-24T08:12:09.537555+0000","last_became_peered":"2023-11-24T08:12:09.537555+0000","last_unstale":"2023-11-24T08:12:09.538991+0000","last_undegraded":"2023-11-24T08:12:09.538991+0000","last_fullsized":"2023-11-24T08:12:09.538991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:54:04.166572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.515379+0000","last_change":"2023-11-24T08:12:09.515379+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.515379+0000","last_undegraded":"2023-11-24T08:12:09.515379+0000","last_fullsized":"2023-11-24T08:12:09.515379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531034+0000","last_change":"2023-11-24T08:12:09.524384+0000","last_active":"2023-11-24T08:12:09.531034+0000","last_peered":"2023-11-24T08:12:09.531034+0000","last_clean":"2023-11-24T08:12:09.531034+0000","last_became_active":"2023-11-24T08:12:09.524038+0000","last_became_peered":"2023-11-24T08:12:09.524038+0000","last_unstale":"2023-11-24T08:12:09.531034+0000","last_undegraded":"2023-11-24T08:12:09.531034+0000","last_fullsized":"2023-11-24T08:12:09.531034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:45.336925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.513394+0000","last_change":"2023-11-24T08:12:09.513394+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.513394+0000","last_undegraded":"2023-11-24T08:12:09.513394+0000","last_fullsized":"2023-11-24T08:12:09.513394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518054+0000","last_change":"2023-11-24T08:12:09.514045+0000","last_active":"2023-11-24T08:12:09.518054+0000","last_peered":"2023-11-24T08:12:09.518054+0000","last_clean":"2023-11-24T08:12:09.518054+0000","last_became_active":"2023-11-24T08:12:09.513337+0000","last_became_peered":"2023-11-24T08:12:09.513337+0000","last_unstale":"2023-11-24T08:12:09.518054+0000","last_undegraded":"2023-11-24T08:12:09.518054+0000","last_fullsized":"2023-11-24T08:12:09.518054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:17.346075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.518834+0000","last_change":"2023-11-24T08:12:09.518834+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.518834+0000","last_undegraded":"2023-11-24T08:12:09.518834+0000","last_fullsized":"2023-11-24T08:12:09.518834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530522+0000","last_change":"2023-11-24T08:12:09.528456+0000","last_active":"2023-11-24T08:12:09.530522+0000","last_peered":"2023-11-24T08:12:09.530522+0000","last_clean":"2023-11-24T08:12:09.530522+0000","last_became_active":"2023-11-24T08:12:09.528140+0000","last_became_peered":"2023-11-24T08:12:09.528140+0000","last_unstale":"2023-11-24T08:12:09.530522+0000","last_undegraded":"2023-11-24T08:12:09.530522+0000","last_fullsized":"2023-11-24T08:12:09.530522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:12.633925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.529872+0000","last_change":"2023-11-24T08:12:10.529872+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.529872+0000","last_undegraded":"2023-11-24T08:12:10.529872+0000","last_fullsized":"2023-11-24T08:12:10.529872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:09.099806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538979+0000","last_change":"2023-11-24T08:12:09.537450+0000","last_active":"2023-11-24T08:12:09.538979+0000","last_peered":"2023-11-24T08:12:09.538979+0000","last_clean":"2023-11-24T08:12:09.538979+0000","last_became_active":"2023-11-24T08:12:09.537196+0000","last_became_peered":"2023-11-24T08:12:09.537196+0000","last_unstale":"2023-11-24T08:12:09.538979+0000","last_undegraded":"2023-11-24T08:12:09.538979+0000","last_fullsized":"2023-11-24T08:12:09.538979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:19.005358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517657+0000","last_change":"2023-11-24T08:12:09.517657+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517657+0000","last_undegraded":"2023-11-24T08:12:09.517657+0000","last_fullsized":"2023-11-24T08:12:09.517657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533269+0000","last_change":"2023-11-24T08:12:09.530752+0000","last_active":"2023-11-24T08:12:10.533269+0000","last_peered":"2023-11-24T08:12:10.533269+0000","last_clean":"2023-11-24T08:12:10.533269+0000","last_became_active":"2023-11-24T08:12:09.529282+0000","last_became_peered":"2023-11-24T08:12:09.529282+0000","last_unstale":"2023-11-24T08:12:10.533269+0000","last_undegraded":"2023-11-24T08:12:10.533269+0000","last_fullsized":"2023-11-24T08:12:10.533269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:31.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510752+0000","last_change":"2023-11-24T08:12:09.510752+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510752+0000","last_undegraded":"2023-11-24T08:12:09.510752+0000","last_fullsized":"2023-11-24T08:12:09.510752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.534559+0000","last_change":"2023-11-24T08:12:09.530860+0000","last_active":"2023-11-24T08:12:09.534559+0000","last_peered":"2023-11-24T08:12:09.534559+0000","last_clean":"2023-11-24T08:12:09.534559+0000","last_became_active":"2023-11-24T08:12:09.530167+0000","last_became_peered":"2023-11-24T08:12:09.530167+0000","last_unstale":"2023-11-24T08:12:09.534559+0000","last_undegraded":"2023-11-24T08:12:09.534559+0000","last_fullsized":"2023-11-24T08:12:09.534559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:08.559029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.504790+0000","last_change":"2023-11-24T08:12:09.504790+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.504790+0000","last_undegraded":"2023-11-24T08:12:09.504790+0000","last_fullsized":"2023-11-24T08:12:09.504790+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.539017+0000","last_change":"2023-11-24T08:12:09.538063+0000","last_active":"2023-11-24T08:12:09.539017+0000","last_peered":"2023-11-24T08:12:09.539017+0000","last_clean":"2023-11-24T08:12:09.539017+0000","last_became_active":"2023-11-24T08:12:09.536932+0000","last_became_peered":"2023-11-24T08:12:09.536932+0000","last_unstale":"2023-11-24T08:12:09.539017+0000","last_undegraded":"2023-11-24T08:12:09.539017+0000","last_fullsized":"2023-11-24T08:12:09.539017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:31.591358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523784+0000","last_change":"2023-11-24T08:12:09.523784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523784+0000","last_undegraded":"2023-11-24T08:12:09.523784+0000","last_fullsized":"2023-11-24T08:12:09.523784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538170+0000","last_change":"2023-11-24T08:12:09.533762+0000","last_active":"2023-11-24T08:12:09.538170+0000","last_peered":"2023-11-24T08:12:09.538170+0000","last_clean":"2023-11-24T08:12:09.538170+0000","last_became_active":"2023-11-24T08:12:09.533465+0000","last_became_peered":"2023-11-24T08:12:09.533465+0000","last_unstale":"2023-11-24T08:12:09.538170+0000","last_undegraded":"2023-11-24T08:12:09.538170+0000","last_fullsized":"2023-11-24T08:12:09.538170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:44.590676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505924+0000","last_change":"2023-11-24T08:12:09.505924+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505924+0000","last_undegraded":"2023-11-24T08:12:09.505924+0000","last_fullsized":"2023-11-24T08:12:09.505924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537888+0000","last_change":"2023-11-24T08:12:09.530831+0000","last_active":"2023-11-24T08:12:09.537888+0000","last_peered":"2023-11-24T08:12:09.537888+0000","last_clean":"2023-11-24T08:12:09.537888+0000","last_became_active":"2023-11-24T08:12:09.530459+0000","last_became_peered":"2023-11-24T08:12:09.530459+0000","last_unstale":"2023-11-24T08:12:09.537888+0000","last_undegraded":"2023-11-24T08:12:09.537888+0000","last_fullsized":"2023-11-24T08:12:09.537888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:40.045785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509492+0000","last_change":"2023-11-24T08:12:09.509492+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509492+0000","last_undegraded":"2023-11-24T08:12:09.509492+0000","last_fullsized":"2023-11-24T08:12:09.509492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537541+0000","last_change":"2023-11-24T08:12:09.535763+0000","last_active":"2023-11-24T08:12:09.537541+0000","last_peered":"2023-11-24T08:12:09.537541+0000","last_clean":"2023-11-24T08:12:09.537541+0000","last_became_active":"2023-11-24T08:12:09.535048+0000","last_became_peered":"2023-11-24T08:12:09.535048+0000","last_unstale":"2023-11-24T08:12:09.537541+0000","last_undegraded":"2023-11-24T08:12:09.537541+0000","last_fullsized":"2023-11-24T08:12:09.537541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:42.819752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511207+0000","last_change":"2023-11-24T08:12:09.511207+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511207+0000","last_undegraded":"2023-11-24T08:12:09.511207+0000","last_fullsized":"2023-11-24T08:12:09.511207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538446+0000","last_change":"2023-11-24T08:12:09.536935+0000","last_active":"2023-11-24T08:12:09.538446+0000","last_peered":"2023-11-24T08:12:09.538446+0000","last_clean":"2023-11-24T08:12:09.538446+0000","last_became_active":"2023-11-24T08:12:09.535354+0000","last_became_peered":"2023-11-24T08:12:09.535354+0000","last_unstale":"2023-11-24T08:12:09.538446+0000","last_undegraded":"2023-11-24T08:12:09.538446+0000","last_fullsized":"2023-11-24T08:12:09.538446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:49.306476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507471+0000","last_change":"2023-11-24T08:12:09.507471+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507471+0000","last_undegraded":"2023-11-24T08:12:09.507471+0000","last_fullsized":"2023-11-24T08:12:09.507471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533487+0000","last_change":"2023-11-24T08:12:09.530273+0000","last_active":"2023-11-24T08:12:10.533487+0000","last_peered":"2023-11-24T08:12:10.533487+0000","last_clean":"2023-11-24T08:12:10.533487+0000","last_became_active":"2023-11-24T08:12:09.529967+0000","last_became_peered":"2023-11-24T08:12:09.529967+0000","last_unstale":"2023-11-24T08:12:10.533487+0000","last_undegraded":"2023-11-24T08:12:10.533487+0000","last_fullsized":"2023-11-24T08:12:10.533487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:37.818727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508506+0000","last_change":"2023-11-24T08:12:09.508506+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508506+0000","last_undegraded":"2023-11-24T08:12:09.508506+0000","last_fullsized":"2023-11-24T08:12:09.508506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.535270+0000","last_change":"2023-11-24T08:12:09.530296+0000","last_active":"2023-11-24T08:12:09.535270+0000","last_peered":"2023-11-24T08:12:09.535270+0000","last_clean":"2023-11-24T08:12:09.535270+0000","last_became_active":"2023-11-24T08:12:09.529840+0000","last_became_peered":"2023-11-24T08:12:09.529840+0000","last_unstale":"2023-11-24T08:12:09.535270+0000","last_undegraded":"2023-11-24T08:12:09.535270+0000","last_fullsized":"2023-11-24T08:12:09.535270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:42.718534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519744+0000","last_change":"2023-11-24T08:12:09.519744+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519744+0000","last_undegraded":"2023-11-24T08:12:09.519744+0000","last_fullsized":"2023-11-24T08:12:09.519744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538516+0000","last_change":"2023-11-24T08:12:09.536994+0000","last_active":"2023-11-24T08:12:09.538516+0000","last_peered":"2023-11-24T08:12:09.538516+0000","last_clean":"2023-11-24T08:12:09.538516+0000","last_became_active":"2023-11-24T08:12:09.536664+0000","last_became_peered":"2023-11-24T08:12:09.536664+0000","last_unstale":"2023-11-24T08:12:09.538516+0000","last_undegraded":"2023-11-24T08:12:09.538516+0000","last_fullsized":"2023-11-24T08:12:09.538516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:10.151500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.533267+0000","last_change":"2023-11-24T08:12:10.533267+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.533267+0000","last_undegraded":"2023-11-24T08:12:10.533267+0000","last_fullsized":"2023-11-24T08:12:10.533267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:10.427272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.534767+0000","last_change":"2023-11-24T08:12:09.534163+0000","last_active":"2023-11-24T08:12:10.534767+0000","last_peered":"2023-11-24T08:12:10.534767+0000","last_clean":"2023-11-24T08:12:10.534767+0000","last_became_active":"2023-11-24T08:12:09.533384+0000","last_became_peered":"2023-11-24T08:12:09.533384+0000","last_unstale":"2023-11-24T08:12:10.534767+0000","last_undegraded":"2023-11-24T08:12:10.534767+0000","last_fullsized":"2023-11-24T08:12:10.534767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:39.337647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522771+0000","last_change":"2023-11-24T08:12:09.522771+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522771+0000","last_undegraded":"2023-11-24T08:12:09.522771+0000","last_fullsized":"2023-11-24T08:12:09.522771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.529558+0000","last_change":"2023-11-24T08:12:09.524480+0000","last_active":"2023-11-24T08:12:09.529558+0000","last_peered":"2023-11-24T08:12:09.529558+0000","last_clean":"2023-11-24T08:12:09.529558+0000","last_became_active":"2023-11-24T08:12:09.524212+0000","last_became_peered":"2023-11-24T08:12:09.524212+0000","last_unstale":"2023-11-24T08:12:09.529558+0000","last_undegraded":"2023-11-24T08:12:09.529558+0000","last_fullsized":"2023-11-24T08:12:09.529558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:57.611814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520763+0000","last_change":"2023-11-24T08:12:09.520763+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520763+0000","last_undegraded":"2023-11-24T08:12:09.520763+0000","last_fullsized":"2023-11-24T08:12:09.520763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.532684+0000","last_change":"2023-11-24T08:12:09.530037+0000","last_active":"2023-11-24T08:12:09.532684+0000","last_peered":"2023-11-24T08:12:09.532684+0000","last_clean":"2023-11-24T08:12:09.532684+0000","last_became_active":"2023-11-24T08:12:09.529803+0000","last_became_peered":"2023-11-24T08:12:09.529803+0000","last_unstale":"2023-11-24T08:12:09.532684+0000","last_undegraded":"2023-11-24T08:12:09.532684+0000","last_fullsized":"2023-11-24T08:12:09.532684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:48.699065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510858+0000","last_change":"2023-11-24T08:12:09.510858+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510858+0000","last_undegraded":"2023-11-24T08:12:09.510858+0000","last_fullsized":"2023-11-24T08:12:09.510858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526913+0000","last_change":"2023-11-24T08:12:09.522451+0000","last_active":"2023-11-24T08:12:09.526913+0000","last_peered":"2023-11-24T08:12:09.526913+0000","last_clean":"2023-11-24T08:12:09.526913+0000","last_became_active":"2023-11-24T08:12:09.521531+0000","last_became_peered":"2023-11-24T08:12:09.521531+0000","last_unstale":"2023-11-24T08:12:09.526913+0000","last_undegraded":"2023-11-24T08:12:09.526913+0000","last_fullsized":"2023-11-24T08:12:09.526913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.342039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516090+0000","last_change":"2023-11-24T08:12:09.516090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516090+0000","last_undegraded":"2023-11-24T08:12:09.516090+0000","last_fullsized":"2023-11-24T08:12:09.516090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.527191+0000","last_change":"2023-11-24T08:12:09.521396+0000","last_active":"2023-11-24T08:12:09.527191+0000","last_peered":"2023-11-24T08:12:09.527191+0000","last_clean":"2023-11-24T08:12:09.527191+0000","last_became_active":"2023-11-24T08:12:09.520385+0000","last_became_peered":"2023-11-24T08:12:09.520385+0000","last_unstale":"2023-11-24T08:12:09.527191+0000","last_undegraded":"2023-11-24T08:12:09.527191+0000","last_fullsized":"2023-11-24T08:12:09.527191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:25.592586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506002+0000","last_change":"2023-11-24T08:12:09.506002+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506002+0000","last_undegraded":"2023-11-24T08:12:09.506002+0000","last_fullsized":"2023-11-24T08:12:09.506002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531702+0000","last_change":"2023-11-24T08:12:09.529444+0000","last_active":"2023-11-24T08:12:09.531702+0000","last_peered":"2023-11-24T08:12:09.531702+0000","last_clean":"2023-11-24T08:12:09.531702+0000","last_became_active":"2023-11-24T08:12:09.528128+0000","last_became_peered":"2023-11-24T08:12:09.528128+0000","last_unstale":"2023-11-24T08:12:09.531702+0000","last_undegraded":"2023-11-24T08:12:09.531702+0000","last_fullsized":"2023-11-24T08:12:09.531702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:47:06.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511825+0000","last_change":"2023-11-24T08:12:09.511825+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511825+0000","last_undegraded":"2023-11-24T08:12:09.511825+0000","last_fullsized":"2023-11-24T08:12:09.511825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538378+0000","last_change":"2023-11-24T08:12:09.535030+0000","last_active":"2023-11-24T08:12:09.538378+0000","last_peered":"2023-11-24T08:12:09.538378+0000","last_clean":"2023-11-24T08:12:09.538378+0000","last_became_active":"2023-11-24T08:12:09.530814+0000","last_became_peered":"2023-11-24T08:12:09.530814+0000","last_unstale":"2023-11-24T08:12:09.538378+0000","last_undegraded":"2023-11-24T08:12:09.538378+0000","last_fullsized":"2023-11-24T08:12:09.538378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:52.597123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508831+0000","last_change":"2023-11-24T08:12:09.508831+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508831+0000","last_undegraded":"2023-11-24T08:12:09.508831+0000","last_fullsized":"2023-11-24T08:12:09.508831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538420+0000","last_change":"2023-11-24T08:12:09.535433+0000","last_active":"2023-11-24T08:12:09.538420+0000","last_peered":"2023-11-24T08:12:09.538420+0000","last_clean":"2023-11-24T08:12:09.538420+0000","last_became_active":"2023-11-24T08:12:09.534941+0000","last_became_peered":"2023-11-24T08:12:09.534941+0000","last_unstale":"2023-11-24T08:12:09.538420+0000","last_undegraded":"2023-11-24T08:12:09.538420+0000","last_fullsized":"2023-11-24T08:12:09.538420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:06.870016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":205,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T08:12:08.502638+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:08.502638+0000","last_peered":"2023-11-24T08:12:08.502638+0000","last_clean":"2023-11-24T08:12:08.502638+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:08.502638+0000","last_undegraded":"2023-11-24T08:12:08.502638+0000","last_fullsized":"2023-11-24T08:12:08.502638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.014,"15min":1.012},"min":{"1min":0.64300000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.456,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.367},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.03,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3340000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.125},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5149999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.157},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1739999999999999,"15min":1.1659999999999999},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.454,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1120000000000001,"15min":1.0660000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.534,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.214,"15min":1.206},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.03}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.246,"15min":1.236},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.3959999999999999,"5min":3.387,"15min":3.387},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.246,"15min":1.22},"min":{"1min":0.90500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.583,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.3140000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.234,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.448,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.391,"15min":1.3759999999999999},"min":{"1min":0.96599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6259999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4039999999999999,"15min":1.427},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6399999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.25},{"interface":"front","average":{"1min":1.391,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.129,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.698,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5069999999999999,"15min":1.4830000000000001},"min":{"1min":1.3759999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.806,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.542},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.492,"15min":1.4790000000000001},"min":{"1min":1.1100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.728,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0780000000000001,"15min":1.095},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.472,"5min":2.085,"15min":2.085},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.214,"15min":1.2370000000000001},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4850000000000001,"5min":1.794,"15min":1.794},"last":0.93000000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.70999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6599999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.413},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.80400000000000005,"5min":0.749,"15min":0.749},"max":{"1min":1.7370000000000001,"5min":1.98,"15min":1.98},"last":0.88}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2250000000000001,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6950000000000001,"5min":1.778,"15min":1.778},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6899999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7849999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.446},"min":{"1min":0.98299999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.847,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3500000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.343,"15min":1.369},"min":{"1min":0.77600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.242,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4359999999999999,"5min":2.661,"15min":2.661},"last":1.5700000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.625,"15min":1.708},"min":{"1min":1.151,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.5900000000000001,"15min":1.651},"min":{"1min":1.0249999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.921,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.4970000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.726,"15min":1.78},"min":{"1min":1.2949999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.016,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.7250000000000001,"15min":1.7849999999999999},"min":{"1min":1.2050000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1499999999999999,"5min":2.657,"15min":2.657},"last":1.722}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0529999999999999,"15min":1.0249999999999999},"min":{"1min":0.72399999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6100000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.133,"15min":1.157},"min":{"1min":0.77700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":1.837},"last":1.454}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1890000000000001,"15min":1.198},"min":{"1min":0.82799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4850000000000001,"5min":1.722,"15min":1.722},"last":1.3700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2649999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4970000000000001,"5min":2.085,"15min":2.085},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3620000000000001},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.399,"15min":1.4339999999999999},"min":{"1min":0.95399999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.573,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.468,"15min":1.5169999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.7130000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.397}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":1.1259999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8500000000000001,"5min":3.577,"15min":3.577},"last":1.153},{"interface":"front","average":{"1min":1.49,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.0620000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.958,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5780000000000001,"15min":1.6060000000000001},"min":{"1min":1.0209999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.869,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.484},{"interface":"front","average":{"1min":1.554,"5min":1.5489999999999999,"15min":1.5409999999999999},"min":{"1min":1.0089999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0030000000000001,"5min":2.512,"15min":2.512},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.6839999999999999,"15min":1.716},"min":{"1min":1.21,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.8839999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.708,"15min":1.7370000000000001},"min":{"1min":1.2949999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.9390000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.581}]}]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.01,"15min":0.98699999999999999},"min":{"1min":0.64700000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6240000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.099,"15min":1.093},"min":{"1min":0.71999999999999997,"5min":0.626,"15min":0.626},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2310000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6140000000000001,"5min":1.859,"15min":1.859},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1919999999999999,"15min":1.1639999999999999},"min":{"1min":0.79400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1230000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.625}]},{"osd":2,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2749999999999999},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.746},{"interface":"front","average":{"1min":1.115,"5min":1.099,"15min":1.105},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8879999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78900000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.659,"5min":2.25,"15min":2.25},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.86599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8360000000000001,"5min":2.746,"15min":2.746},"last":1.8120000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.331},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.452,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8440000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4259999999999999,"15min":1.446},"min":{"1min":0.73799999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.4809999999999999,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.532},{"interface":"front","average":{"1min":1.47,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8500000000000001,"5min":2.641,"15min":2.641},"last":1.393}]},{"osd":7,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5149999999999999,"15min":1.5129999999999999},"min":{"1min":0.69599999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2269999999999999,"5min":2.548,"15min":2.548},"last":1.865},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.534,"15min":1.5489999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.4470000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0489999999999999,"15min":1.145},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3180000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0589999999999999,"15min":1.143},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.085,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.95899999999999996,"15min":0.995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4630000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.534,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.5740000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.1559999999999999,"15min":1.22},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5269999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.385,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.631,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.105},{"interface":"front","average":{"1min":1.141,"5min":1.337,"15min":1.373},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4670000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.125}]},{"osd":5,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.5069999999999999,"15min":1.5680000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.486,"15min":1.536},"min":{"1min":0.872,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8700000000000001,"5min":3.339,"15min":3.339},"last":1.4610000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.5609999999999999,"15min":1.6659999999999999},"min":{"1min":0.746,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.742,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.522,"15min":1.5580000000000001},"min":{"1min":1.0249999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.681,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.403}]},{"osd":7,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.605,"15min":1.675},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.603,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.8320000000000001,"15min":2.016},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7629999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":365703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.47899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.21,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.81699999999999995,"15min":0.77700000000000002},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6950000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.6679999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.244,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.92800000000000005},"min":{"1min":0.58999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.514,"5min":1.671,"15min":1.671},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98599999999999999,"15min":0.97399999999999998},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.387,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.254},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.232,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1699999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.421},{"interface":"front","average":{"1min":1.294,"5min":1.23,"15min":1.224},"min":{"1min":0.876,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.389}]},{"osd":5,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3620000000000001,"15min":1.431},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.629,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.294,"15min":1.286},"min":{"1min":0.872,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.516}]},{"osd":6,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3839999999999999,"15min":1.4159999999999999},"min":{"1min":0.94799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.895,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3260000000000001,"15min":1.337},"min":{"1min":0.93500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9490000000000001,"5min":1.958,"15min":1.958},"last":1.3149999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4770000000000001},"min":{"1min":0.91000000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.99,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.87,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0739999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5449999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.171},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1359999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.121},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.024,"15min":1.0109999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.391,"5min":1.621,"15min":1.621},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.143,"15min":1.1579999999999999},"min":{"1min":0.66100000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.534,"5min":1.907,"15min":1.907},"last":1.228},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.018,"15min":1.046},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3740000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.022}]},{"osd":4,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.161,"15min":1.163},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.425,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.874,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6919999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.83399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.653,"5min":2.661,"15min":2.661},"last":1.478},{"interface":"front","average":{"1min":1.258,"5min":1.3120000000000001,"15min":1.3360000000000001},"min":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.788,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.409},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.343,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.7070000000000001,"5min":2.105,"15min":2.105},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4099999999999999,"15min":1.4039999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8600000000000001,"5min":2.008,"15min":2.008},"last":1.6830000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1739999999999999,"5min":1.403,"15min":1.403},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95399999999999996},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2989999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.327},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5209999999999999,"5min":1.986,"15min":1.986},"last":1.081}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0489999999999999,"15min":1.0720000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3380000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.96099999999999997},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.23,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.171},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.641,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.137},{"interface":"front","average":{"1min":1.272,"5min":1.2869999999999999,"15min":1.29},"min":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.78,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.216}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.202,"15min":1.22},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4550000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8220000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.474}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.272},"min":{"1min":0.81299999999999994,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.546,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.284,"15min":1.286},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.742,"5min":2.149,"15min":2.149},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3979999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.865,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3839999999999999,"15min":1.4119999999999999},"min":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.603,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:12:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-11-24T08:12:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.524784+0000', 'last_change': '2023-11-24T08:12:09.524784+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.524784+0000', 'last_undegraded': '2023-11-24T08:12:09.524784+0000', 'last_fullsized': '2023-11-24T08:12:09.524784+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-11-24T08:12:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.517595+0000', 'last_change': '2023-11-24T08:12:09.517595+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.517595+0000', 'last_undegraded': '2023-11-24T08:12:09.517595+0000', 'last_fullsized': '2023-11-24T08:12:09.517595+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.968 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.508940+0000', 'last_change': '2023-11-24T08:12:09.508940+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.508940+0000', 'last_undegraded': '2023-11-24T08:12:09.508940+0000', 'last_fullsized': '2023-11-24T08:12:09.508940+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.508376+0000', 'last_change': '2023-11-24T08:12:09.508376+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.508376+0000', 'last_undegraded': '2023-11-24T08:12:09.508376+0000', 'last_fullsized': '2023-11-24T08:12:09.508376+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.511217+0000', 'last_change': '2023-11-24T08:12:09.511217+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.511217+0000', 'last_undegraded': '2023-11-24T08:12:09.511217+0000', 'last_fullsized': '2023-11-24T08:12:09.511217+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.514317+0000', 'last_change': '2023-11-24T08:12:09.514317+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.514317+0000', 'last_undegraded': '2023-11-24T08:12:09.514317+0000', 'last_fullsized': '2023-11-24T08:12:09.514317+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 74, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:12:10.532518+0000', 'last_change': '2023-11-24T08:12:10.532518+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:10.532518+0000', 'last_undegraded': '2023-11-24T08:12:10.532518+0000', 'last_fullsized': '2023-11-24T08:12:10.532518+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T13:29:09.759327+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:12:15.969 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.511431+0000', 'last_change': '2023-11-24T08:12:09.511431+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.511431+0000', 'last_undegraded': '2023-11-24T08:12:09.511431+0000', 'last_fullsized': '2023-11-24T08:12:09.511431+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.510124+0000', 'last_change': '2023-11-24T08:12:09.510124+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.510124+0000', 'last_undegraded': '2023-11-24T08:12:09.510124+0000', 'last_fullsized': '2023-11-24T08:12:09.510124+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.505932+0000', 'last_change': '2023-11-24T08:12:09.505932+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.505932+0000', 'last_undegraded': '2023-11-24T08:12:09.505932+0000', 'last_fullsized': '2023-11-24T08:12:09.505932+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.523117+0000', 'last_change': '2023-11-24T08:12:09.523117+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.523117+0000', 'last_undegraded': '2023-11-24T08:12:09.523117+0000', 'last_fullsized': '2023-11-24T08:12:09.523117+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-11-24T08:12:15.970 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.507941+0000', 'last_change': '2023-11-24T08:12:09.507941+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.507941+0000', 'last_undegraded': '2023-11-24T08:12:09.507941+0000', 'last_fullsized': '2023-11-24T08:12:09.507941+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.516971+0000', 'last_change': '2023-11-24T08:12:09.516971+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.516971+0000', 'last_undegraded': '2023-11-24T08:12:09.516971+0000', 'last_fullsized': '2023-11-24T08:12:09.516971+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.511315+0000', 'last_change': '2023-11-24T08:12:09.511315+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.511315+0000', 'last_undegraded': '2023-11-24T08:12:09.511315+0000', 'last_fullsized': '2023-11-24T08:12:09.511315+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.519786+0000', 'last_change': '2023-11-24T08:12:09.519786+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.519786+0000', 'last_undegraded': '2023-11-24T08:12:09.519786+0000', 'last_fullsized': '2023-11-24T08:12:09.519786+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.503356+0000', 'last_change': '2023-11-24T08:12:09.503356+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.503356+0000', 'last_undegraded': '2023-11-24T08:12:09.503356+0000', 'last_fullsized': '2023-11-24T08:12:09.503356+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-11-24T08:12:15.971 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.509693+0000', 'last_change': '2023-11-24T08:12:09.509693+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.509693+0000', 'last_undegraded': '2023-11-24T08:12:09.509693+0000', 'last_fullsized': '2023-11-24T08:12:09.509693+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.506814+0000', 'last_change': '2023-11-24T08:12:09.506814+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.506814+0000', 'last_undegraded': '2023-11-24T08:12:09.506814+0000', 'last_fullsized': '2023-11-24T08:12:09.506814+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.524401+0000', 'last_change': '2023-11-24T08:12:09.524401+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.524401+0000', 'last_undegraded': '2023-11-24T08:12:09.524401+0000', 'last_fullsized': '2023-11-24T08:12:09.524401+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.516989+0000', 'last_change': '2023-11-24T08:12:09.516989+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.516989+0000', 'last_undegraded': '2023-11-24T08:12:09.516989+0000', 'last_fullsized': '2023-11-24T08:12:09.516989+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.506189+0000', 'last_change': '2023-11-24T08:12:09.506189+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.506189+0000', 'last_undegraded': '2023-11-24T08:12:09.506189+0000', 'last_fullsized': '2023-11-24T08:12:09.506189+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.972 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.509465+0000', 'last_change': '2023-11-24T08:12:09.509465+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.509465+0000', 'last_undegraded': '2023-11-24T08:12:09.509465+0000', 'last_fullsized': '2023-11-24T08:12:09.509465+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.520990+0000', 'last_change': '2023-11-24T08:12:09.520990+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.520990+0000', 'last_undegraded': '2023-11-24T08:12:09.520990+0000', 'last_fullsized': '2023-11-24T08:12:09.520990+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.524248+0000', 'last_change': '2023-11-24T08:12:09.524248+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.524248+0000', 'last_undegraded': '2023-11-24T08:12:09.524248+0000', 'last_fullsized': '2023-11-24T08:12:09.524248+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.510034+0000', 'last_change': '2023-11-24T08:12:09.510034+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.510034+0000', 'last_undegraded': '2023-11-24T08:12:09.510034+0000', 'last_fullsized': '2023-11-24T08:12:09.510034+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-11-24T08:12:15.973 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 74, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:12:10.531598+0000', 'last_change': '2023-11-24T08:12:10.531598+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:10.531598+0000', 'last_undegraded': '2023-11-24T08:12:10.531598+0000', 'last_fullsized': '2023-11-24T08:12:10.531598+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T17:24:31.654975+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.508296+0000', 'last_change': '2023-11-24T08:12:09.508296+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.508296+0000', 'last_undegraded': '2023-11-24T08:12:09.508296+0000', 'last_fullsized': '2023-11-24T08:12:09.508296+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.503786+0000', 'last_change': '2023-11-24T08:12:09.503786+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.503786+0000', 'last_undegraded': '2023-11-24T08:12:09.503786+0000', 'last_fullsized': '2023-11-24T08:12:09.503786+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.511659+0000', 'last_change': '2023-11-24T08:12:09.511659+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.511659+0000', 'last_undegraded': '2023-11-24T08:12:09.511659+0000', 'last_fullsized': '2023-11-24T08:12:09.511659+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 74, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:12:10.524767+0000', 'last_change': '2023-11-24T08:12:10.524767+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:10.524767+0000', 'last_undegraded': '2023-11-24T08:12:10.524767+0000', 'last_fullsized': '2023-11-24T08:12:10.524767+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T10:21:52.679838+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': ['5', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:12:15.974 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.507818+0000', 'last_change': '2023-11-24T08:12:09.507818+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.507818+0000', 'last_undegraded': '2023-11-24T08:12:09.507818+0000', 'last_fullsized': '2023-11-24T08:12:09.507818+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.520772+0000', 'last_change': '2023-11-24T08:12:09.520772+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.520772+0000', 'last_undegraded': '2023-11-24T08:12:09.520772+0000', 'last_fullsized': '2023-11-24T08:12:09.520772+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.509116+0000', 'last_change': '2023-11-24T08:12:09.509116+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.509116+0000', 'last_undegraded': '2023-11-24T08:12:09.509116+0000', 'last_fullsized': '2023-11-24T08:12:09.509116+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-24T08:12:15.975 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.524895+0000', 'last_change': '2023-11-24T08:12:09.524895+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.524895+0000', 'last_undegraded': '2023-11-24T08:12:09.524895+0000', 'last_fullsized': '2023-11-24T08:12:09.524895+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-24T08:12:15.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.522446+0000', 'last_change': '2023-11-24T08:12:09.522446+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.522446+0000', 'last_undegraded': '2023-11-24T08:12:09.522446+0000', 'last_fullsized': '2023-11-24T08:12:09.522446+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-24T08:12:15.977 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 74, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:12:10.525674+0000', 'last_change': '2023-11-24T08:12:10.525674+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:10.525674+0000', 'last_undegraded': '2023-11-24T08:12:10.525674+0000', 'last_fullsized': '2023-11-24T08:12:10.525674+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T11:43:27.094673+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:12:15.977 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.503392+0000', 'last_change': '2023-11-24T08:12:09.503392+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.503392+0000', 'last_undegraded': '2023-11-24T08:12:09.503392+0000', 'last_fullsized': '2023-11-24T08:12:09.503392+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.510090+0000', 'last_change': '2023-11-24T08:12:09.510090+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.510090+0000', 'last_undegraded': '2023-11-24T08:12:09.510090+0000', 'last_fullsized': '2023-11-24T08:12:09.510090+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.517039+0000', 'last_change': '2023-11-24T08:12:09.517039+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.517039+0000', 'last_undegraded': '2023-11-24T08:12:09.517039+0000', 'last_fullsized': '2023-11-24T08:12:09.517039+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.524304+0000', 'last_change': '2023-11-24T08:12:09.524304+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.524304+0000', 'last_undegraded': '2023-11-24T08:12:09.524304+0000', 'last_fullsized': '2023-11-24T08:12:09.524304+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-24T08:12:15.978 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.515379+0000', 'last_change': '2023-11-24T08:12:09.515379+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.515379+0000', 'last_undegraded': '2023-11-24T08:12:09.515379+0000', 'last_fullsized': '2023-11-24T08:12:09.515379+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.513394+0000', 'last_change': '2023-11-24T08:12:09.513394+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.513394+0000', 'last_undegraded': '2023-11-24T08:12:09.513394+0000', 'last_fullsized': '2023-11-24T08:12:09.513394+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.518834+0000', 'last_change': '2023-11-24T08:12:09.518834+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.518834+0000', 'last_undegraded': '2023-11-24T08:12:09.518834+0000', 'last_fullsized': '2023-11-24T08:12:09.518834+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 74, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:12:10.529872+0000', 'last_change': '2023-11-24T08:12:10.529872+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:10.529872+0000', 'last_undegraded': '2023-11-24T08:12:10.529872+0000', 'last_fullsized': '2023-11-24T08:12:10.529872+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T09:59:09.099806+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.517657+0000', 'last_change': '2023-11-24T08:12:09.517657+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.517657+0000', 'last_undegraded': '2023-11-24T08:12:09.517657+0000', 'last_fullsized': '2023-11-24T08:12:09.517657+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:12:15.979 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.510752+0000', 'last_change': '2023-11-24T08:12:09.510752+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.510752+0000', 'last_undegraded': '2023-11-24T08:12:09.510752+0000', 'last_fullsized': '2023-11-24T08:12:09.510752+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.504790+0000', 'last_change': '2023-11-24T08:12:09.504790+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.504790+0000', 'last_undegraded': '2023-11-24T08:12:09.504790+0000', 'last_fullsized': '2023-11-24T08:12:09.504790+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.523784+0000', 'last_change': '2023-11-24T08:12:09.523784+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.523784+0000', 'last_undegraded': '2023-11-24T08:12:09.523784+0000', 'last_fullsized': '2023-11-24T08:12:09.523784+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.505924+0000', 'last_change': '2023-11-24T08:12:09.505924+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.505924+0000', 'last_undegraded': '2023-11-24T08:12:09.505924+0000', 'last_fullsized': '2023-11-24T08:12:09.505924+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.509492+0000', 'last_change': '2023-11-24T08:12:09.509492+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.509492+0000', 'last_undegraded': '2023-11-24T08:12:09.509492+0000', 'last_fullsized': '2023-11-24T08:12:09.509492+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.511207+0000', 'last_change': '2023-11-24T08:12:09.511207+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.511207+0000', 'last_undegraded': '2023-11-24T08:12:09.511207+0000', 'last_fullsized': '2023-11-24T08:12:09.511207+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.507471+0000', 'last_change': '2023-11-24T08:12:09.507471+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.507471+0000', 'last_undegraded': '2023-11-24T08:12:09.507471+0000', 'last_fullsized': '2023-11-24T08:12:09.507471+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.508506+0000', 'last_change': '2023-11-24T08:12:09.508506+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.508506+0000', 'last_undegraded': '2023-11-24T08:12:09.508506+0000', 'last_fullsized': '2023-11-24T08:12:09.508506+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.519744+0000', 'last_change': '2023-11-24T08:12:09.519744+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.519744+0000', 'last_undegraded': '2023-11-24T08:12:09.519744+0000', 'last_fullsized': '2023-11-24T08:12:09.519744+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 74, 'state': 'creating+activating', 'last_fresh': '2023-11-24T08:12:10.533267+0000', 'last_change': '2023-11-24T08:12:10.533267+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:10.533267+0000', 'last_undegraded': '2023-11-24T08:12:10.533267+0000', 'last_fullsized': '2023-11-24T08:12:10.533267+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-25T19:05:10.427272+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': ['5', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-24T08:12:15.981 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.522771+0000', 'last_change': '2023-11-24T08:12:09.522771+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.522771+0000', 'last_undegraded': '2023-11-24T08:12:09.522771+0000', 'last_fullsized': '2023-11-24T08:12:09.522771+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.520763+0000', 'last_change': '2023-11-24T08:12:09.520763+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.520763+0000', 'last_undegraded': '2023-11-24T08:12:09.520763+0000', 'last_fullsized': '2023-11-24T08:12:09.520763+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.510858+0000', 'last_change': '2023-11-24T08:12:09.510858+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.510858+0000', 'last_undegraded': '2023-11-24T08:12:09.510858+0000', 'last_fullsized': '2023-11-24T08:12:09.510858+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.516090+0000', 'last_change': '2023-11-24T08:12:09.516090+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.516090+0000', 'last_undegraded': '2023-11-24T08:12:09.516090+0000', 'last_fullsized': '2023-11-24T08:12:09.516090+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-24T08:12:15.982 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.506002+0000', 'last_change': '2023-11-24T08:12:09.506002+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.506002+0000', 'last_undegraded': '2023-11-24T08:12:09.506002+0000', 'last_fullsized': '2023-11-24T08:12:09.506002+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-24T08:12:15.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-24T08:12:15.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.511825+0000', 'last_change': '2023-11-24T08:12:09.511825+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.511825+0000', 'last_undegraded': '2023-11-24T08:12:09.511825+0000', 'last_fullsized': '2023-11-24T08:12:09.511825+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-24T08:12:15.983 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-24T08:12:15.983 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 73, 'state': 'creating+peering', 'last_fresh': '2023-11-24T08:12:09.508831+0000', 'last_change': '2023-11-24T08:12:09.508831+0000', 'last_active': '2023-11-24T08:12:09.487651+0000', 'last_peered': '2023-11-24T08:12:09.487651+0000', 'last_clean': '2023-11-24T08:12:09.487651+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-24T08:12:09.508831+0000', 'last_undegraded': '2023-11-24T08:12:09.508831+0000', 'last_fullsized': '2023-11-24T08:12:09.508831+0000', 'mapping_epoch': 73, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 73, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'last_clean_scrub_stamp': '2023-11-24T08:12:09.487651+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-24T08:12:15.983 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-24T08:12:16.171 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:12:16.178 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:12:16.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.216+0000 7f1ce3f3c640 1 Processor -- start 2023-11-24T08:12:16.218 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.216+0000 7f1ce3f3c640 1 -- start start 2023-11-24T08:12:16.219 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce3f3c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1cdc0d31a0 0x7f1cdc0d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce3f3c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0ceb40 con 0x7f1cdc1609e0 2023-11-24T08:12:16.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce3f3c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cdc0d1070 con 0x7f1cdc0ce010 2023-11-24T08:12:16.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce3f3c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cdc0d3b50 con 0x7f1cdc0d31a0 2023-11-24T08:12:16.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce1cd8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1cdc1609e0 0x7f1cdc06bcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51860/0 (socket says 172.21.15.106:51860) 2023-11-24T08:12:16.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce1cd8640 1 -- 172.21.15.106:0/2172315215 learned_addr learned my addr 172.21.15.106:0/2172315215 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:16.220 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.218+0000 7f1ce14d7640 1 --2- 172.21.15.106:0/2172315215 >> v2:172.21.15.156:3300/0 conn(0x7f1cdc0d31a0 0x7f1cdc0d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.219+0000 7f1ce0cd6640 1 -- 172.21.15.106:0/2172315215 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390139355 0 0) 0x7f1cdc0ceb40 con 0x7f1cdc1609e0 2023-11-24T08:12:16.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.219+0000 7f1ce0cd6640 1 -- 172.21.15.106:0/2172315215 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc8003520 con 0x7f1cdc1609e0 2023-11-24T08:12:16.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.219+0000 7f1ce14d7640 1 -- 172.21.15.106:0/2172315215 >> v1:172.21.15.156:6789/0 conn(0x7f1cdc0ce010 legacy=0x7f1cdc0ce430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.219+0000 7f1ce14d7640 1 -- 172.21.15.106:0/2172315215 >> v1:172.21.15.106:6789/0 conn(0x7f1cdc1609e0 legacy=0x7f1cdc06bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.219+0000 7f1ce14d7640 1 -- 172.21.15.106:0/2172315215 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cdc0cedc0 con 0x7f1cdc0d31a0 2023-11-24T08:12:16.221 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.220+0000 7f1ce14d7640 1 --2- 172.21.15.106:0/2172315215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0d31a0 0x7f1cdc0d3610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0097a0 tx=0x7f1ccc02b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdf5fb4ec8b6e1e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:16.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.220+0000 7f1ce0cd6640 1 -- 172.21.15.106:0/2172315215 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc030610 con 0x7f1cdc0d31a0 2023-11-24T08:12:16.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.220+0000 7f1ce0cd6640 1 -- 172.21.15.106:0/2172315215 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ccc030770 con 0x7f1cdc0d31a0 2023-11-24T08:12:16.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce0cd6640 1 -- 172.21.15.106:0/2172315215 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ccc030b00 con 0x7f1cdc0d31a0 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/2172315215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0d31a0 msgr2=0x7f1cdc0d3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/2172315215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0d31a0 0x7f1cdc0d3610 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc0097a0 tx=0x7f1ccc02b960 comp rx=0 tx=0).stop 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/2172315215 shutdown_connections 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/2172315215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0d31a0 0x7f1cdc0d3610 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/2172315215 >> 172.21.15.106:0/2172315215 conn(0x7f1cdc08af60 msgr2=0x7f1cdc09c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/2172315215 shutdown_connections 2023-11-24T08:12:16.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.221+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/2172315215 wait complete. 2023-11-24T08:12:16.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.223+0000 7f1ce3f3c640 1 Processor -- start 2023-11-24T08:12:16.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.223+0000 7f1ce3f3c640 1 -- start start 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce3f3c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 0x7f1cdc157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce3f3c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cdc0d31a0 0x7f1cdc0b5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce3f3c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 0x7f1cdc0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce3f3c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b6a70 con 0x7f1cdc1609e0 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce3f3c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b6d50 con 0x7f1cdc0ce010 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce3f3c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cdc0b7030 con 0x7f1cdc0d31a0 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce24d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 0x7f1cdc0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.224+0000 7f1ce1cd8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 0x7f1cdc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce14d7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cdc0d31a0 0x7f1cdc0b5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 0x7f1cdc157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:53646/0 (socket says 172.21.15.106:53646) 2023-11-24T08:12:16.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce24d9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 0x7f1cdc0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43650/0 (socket says 172.21.15.106:43650) 2023-11-24T08:12:16.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 -- 172.21.15.106:0/1395887186 learned_addr learned my addr 172.21.15.106:0/1395887186 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 -- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cdc0d31a0 msgr2=0x7f1cdc0b5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cdc0d31a0 0x7f1cdc0b5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 -- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 msgr2=0x7f1cdc0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 0x7f1cdc0c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 -- 172.21.15.106:0/1395887186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ccc009400 con 0x7f1cdc0ce010 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce24d9640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 0x7f1cdc0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.225+0000 7f1ce1cd8640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 0x7f1cdc157370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c500 tx=0x7f1cd800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.226+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd8005ce0 con 0x7f1cdc0ce010 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.226+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cdc0b7310 con 0x7f1cdc0ce010 2023-11-24T08:12:16.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.226+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cd8005e40 con 0x7f1cdc0ce010 2023-11-24T08:12:16.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.226+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cd80072f0 con 0x7f1cdc0ce010 2023-11-24T08:12:16.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.226+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cdc0b5ad0 con 0x7f1cdc0ce010 2023-11-24T08:12:16.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.227+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cdc067470 con 0x7f1cdc0ce010 2023-11-24T08:12:16.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.229+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1cd8007510 con 0x7f1cdc0ce010 2023-11-24T08:12:16.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.230+0000 7f1cd27fc640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1cc006b260 0x7f1cc006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.230+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cd8089290 con 0x7f1cdc0ce010 2023-11-24T08:12:16.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.230+0000 7f1ce14d7640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1cc006b260 0x7f1cc006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.231+0000 7f1ce14d7640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1cc006b260 0x7f1cc006d720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc036000 tx=0x7f1ccc008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:16.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.238+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1cd8053f60 con 0x7f1cdc0ce010 2023-11-24T08:12:16.470 INFO:tasks.ceph.mds.c.smithi106.stderr:2023-11-24T08:12:16.469+0000 7fa4a589c640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:12:16.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.503+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1cdc0b6e90 con 0x7f1cdc0ce010 2023-11-24T08:12:16.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.504+0000 7f1cd27fc640 1 -- 172.21.15.106:0/1395887186 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1499 (secure 0 0 0) 0x7f1cdc0b6e90 con 0x7f1cdc0ce010 2023-11-24T08:12:16.506 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:16.506 INFO:teuthology.orchestra.run.smithi106.stdout:{"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":228,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":75,"num_osds":8,"num_up_osds":8,"osd_up_since":1700813319,"num_in_osds":8,"osd_in_since":1700813311,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":56},{"state_name":"active+clean","count":35},{"state_name":"creating+activating","count":6}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":255164416,"bytes_avail":772838948864,"bytes_total":773094113280,"inactive_pgs_ratio":0.63917523622512817},"fsmap":{"epoch":43,"id":5,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":5,"rank":0,"name":"c","status":"up:active","gid":6516}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-11-24T08:11:57.331338+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-24T08:12:16.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.506+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1cc006b260 msgr2=0x7f1cc006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.506+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1cc006b260 0x7f1cc006d720 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc036000 tx=0x7f1ccc008030 comp rx=0 tx=0).stop 2023-11-24T08:12:16.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.506+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 msgr2=0x7f1cdc157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.506+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 0x7f1cdc157370 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800c500 tx=0x7f1cd800c9c0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 shutdown_connections 2023-11-24T08:12:16.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1cdc1609e0 0x7f1cdc0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1cc006b260 0x7f1cc006d720 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1cdc0d31a0 0x7f1cdc0b5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 --2- 172.21.15.106:0/1395887186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1cdc0ce010 0x7f1cdc157370 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 >> 172.21.15.106:0/1395887186 conn(0x7f1cdc08af60 msgr2=0x7f1cdc0b1fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:16.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 shutdown_connections 2023-11-24T08:12:16.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.507+0000 7f1ce3f3c640 1 -- 172.21.15.106:0/1395887186 wait complete. 2023-11-24T08:12:16.527 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-24T08:12:16.528 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:12:16.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.733+0000 7f33a0c59640 1 Processor -- start 2023-11-24T08:12:16.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.733+0000 7f33a0c59640 1 -- start start 2023-11-24T08:12:16.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.734+0000 7f33a0c59640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f339c159e80 0x7f339c071610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.734+0000 7f33a0c59640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c0cd6a0 0x7f339c15a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.734+0000 7f33a0c59640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339c0ce870 con 0x7f339c15a7a0 2023-11-24T08:12:16.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.734+0000 7f33a0c59640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339c0cdc60 con 0x7f339c159e80 2023-11-24T08:12:16.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.735+0000 7f33a0c59640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0cdf40 con 0x7f339c0cd6a0 2023-11-24T08:12:16.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.735+0000 7f339b7fe640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f339c159e80 0x7f339c071610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.735+0000 7f3393fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c0cd6a0 0x7f339c15a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.735+0000 7f339b7fe640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f339c159e80 0x7f339c071610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:53656/0 (socket says 172.21.15.106:53656) 2023-11-24T08:12:16.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.735+0000 7f339b7fe640 1 -- 172.21.15.106:0/3748262637 learned_addr learned my addr 172.21.15.106:0/3748262637 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:16.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.735+0000 7f339bfff640 1 --1- 172.21.15.106:0/3748262637 >> v1:172.21.15.156:6789/0 conn(0x7f339c15a7a0 0x7f339c06c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.156:6789/0 says I am v1:172.21.15.106:40838/0 (socket says 172.21.15.106:40838) 2023-11-24T08:12:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.736+0000 7f339b7fe640 1 -- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c0cd6a0 msgr2=0x7f339c15a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.736+0000 7f339b7fe640 1 --2- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c0cd6a0 0x7f339c15a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.736+0000 7f339b7fe640 1 -- 172.21.15.106:0/3748262637 >> v1:172.21.15.156:6789/0 conn(0x7f339c15a7a0 legacy=0x7f339c06c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.736+0000 7f339b7fe640 1 -- 172.21.15.106:0/3748262637 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339c0ce220 con 0x7f339c159e80 2023-11-24T08:12:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.737+0000 7f339b7fe640 1 --2- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c159e80 0x7f339c071610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f338c009870 tx=0x7f338c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c374cbed44340fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:16.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.738+0000 7f339affd640 1 -- 172.21.15.106:0/3748262637 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c031640 con 0x7f339c159e80 2023-11-24T08:12:16.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.738+0000 7f339affd640 1 -- 172.21.15.106:0/3748262637 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c0317a0 con 0x7f339c159e80 2023-11-24T08:12:16.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.738+0000 7f339affd640 1 -- 172.21.15.106:0/3748262637 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c031b30 con 0x7f339c159e80 2023-11-24T08:12:16.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.738+0000 7f33a0c59640 1 -- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c159e80 msgr2=0x7f339c071610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.738+0000 7f33a0c59640 1 --2- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c159e80 0x7f339c071610 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f338c009870 tx=0x7f338c02b8f0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.739+0000 7f33a0c59640 1 -- 172.21.15.106:0/3748262637 shutdown_connections 2023-11-24T08:12:16.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.739+0000 7f33a0c59640 1 --2- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c0cd6a0 0x7f339c15a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.739+0000 7f33a0c59640 1 --2- 172.21.15.106:0/3748262637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c159e80 0x7f339c071610 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.739+0000 7f33a0c59640 1 -- 172.21.15.106:0/3748262637 >> 172.21.15.106:0/3748262637 conn(0x7f339c0cb830 msgr2=0x7f339c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.739+0000 7f33a0c59640 1 -- 172.21.15.106:0/3748262637 shutdown_connections 2023-11-24T08:12:16.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.739+0000 7f33a0c59640 1 -- 172.21.15.106:0/3748262637 wait complete. 2023-11-24T08:12:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.741+0000 7f33a0c59640 1 Processor -- start 2023-11-24T08:12:16.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.741+0000 7f33a0c59640 1 -- start start 2023-11-24T08:12:16.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f33a0c59640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c0cd6a0 0x7f339c1e6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f33a0c59640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c159e80 0x7f339c1e2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f33a0c59640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 0x7f339c1f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f33a0c59640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c1e60b0 con 0x7f339c15a7a0 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f33a0c59640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f339c0adfe0 con 0x7f339c0cd6a0 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f33a0c59640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f339c1e51c0 con 0x7f339c159e80 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 0x7f339c1f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 0x7f339c1f3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43664/0 (socket says 172.21.15.106:43664) 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 -- 172.21.15.106:0/4227016489 learned_addr learned my addr 172.21.15.106:0/4227016489 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339b7fe640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c0cd6a0 0x7f339c1e6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f3393fff640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c159e80 0x7f339c1e2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 -- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c159e80 msgr2=0x7f339c1e2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c159e80 0x7f339c1e2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 -- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c0cd6a0 msgr2=0x7f339c1e6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c0cd6a0 0x7f339c1e6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.742+0000 7f339bfff640 1 -- 172.21.15.106:0/4227016489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c0094d0 con 0x7f339c15a7a0 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f339b7fe640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c0cd6a0 0x7f339c1e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f339bfff640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 0x7f339c1f3060 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f339400c7b0 tx=0x7f339400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339400a640 con 0x7f339c15a7a0 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f339400a7a0 con 0x7f339c15a7a0 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f339c1e54a0 con 0x7f339c15a7a0 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339400ab10 con 0x7f339c15a7a0 2023-11-24T08:12:16.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.743+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f339c1e5a60 con 0x7f339c15a7a0 2023-11-24T08:12:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.744+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c1e6260 con 0x7f339c15a7a0 2023-11-24T08:12:16.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.745+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f339400ac70 con 0x7f339c15a7a0 2023-11-24T08:12:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.746+0000 7f3398ff9640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f337c06b230 0x7f337c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.746+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3394089480 con 0x7f339c15a7a0 2023-11-24T08:12:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.746+0000 7f339b7fe640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f337c06b230 0x7f337c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:16.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.747+0000 7f339b7fe640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f337c06b230 0x7f337c06d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f338c036000 tx=0x7f338c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:16.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.750+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f339c1e6260 con 0x7f339c15a7a0 2023-11-24T08:12:16.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.921+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f339c15abc0 con 0x7f337c06b230 2023-11-24T08:12:16.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.933+0000 7f3398ff9640 1 -- 172.21.15.106:0/4227016489 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282238 (secure 0 0 0) 0x7f338c03a420 con 0x7f337c06b230 2023-11-24T08:12:16.938 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:16.939 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:12:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.937+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f337c06b230 msgr2=0x7f337c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f337c06b230 0x7f337c06d6f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f338c036000 tx=0x7f338c03c040 comp rx=0 tx=0).stop 2023-11-24T08:12:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 msgr2=0x7f339c1f3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 0x7f339c1f3060 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f339400c7b0 tx=0x7f339400cc70 comp rx=0 tx=0).stop 2023-11-24T08:12:16.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 shutdown_connections 2023-11-24T08:12:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f339c15a7a0 0x7f339c1f3060 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f337c06b230 0x7f337c06d6f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f339c159e80 0x7f339c1e2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 --2- 172.21.15.106:0/4227016489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f339c0cd6a0 0x7f339c1e6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:16.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 >> 172.21.15.106:0/4227016489 conn(0x7f339c0cb830 msgr2=0x7f339c0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:16.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 shutdown_connections 2023-11-24T08:12:16.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:16.938+0000 7f33a0c59640 1 -- 172.21.15.106:0/4227016489 wait complete. 2023-11-24T08:12:16.958 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2023-11-24T08:12:15.336671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":249184,"kb_used_data":8060,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754725536,"statfs":{"total":773094113280,"available":772838948864,"internally_reserved":0,"allocated":8253440,"data_stored":4150285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":246795824},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001906"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524784+0000","last_change":"2023-11-24T08:12:09.524784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524784+0000","last_undegraded":"2023-11-24T08:12:09.524784+0000","last_fullsized":"2023-11-24T08:12:09.524784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517595+0000","last_change":"2023-11-24T08:12:09.517595+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517595+0000","last_undegraded":"2023-11-24T08:12:09.517595+0000","last_fullsized":"2023-11-24T08:12:09.517595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508940+0000","last_change":"2023-11-24T08:12:09.508940+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508940+0000","last_undegraded":"2023-11-24T08:12:09.508940+0000","last_fullsized":"2023-11-24T08:12:09.508940+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508376+0000","last_change":"2023-11-24T08:12:09.508376+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508376+0000","last_undegraded":"2023-11-24T08:12:09.508376+0000","last_fullsized":"2023-11-24T08:12:09.508376+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511217+0000","last_change":"2023-11-24T08:12:09.511217+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511217+0000","last_undegraded":"2023-11-24T08:12:09.511217+0000","last_fullsized":"2023-11-24T08:12:09.511217+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.514317+0000","last_change":"2023-11-24T08:12:09.514317+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.514317+0000","last_undegraded":"2023-11-24T08:12:09.514317+0000","last_fullsized":"2023-11-24T08:12:09.514317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.532518+0000","last_change":"2023-11-24T08:12:10.532518+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.532518+0000","last_undegraded":"2023-11-24T08:12:10.532518+0000","last_fullsized":"2023-11-24T08:12:10.532518+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:29:09.759327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511431+0000","last_change":"2023-11-24T08:12:09.511431+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511431+0000","last_undegraded":"2023-11-24T08:12:09.511431+0000","last_fullsized":"2023-11-24T08:12:09.511431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510124+0000","last_change":"2023-11-24T08:12:09.510124+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510124+0000","last_undegraded":"2023-11-24T08:12:09.510124+0000","last_fullsized":"2023-11-24T08:12:09.510124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505932+0000","last_change":"2023-11-24T08:12:09.505932+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505932+0000","last_undegraded":"2023-11-24T08:12:09.505932+0000","last_fullsized":"2023-11-24T08:12:09.505932+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523117+0000","last_change":"2023-11-24T08:12:09.523117+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523117+0000","last_undegraded":"2023-11-24T08:12:09.523117+0000","last_fullsized":"2023-11-24T08:12:09.523117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507941+0000","last_change":"2023-11-24T08:12:09.507941+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507941+0000","last_undegraded":"2023-11-24T08:12:09.507941+0000","last_fullsized":"2023-11-24T08:12:09.507941+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516971+0000","last_change":"2023-11-24T08:12:09.516971+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516971+0000","last_undegraded":"2023-11-24T08:12:09.516971+0000","last_fullsized":"2023-11-24T08:12:09.516971+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511315+0000","last_change":"2023-11-24T08:12:09.511315+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511315+0000","last_undegraded":"2023-11-24T08:12:09.511315+0000","last_fullsized":"2023-11-24T08:12:09.511315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519786+0000","last_change":"2023-11-24T08:12:09.519786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519786+0000","last_undegraded":"2023-11-24T08:12:09.519786+0000","last_fullsized":"2023-11-24T08:12:09.519786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503356+0000","last_change":"2023-11-24T08:12:09.503356+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503356+0000","last_undegraded":"2023-11-24T08:12:09.503356+0000","last_fullsized":"2023-11-24T08:12:09.503356+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509693+0000","last_change":"2023-11-24T08:12:09.509693+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509693+0000","last_undegraded":"2023-11-24T08:12:09.509693+0000","last_fullsized":"2023-11-24T08:12:09.509693+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506814+0000","last_change":"2023-11-24T08:12:09.506814+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506814+0000","last_undegraded":"2023-11-24T08:12:09.506814+0000","last_fullsized":"2023-11-24T08:12:09.506814+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524401+0000","last_change":"2023-11-24T08:12:09.524401+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524401+0000","last_undegraded":"2023-11-24T08:12:09.524401+0000","last_fullsized":"2023-11-24T08:12:09.524401+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516989+0000","last_change":"2023-11-24T08:12:09.516989+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516989+0000","last_undegraded":"2023-11-24T08:12:09.516989+0000","last_fullsized":"2023-11-24T08:12:09.516989+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506189+0000","last_change":"2023-11-24T08:12:09.506189+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506189+0000","last_undegraded":"2023-11-24T08:12:09.506189+0000","last_fullsized":"2023-11-24T08:12:09.506189+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509465+0000","last_change":"2023-11-24T08:12:09.509465+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509465+0000","last_undegraded":"2023-11-24T08:12:09.509465+0000","last_fullsized":"2023-11-24T08:12:09.509465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520990+0000","last_change":"2023-11-24T08:12:09.520990+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520990+0000","last_undegraded":"2023-11-24T08:12:09.520990+0000","last_fullsized":"2023-11-24T08:12:09.520990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524248+0000","last_change":"2023-11-24T08:12:09.524248+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524248+0000","last_undegraded":"2023-11-24T08:12:09.524248+0000","last_fullsized":"2023-11-24T08:12:09.524248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510034+0000","last_change":"2023-11-24T08:12:09.510034+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510034+0000","last_undegraded":"2023-11-24T08:12:09.510034+0000","last_fullsized":"2023-11-24T08:12:09.510034+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.531598+0000","last_change":"2023-11-24T08:12:10.531598+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.531598+0000","last_undegraded":"2023-11-24T08:12:10.531598+0000","last_fullsized":"2023-11-24T08:12:10.531598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:24:31.654975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508296+0000","last_change":"2023-11-24T08:12:09.508296+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508296+0000","last_undegraded":"2023-11-24T08:12:09.508296+0000","last_fullsized":"2023-11-24T08:12:09.508296+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503786+0000","last_change":"2023-11-24T08:12:09.503786+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503786+0000","last_undegraded":"2023-11-24T08:12:09.503786+0000","last_fullsized":"2023-11-24T08:12:09.503786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511659+0000","last_change":"2023-11-24T08:12:09.511659+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511659+0000","last_undegraded":"2023-11-24T08:12:09.511659+0000","last_fullsized":"2023-11-24T08:12:09.511659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.544012+0000","last_change":"2023-11-24T08:12:10.544012+0000","last_active":"2023-11-24T08:12:10.544012+0000","last_peered":"2023-11-24T08:12:10.544012+0000","last_clean":"2023-11-24T08:12:10.544012+0000","last_became_active":"2023-11-24T08:12:10.543658+0000","last_became_peered":"2023-11-24T08:12:10.543658+0000","last_unstale":"2023-11-24T08:12:10.544012+0000","last_undegraded":"2023-11-24T08:12:10.544012+0000","last_fullsized":"2023-11-24T08:12:10.544012+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:55:34.774986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.524767+0000","last_change":"2023-11-24T08:12:10.524767+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.524767+0000","last_undegraded":"2023-11-24T08:12:10.524767+0000","last_fullsized":"2023-11-24T08:12:10.524767+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:21:52.679838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":["5","2"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507818+0000","last_change":"2023-11-24T08:12:09.507818+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507818+0000","last_undegraded":"2023-11-24T08:12:09.507818+0000","last_fullsized":"2023-11-24T08:12:09.507818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520772+0000","last_change":"2023-11-24T08:12:09.520772+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520772+0000","last_undegraded":"2023-11-24T08:12:09.520772+0000","last_fullsized":"2023-11-24T08:12:09.520772+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537413+0000","last_change":"2023-11-24T08:12:09.535765+0000","last_active":"2023-11-24T08:12:09.537413+0000","last_peered":"2023-11-24T08:12:09.537413+0000","last_clean":"2023-11-24T08:12:09.537413+0000","last_became_active":"2023-11-24T08:12:09.535385+0000","last_became_peered":"2023-11-24T08:12:09.535385+0000","last_unstale":"2023-11-24T08:12:09.537413+0000","last_undegraded":"2023-11-24T08:12:09.537413+0000","last_fullsized":"2023-11-24T08:12:09.537413+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:18.985711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509116+0000","last_change":"2023-11-24T08:12:09.509116+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509116+0000","last_undegraded":"2023-11-24T08:12:09.509116+0000","last_fullsized":"2023-11-24T08:12:09.509116+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530987+0000","last_change":"2023-11-24T08:12:09.524544+0000","last_active":"2023-11-24T08:12:09.530987+0000","last_peered":"2023-11-24T08:12:09.530987+0000","last_clean":"2023-11-24T08:12:09.530987+0000","last_became_active":"2023-11-24T08:12:09.524188+0000","last_became_peered":"2023-11-24T08:12:09.524188+0000","last_unstale":"2023-11-24T08:12:09.530987+0000","last_undegraded":"2023-11-24T08:12:09.530987+0000","last_fullsized":"2023-11-24T08:12:09.530987+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:56.808696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524895+0000","last_change":"2023-11-24T08:12:09.524895+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524895+0000","last_undegraded":"2023-11-24T08:12:09.524895+0000","last_fullsized":"2023-11-24T08:12:09.524895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538199+0000","last_change":"2023-11-24T08:12:09.533833+0000","last_active":"2023-11-24T08:12:09.538199+0000","last_peered":"2023-11-24T08:12:09.538199+0000","last_clean":"2023-11-24T08:12:09.538199+0000","last_became_active":"2023-11-24T08:12:09.533263+0000","last_became_peered":"2023-11-24T08:12:09.533263+0000","last_unstale":"2023-11-24T08:12:09.538199+0000","last_undegraded":"2023-11-24T08:12:09.538199+0000","last_fullsized":"2023-11-24T08:12:09.538199+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:54:50.328328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522446+0000","last_change":"2023-11-24T08:12:09.522446+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522446+0000","last_undegraded":"2023-11-24T08:12:09.522446+0000","last_fullsized":"2023-11-24T08:12:09.522446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538124+0000","last_change":"2023-11-24T08:12:09.529735+0000","last_active":"2023-11-24T08:12:09.538124+0000","last_peered":"2023-11-24T08:12:09.538124+0000","last_clean":"2023-11-24T08:12:09.538124+0000","last_became_active":"2023-11-24T08:12:09.529517+0000","last_became_peered":"2023-11-24T08:12:09.529517+0000","last_unstale":"2023-11-24T08:12:09.538124+0000","last_undegraded":"2023-11-24T08:12:09.538124+0000","last_fullsized":"2023-11-24T08:12:09.538124+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:14.043315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.525674+0000","last_change":"2023-11-24T08:12:10.525674+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.525674+0000","last_undegraded":"2023-11-24T08:12:10.525674+0000","last_fullsized":"2023-11-24T08:12:10.525674+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:43:27.094673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526557+0000","last_change":"2023-11-24T08:12:09.522576+0000","last_active":"2023-11-24T08:12:09.526557+0000","last_peered":"2023-11-24T08:12:09.526557+0000","last_clean":"2023-11-24T08:12:09.526557+0000","last_became_active":"2023-11-24T08:12:09.522114+0000","last_became_peered":"2023-11-24T08:12:09.522114+0000","last_unstale":"2023-11-24T08:12:09.526557+0000","last_undegraded":"2023-11-24T08:12:09.526557+0000","last_fullsized":"2023-11-24T08:12:09.526557+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:27:40.788887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.503392+0000","last_change":"2023-11-24T08:12:09.503392+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.503392+0000","last_undegraded":"2023-11-24T08:12:09.503392+0000","last_fullsized":"2023-11-24T08:12:09.503392+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537905+0000","last_change":"2023-11-24T08:12:09.531869+0000","last_active":"2023-11-24T08:12:09.537905+0000","last_peered":"2023-11-24T08:12:09.537905+0000","last_clean":"2023-11-24T08:12:09.537905+0000","last_became_active":"2023-11-24T08:12:09.531598+0000","last_became_peered":"2023-11-24T08:12:09.531598+0000","last_unstale":"2023-11-24T08:12:09.537905+0000","last_undegraded":"2023-11-24T08:12:09.537905+0000","last_fullsized":"2023-11-24T08:12:09.537905+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:24.358046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510090+0000","last_change":"2023-11-24T08:12:09.510090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510090+0000","last_undegraded":"2023-11-24T08:12:09.510090+0000","last_fullsized":"2023-11-24T08:12:09.510090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518079+0000","last_change":"2023-11-24T08:12:09.513949+0000","last_active":"2023-11-24T08:12:09.518079+0000","last_peered":"2023-11-24T08:12:09.518079+0000","last_clean":"2023-11-24T08:12:09.518079+0000","last_became_active":"2023-11-24T08:12:09.513517+0000","last_became_peered":"2023-11-24T08:12:09.513517+0000","last_unstale":"2023-11-24T08:12:09.518079+0000","last_undegraded":"2023-11-24T08:12:09.518079+0000","last_fullsized":"2023-11-24T08:12:09.518079+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:51.436372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517039+0000","last_change":"2023-11-24T08:12:09.517039+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517039+0000","last_undegraded":"2023-11-24T08:12:09.517039+0000","last_fullsized":"2023-11-24T08:12:09.517039+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.524795+0000","last_change":"2023-11-24T08:12:09.519208+0000","last_active":"2023-11-24T08:12:09.524795+0000","last_peered":"2023-11-24T08:12:09.524795+0000","last_clean":"2023-11-24T08:12:09.524795+0000","last_became_active":"2023-11-24T08:12:09.518952+0000","last_became_peered":"2023-11-24T08:12:09.518952+0000","last_unstale":"2023-11-24T08:12:09.524795+0000","last_undegraded":"2023-11-24T08:12:09.524795+0000","last_fullsized":"2023-11-24T08:12:09.524795+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:57:11.824082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":8,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.542676+0000","last_change":"2023-11-24T08:12:10.542676+0000","last_active":"2023-11-24T08:12:10.542676+0000","last_peered":"2023-11-24T08:12:10.542676+0000","last_clean":"2023-11-24T08:12:10.542676+0000","last_became_active":"2023-11-24T08:12:10.542269+0000","last_became_peered":"2023-11-24T08:12:10.542269+0000","last_unstale":"2023-11-24T08:12:10.542676+0000","last_undegraded":"2023-11-24T08:12:10.542676+0000","last_fullsized":"2023-11-24T08:12:10.542676+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:58:38.761881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.525878+0000","last_change":"2023-11-24T08:12:09.519666+0000","last_active":"2023-11-24T08:12:09.525878+0000","last_peered":"2023-11-24T08:12:09.525878+0000","last_clean":"2023-11-24T08:12:09.525878+0000","last_became_active":"2023-11-24T08:12:09.519348+0000","last_became_peered":"2023-11-24T08:12:09.519348+0000","last_unstale":"2023-11-24T08:12:09.525878+0000","last_undegraded":"2023-11-24T08:12:09.525878+0000","last_fullsized":"2023-11-24T08:12:09.525878+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:20:08.548384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.524304+0000","last_change":"2023-11-24T08:12:09.524304+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.524304+0000","last_undegraded":"2023-11-24T08:12:09.524304+0000","last_fullsized":"2023-11-24T08:12:09.524304+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538991+0000","last_change":"2023-11-24T08:12:09.537826+0000","last_active":"2023-11-24T08:12:09.538991+0000","last_peered":"2023-11-24T08:12:09.538991+0000","last_clean":"2023-11-24T08:12:09.538991+0000","last_became_active":"2023-11-24T08:12:09.537555+0000","last_became_peered":"2023-11-24T08:12:09.537555+0000","last_unstale":"2023-11-24T08:12:09.538991+0000","last_undegraded":"2023-11-24T08:12:09.538991+0000","last_fullsized":"2023-11-24T08:12:09.538991+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:54:04.166572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.515379+0000","last_change":"2023-11-24T08:12:09.515379+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.515379+0000","last_undegraded":"2023-11-24T08:12:09.515379+0000","last_fullsized":"2023-11-24T08:12:09.515379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531034+0000","last_change":"2023-11-24T08:12:09.524384+0000","last_active":"2023-11-24T08:12:09.531034+0000","last_peered":"2023-11-24T08:12:09.531034+0000","last_clean":"2023-11-24T08:12:09.531034+0000","last_became_active":"2023-11-24T08:12:09.524038+0000","last_became_peered":"2023-11-24T08:12:09.524038+0000","last_unstale":"2023-11-24T08:12:09.531034+0000","last_undegraded":"2023-11-24T08:12:09.531034+0000","last_fullsized":"2023-11-24T08:12:09.531034+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:45.336925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.513394+0000","last_change":"2023-11-24T08:12:09.513394+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.513394+0000","last_undegraded":"2023-11-24T08:12:09.513394+0000","last_fullsized":"2023-11-24T08:12:09.513394+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.518054+0000","last_change":"2023-11-24T08:12:09.514045+0000","last_active":"2023-11-24T08:12:09.518054+0000","last_peered":"2023-11-24T08:12:09.518054+0000","last_clean":"2023-11-24T08:12:09.518054+0000","last_became_active":"2023-11-24T08:12:09.513337+0000","last_became_peered":"2023-11-24T08:12:09.513337+0000","last_unstale":"2023-11-24T08:12:09.518054+0000","last_undegraded":"2023-11-24T08:12:09.518054+0000","last_fullsized":"2023-11-24T08:12:09.518054+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:17.346075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.518834+0000","last_change":"2023-11-24T08:12:09.518834+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.518834+0000","last_undegraded":"2023-11-24T08:12:09.518834+0000","last_fullsized":"2023-11-24T08:12:09.518834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.530522+0000","last_change":"2023-11-24T08:12:09.528456+0000","last_active":"2023-11-24T08:12:09.530522+0000","last_peered":"2023-11-24T08:12:09.530522+0000","last_clean":"2023-11-24T08:12:09.530522+0000","last_became_active":"2023-11-24T08:12:09.528140+0000","last_became_peered":"2023-11-24T08:12:09.528140+0000","last_unstale":"2023-11-24T08:12:09.530522+0000","last_undegraded":"2023-11-24T08:12:09.530522+0000","last_fullsized":"2023-11-24T08:12:09.530522+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:12.633925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.529872+0000","last_change":"2023-11-24T08:12:10.529872+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.529872+0000","last_undegraded":"2023-11-24T08:12:10.529872+0000","last_fullsized":"2023-11-24T08:12:10.529872+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:59:09.099806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538979+0000","last_change":"2023-11-24T08:12:09.537450+0000","last_active":"2023-11-24T08:12:09.538979+0000","last_peered":"2023-11-24T08:12:09.538979+0000","last_clean":"2023-11-24T08:12:09.538979+0000","last_became_active":"2023-11-24T08:12:09.537196+0000","last_became_peered":"2023-11-24T08:12:09.537196+0000","last_unstale":"2023-11-24T08:12:09.538979+0000","last_undegraded":"2023-11-24T08:12:09.538979+0000","last_fullsized":"2023-11-24T08:12:09.538979+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:19.005358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.517657+0000","last_change":"2023-11-24T08:12:09.517657+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.517657+0000","last_undegraded":"2023-11-24T08:12:09.517657+0000","last_fullsized":"2023-11-24T08:12:09.517657+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533269+0000","last_change":"2023-11-24T08:12:09.530752+0000","last_active":"2023-11-24T08:12:10.533269+0000","last_peered":"2023-11-24T08:12:10.533269+0000","last_clean":"2023-11-24T08:12:10.533269+0000","last_became_active":"2023-11-24T08:12:09.529282+0000","last_became_peered":"2023-11-24T08:12:09.529282+0000","last_unstale":"2023-11-24T08:12:10.533269+0000","last_undegraded":"2023-11-24T08:12:10.533269+0000","last_fullsized":"2023-11-24T08:12:10.533269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:31.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510752+0000","last_change":"2023-11-24T08:12:09.510752+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510752+0000","last_undegraded":"2023-11-24T08:12:09.510752+0000","last_fullsized":"2023-11-24T08:12:09.510752+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.534559+0000","last_change":"2023-11-24T08:12:09.530860+0000","last_active":"2023-11-24T08:12:09.534559+0000","last_peered":"2023-11-24T08:12:09.534559+0000","last_clean":"2023-11-24T08:12:09.534559+0000","last_became_active":"2023-11-24T08:12:09.530167+0000","last_became_peered":"2023-11-24T08:12:09.530167+0000","last_unstale":"2023-11-24T08:12:09.534559+0000","last_undegraded":"2023-11-24T08:12:09.534559+0000","last_fullsized":"2023-11-24T08:12:09.534559+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:08.559029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.504790+0000","last_change":"2023-11-24T08:12:09.504790+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.504790+0000","last_undegraded":"2023-11-24T08:12:09.504790+0000","last_fullsized":"2023-11-24T08:12:09.504790+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.539017+0000","last_change":"2023-11-24T08:12:09.538063+0000","last_active":"2023-11-24T08:12:09.539017+0000","last_peered":"2023-11-24T08:12:09.539017+0000","last_clean":"2023-11-24T08:12:09.539017+0000","last_became_active":"2023-11-24T08:12:09.536932+0000","last_became_peered":"2023-11-24T08:12:09.536932+0000","last_unstale":"2023-11-24T08:12:09.539017+0000","last_undegraded":"2023-11-24T08:12:09.539017+0000","last_fullsized":"2023-11-24T08:12:09.539017+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:31.591358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.523784+0000","last_change":"2023-11-24T08:12:09.523784+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.523784+0000","last_undegraded":"2023-11-24T08:12:09.523784+0000","last_fullsized":"2023-11-24T08:12:09.523784+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538170+0000","last_change":"2023-11-24T08:12:09.533762+0000","last_active":"2023-11-24T08:12:09.538170+0000","last_peered":"2023-11-24T08:12:09.538170+0000","last_clean":"2023-11-24T08:12:09.538170+0000","last_became_active":"2023-11-24T08:12:09.533465+0000","last_became_peered":"2023-11-24T08:12:09.533465+0000","last_unstale":"2023-11-24T08:12:09.538170+0000","last_undegraded":"2023-11-24T08:12:09.538170+0000","last_fullsized":"2023-11-24T08:12:09.538170+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:44.590676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.505924+0000","last_change":"2023-11-24T08:12:09.505924+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.505924+0000","last_undegraded":"2023-11-24T08:12:09.505924+0000","last_fullsized":"2023-11-24T08:12:09.505924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537888+0000","last_change":"2023-11-24T08:12:09.530831+0000","last_active":"2023-11-24T08:12:09.537888+0000","last_peered":"2023-11-24T08:12:09.537888+0000","last_clean":"2023-11-24T08:12:09.537888+0000","last_became_active":"2023-11-24T08:12:09.530459+0000","last_became_peered":"2023-11-24T08:12:09.530459+0000","last_unstale":"2023-11-24T08:12:09.537888+0000","last_undegraded":"2023-11-24T08:12:09.537888+0000","last_fullsized":"2023-11-24T08:12:09.537888+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:40.045785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.509492+0000","last_change":"2023-11-24T08:12:09.509492+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.509492+0000","last_undegraded":"2023-11-24T08:12:09.509492+0000","last_fullsized":"2023-11-24T08:12:09.509492+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.537541+0000","last_change":"2023-11-24T08:12:09.535763+0000","last_active":"2023-11-24T08:12:09.537541+0000","last_peered":"2023-11-24T08:12:09.537541+0000","last_clean":"2023-11-24T08:12:09.537541+0000","last_became_active":"2023-11-24T08:12:09.535048+0000","last_became_peered":"2023-11-24T08:12:09.535048+0000","last_unstale":"2023-11-24T08:12:09.537541+0000","last_undegraded":"2023-11-24T08:12:09.537541+0000","last_fullsized":"2023-11-24T08:12:09.537541+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:42.819752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511207+0000","last_change":"2023-11-24T08:12:09.511207+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511207+0000","last_undegraded":"2023-11-24T08:12:09.511207+0000","last_fullsized":"2023-11-24T08:12:09.511207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538446+0000","last_change":"2023-11-24T08:12:09.536935+0000","last_active":"2023-11-24T08:12:09.538446+0000","last_peered":"2023-11-24T08:12:09.538446+0000","last_clean":"2023-11-24T08:12:09.538446+0000","last_became_active":"2023-11-24T08:12:09.535354+0000","last_became_peered":"2023-11-24T08:12:09.535354+0000","last_unstale":"2023-11-24T08:12:09.538446+0000","last_undegraded":"2023-11-24T08:12:09.538446+0000","last_fullsized":"2023-11-24T08:12:09.538446+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:49.306476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.507471+0000","last_change":"2023-11-24T08:12:09.507471+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.507471+0000","last_undegraded":"2023-11-24T08:12:09.507471+0000","last_fullsized":"2023-11-24T08:12:09.507471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.533487+0000","last_change":"2023-11-24T08:12:09.530273+0000","last_active":"2023-11-24T08:12:10.533487+0000","last_peered":"2023-11-24T08:12:10.533487+0000","last_clean":"2023-11-24T08:12:10.533487+0000","last_became_active":"2023-11-24T08:12:09.529967+0000","last_became_peered":"2023-11-24T08:12:09.529967+0000","last_unstale":"2023-11-24T08:12:10.533487+0000","last_undegraded":"2023-11-24T08:12:10.533487+0000","last_fullsized":"2023-11-24T08:12:10.533487+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:37.818727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508506+0000","last_change":"2023-11-24T08:12:09.508506+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508506+0000","last_undegraded":"2023-11-24T08:12:09.508506+0000","last_fullsized":"2023-11-24T08:12:09.508506+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.535270+0000","last_change":"2023-11-24T08:12:09.530296+0000","last_active":"2023-11-24T08:12:09.535270+0000","last_peered":"2023-11-24T08:12:09.535270+0000","last_clean":"2023-11-24T08:12:09.535270+0000","last_became_active":"2023-11-24T08:12:09.529840+0000","last_became_peered":"2023-11-24T08:12:09.529840+0000","last_unstale":"2023-11-24T08:12:09.535270+0000","last_undegraded":"2023-11-24T08:12:09.535270+0000","last_fullsized":"2023-11-24T08:12:09.535270+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:51:42.718534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.519744+0000","last_change":"2023-11-24T08:12:09.519744+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.519744+0000","last_undegraded":"2023-11-24T08:12:09.519744+0000","last_fullsized":"2023-11-24T08:12:09.519744+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538516+0000","last_change":"2023-11-24T08:12:09.536994+0000","last_active":"2023-11-24T08:12:09.538516+0000","last_peered":"2023-11-24T08:12:09.538516+0000","last_clean":"2023-11-24T08:12:09.538516+0000","last_became_active":"2023-11-24T08:12:09.536664+0000","last_became_peered":"2023-11-24T08:12:09.536664+0000","last_unstale":"2023-11-24T08:12:09.538516+0000","last_undegraded":"2023-11-24T08:12:09.538516+0000","last_fullsized":"2023-11-24T08:12:09.538516+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:10.151500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":3,"reported_epoch":74,"state":"creating+activating","last_fresh":"2023-11-24T08:12:10.533267+0000","last_change":"2023-11-24T08:12:10.533267+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:10.533267+0000","last_undegraded":"2023-11-24T08:12:10.533267+0000","last_fullsized":"2023-11-24T08:12:10.533267+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:05:10.427272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":["5","1"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.534767+0000","last_change":"2023-11-24T08:12:09.534163+0000","last_active":"2023-11-24T08:12:10.534767+0000","last_peered":"2023-11-24T08:12:10.534767+0000","last_clean":"2023-11-24T08:12:10.534767+0000","last_became_active":"2023-11-24T08:12:09.533384+0000","last_became_peered":"2023-11-24T08:12:09.533384+0000","last_unstale":"2023-11-24T08:12:10.534767+0000","last_undegraded":"2023-11-24T08:12:10.534767+0000","last_fullsized":"2023-11-24T08:12:10.534767+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:39.337647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.522771+0000","last_change":"2023-11-24T08:12:09.522771+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.522771+0000","last_undegraded":"2023-11-24T08:12:09.522771+0000","last_fullsized":"2023-11-24T08:12:09.522771+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.529558+0000","last_change":"2023-11-24T08:12:09.524480+0000","last_active":"2023-11-24T08:12:09.529558+0000","last_peered":"2023-11-24T08:12:09.529558+0000","last_clean":"2023-11-24T08:12:09.529558+0000","last_became_active":"2023-11-24T08:12:09.524212+0000","last_became_peered":"2023-11-24T08:12:09.524212+0000","last_unstale":"2023-11-24T08:12:09.529558+0000","last_undegraded":"2023-11-24T08:12:09.529558+0000","last_fullsized":"2023-11-24T08:12:09.529558+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:13:57.611814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.520763+0000","last_change":"2023-11-24T08:12:09.520763+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.520763+0000","last_undegraded":"2023-11-24T08:12:09.520763+0000","last_fullsized":"2023-11-24T08:12:09.520763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.532684+0000","last_change":"2023-11-24T08:12:09.530037+0000","last_active":"2023-11-24T08:12:09.532684+0000","last_peered":"2023-11-24T08:12:09.532684+0000","last_clean":"2023-11-24T08:12:09.532684+0000","last_became_active":"2023-11-24T08:12:09.529803+0000","last_became_peered":"2023-11-24T08:12:09.529803+0000","last_unstale":"2023-11-24T08:12:09.532684+0000","last_undegraded":"2023-11-24T08:12:09.532684+0000","last_fullsized":"2023-11-24T08:12:09.532684+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:48.699065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.510858+0000","last_change":"2023-11-24T08:12:09.510858+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.510858+0000","last_undegraded":"2023-11-24T08:12:09.510858+0000","last_fullsized":"2023-11-24T08:12:09.510858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.526913+0000","last_change":"2023-11-24T08:12:09.522451+0000","last_active":"2023-11-24T08:12:09.526913+0000","last_peered":"2023-11-24T08:12:09.526913+0000","last_clean":"2023-11-24T08:12:09.526913+0000","last_became_active":"2023-11-24T08:12:09.521531+0000","last_became_peered":"2023-11-24T08:12:09.521531+0000","last_unstale":"2023-11-24T08:12:09.526913+0000","last_undegraded":"2023-11-24T08:12:09.526913+0000","last_fullsized":"2023-11-24T08:12:09.526913+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:46:27.342039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.516090+0000","last_change":"2023-11-24T08:12:09.516090+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.516090+0000","last_undegraded":"2023-11-24T08:12:09.516090+0000","last_fullsized":"2023-11-24T08:12:09.516090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.527191+0000","last_change":"2023-11-24T08:12:09.521396+0000","last_active":"2023-11-24T08:12:09.527191+0000","last_peered":"2023-11-24T08:12:09.527191+0000","last_clean":"2023-11-24T08:12:09.527191+0000","last_became_active":"2023-11-24T08:12:09.520385+0000","last_became_peered":"2023-11-24T08:12:09.520385+0000","last_unstale":"2023-11-24T08:12:09.527191+0000","last_undegraded":"2023-11-24T08:12:09.527191+0000","last_fullsized":"2023-11-24T08:12:09.527191+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:44:25.592586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.506002+0000","last_change":"2023-11-24T08:12:09.506002+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.506002+0000","last_undegraded":"2023-11-24T08:12:09.506002+0000","last_fullsized":"2023-11-24T08:12:09.506002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.531702+0000","last_change":"2023-11-24T08:12:09.529444+0000","last_active":"2023-11-24T08:12:09.531702+0000","last_peered":"2023-11-24T08:12:09.531702+0000","last_clean":"2023-11-24T08:12:09.531702+0000","last_became_active":"2023-11-24T08:12:09.528128+0000","last_became_peered":"2023-11-24T08:12:09.528128+0000","last_unstale":"2023-11-24T08:12:09.531702+0000","last_undegraded":"2023-11-24T08:12:09.531702+0000","last_fullsized":"2023-11-24T08:12:09.531702+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:47:06.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.511825+0000","last_change":"2023-11-24T08:12:09.511825+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.511825+0000","last_undegraded":"2023-11-24T08:12:09.511825+0000","last_fullsized":"2023-11-24T08:12:09.511825+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538378+0000","last_change":"2023-11-24T08:12:09.535030+0000","last_active":"2023-11-24T08:12:09.538378+0000","last_peered":"2023-11-24T08:12:09.538378+0000","last_clean":"2023-11-24T08:12:09.538378+0000","last_became_active":"2023-11-24T08:12:09.530814+0000","last_became_peered":"2023-11-24T08:12:09.530814+0000","last_unstale":"2023-11-24T08:12:09.538378+0000","last_undegraded":"2023-11-24T08:12:09.538378+0000","last_fullsized":"2023-11-24T08:12:09.538378+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:52.597123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":1,"reported_epoch":73,"state":"creating+peering","last_fresh":"2023-11-24T08:12:09.508831+0000","last_change":"2023-11-24T08:12:09.508831+0000","last_active":"2023-11-24T08:12:09.487651+0000","last_peered":"2023-11-24T08:12:09.487651+0000","last_clean":"2023-11-24T08:12:09.487651+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:09.508831+0000","last_undegraded":"2023-11-24T08:12:09.508831+0000","last_fullsized":"2023-11-24T08:12:09.508831+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":9,"reported_epoch":73,"state":"active+clean","last_fresh":"2023-11-24T08:12:09.538420+0000","last_change":"2023-11-24T08:12:09.535433+0000","last_active":"2023-11-24T08:12:09.538420+0000","last_peered":"2023-11-24T08:12:09.538420+0000","last_clean":"2023-11-24T08:12:09.538420+0000","last_became_active":"2023-11-24T08:12:09.534941+0000","last_became_peered":"2023-11-24T08:12:09.534941+0000","last_unstale":"2023-11-24T08:12:09.538420+0000","last_undegraded":"2023-11-24T08:12:09.538420+0000","last_fullsized":"2023-11-24T08:12:09.538420+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:06.870016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":205,"reported_epoch":72,"state":"active+clean","last_fresh":"2023-11-24T08:12:08.502638+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:08.502638+0000","last_peered":"2023-11-24T08:12:08.502638+0000","last_clean":"2023-11-24T08:12:08.502638+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:08.502638+0000","last_undegraded":"2023-11-24T08:12:08.502638+0000","last_fullsized":"2023-11-24T08:12:08.502638+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542190,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.014,"15min":1.012},"min":{"1min":0.64300000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.456,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.367},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.03,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.3340000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.125},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5149999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.157},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1739999999999999,"15min":1.1659999999999999},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.454,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1120000000000001,"15min":1.0660000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.534,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.214,"15min":1.206},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.03}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.246,"15min":1.236},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.3959999999999999,"5min":3.387,"15min":3.387},"last":0.98499999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.246,"15min":1.22},"min":{"1min":0.90500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.583,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.3140000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.234,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.448,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.391,"15min":1.3759999999999999},"min":{"1min":0.96599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6259999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.3919999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4039999999999999,"15min":1.427},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6399999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.25},{"interface":"front","average":{"1min":1.391,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.129,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.698,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.103}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5069999999999999,"15min":1.4830000000000001},"min":{"1min":1.3759999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.806,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.542},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.492,"15min":1.4790000000000001},"min":{"1min":1.1100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.728,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.204}]}]},{"osd":6,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0780000000000001,"15min":1.095},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.472,"5min":2.085,"15min":2.085},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.214,"15min":1.2370000000000001},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4850000000000001,"5min":1.794,"15min":1.794},"last":0.93000000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.70999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6599999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.413},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.80400000000000005,"5min":0.749,"15min":0.749},"max":{"1min":1.7370000000000001,"5min":1.98,"15min":1.98},"last":0.88}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2250000000000001,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6950000000000001,"5min":1.778,"15min":1.778},"last":1.1910000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6899999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.6240000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7849999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.446},"min":{"1min":0.98299999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.847,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.3500000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.343,"15min":1.369},"min":{"1min":0.77600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.7709999999999999},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.242,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4359999999999999,"5min":2.661,"15min":2.661},"last":1.5700000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.625,"15min":1.708},"min":{"1min":1.151,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6699999999999999},{"interface":"front","average":{"1min":1.456,"5min":1.5900000000000001,"15min":1.651},"min":{"1min":1.0249999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.921,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.4970000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.726,"15min":1.78},"min":{"1min":1.2949999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.016,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.8240000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.7250000000000001,"15min":1.7849999999999999},"min":{"1min":1.2050000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1499999999999999,"5min":2.657,"15min":2.657},"last":1.722}]}]},{"osd":5,"up_from":14,"seq":60129542188,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0529999999999999,"15min":1.0249999999999999},"min":{"1min":0.72399999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6100000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.133,"15min":1.157},"min":{"1min":0.77700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":1.837},"last":1.454}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.2849999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1890000000000001,"15min":1.198},"min":{"1min":0.82799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4850000000000001,"5min":1.722,"15min":1.722},"last":1.3700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2649999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4970000000000001,"5min":2.085,"15min":2.085},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3620000000000001},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.3400000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.399,"15min":1.4339999999999999},"min":{"1min":0.95399999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.573,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.468,"15min":1.5169999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.7130000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.397}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":1.1259999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8500000000000001,"5min":3.577,"15min":3.577},"last":1.153},{"interface":"front","average":{"1min":1.49,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.0620000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.958,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.0880000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5780000000000001,"15min":1.6060000000000001},"min":{"1min":1.0209999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.869,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.484},{"interface":"front","average":{"1min":1.554,"5min":1.5489999999999999,"15min":1.5409999999999999},"min":{"1min":1.0089999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0030000000000001,"5min":2.512,"15min":2.512},"last":1.4319999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.6839999999999999,"15min":1.716},"min":{"1min":1.21,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.8839999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.708,"15min":1.7370000000000001},"min":{"1min":1.2949999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.9390000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.581}]}]},{"osd":4,"up_from":14,"seq":60129542189,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.01,"15min":0.98699999999999999},"min":{"1min":0.64700000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6240000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.099,"15min":1.093},"min":{"1min":0.71999999999999997,"5min":0.626,"15min":0.626},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.2310000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6140000000000001,"5min":1.859,"15min":1.859},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1919999999999999,"15min":1.1639999999999999},"min":{"1min":0.79400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1230000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.625}]},{"osd":2,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2749999999999999},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.746},{"interface":"front","average":{"1min":1.115,"5min":1.099,"15min":1.105},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8879999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.1319999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78900000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.659,"5min":2.25,"15min":2.25},"last":0.78200000000000003},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.86599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8360000000000001,"5min":2.746,"15min":2.746},"last":1.8120000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.331},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.452,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8440000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.4990000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4259999999999999,"15min":1.446},"min":{"1min":0.73799999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.4809999999999999,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.532},{"interface":"front","average":{"1min":1.47,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8500000000000001,"5min":2.641,"15min":2.641},"last":1.393}]},{"osd":7,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5149999999999999,"15min":1.5129999999999999},"min":{"1min":0.69599999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2269999999999999,"5min":2.548,"15min":2.548},"last":1.865},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.534,"15min":1.5489999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.4470000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27884,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343956,"statfs":{"total":96636764160,"available":96608210944,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0489999999999999,"15min":1.145},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3180000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0589999999999999,"15min":1.143},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.085,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.72199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.95899999999999996,"15min":0.995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4630000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.534,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.5740000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.1559999999999999,"15min":1.22},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0289999999999999},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5269999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2310000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.385,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.631,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.105},{"interface":"front","average":{"1min":1.141,"5min":1.337,"15min":1.373},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4670000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.125}]},{"osd":5,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.5069999999999999,"15min":1.5680000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.486,"15min":1.536},"min":{"1min":0.872,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8700000000000001,"5min":3.339,"15min":3.339},"last":1.4610000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.5609999999999999,"15min":1.6659999999999999},"min":{"1min":0.746,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.742,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.379,"5min":1.522,"15min":1.5580000000000001},"min":{"1min":1.0249999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.681,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.403}]},{"osd":7,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.605,"15min":1.675},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.603,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.8320000000000001,"15min":2.016},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7629999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.3660000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":872448,"data_stored":365703,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.47899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.21,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.81699999999999995,"15min":0.77700000000000002},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6950000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.6679999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.244,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.92800000000000005},"min":{"1min":0.58999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.514,"5min":1.671,"15min":1.671},"last":1.1160000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98599999999999999,"15min":0.97399999999999998},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.387,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.254},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.232,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1699999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.421},{"interface":"front","average":{"1min":1.294,"5min":1.23,"15min":1.224},"min":{"1min":0.876,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.389}]},{"osd":5,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3620000000000001,"15min":1.431},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.629,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.294,"15min":1.286},"min":{"1min":0.872,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.516}]},{"osd":6,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3839999999999999,"15min":1.4159999999999999},"min":{"1min":0.94799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.895,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3260000000000001,"15min":1.337},"min":{"1min":0.93500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9490000000000001,"5min":1.958,"15min":1.958},"last":1.3149999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4770000000000001},"min":{"1min":0.91000000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.99,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.87,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0739999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.5449999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542189,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":884736,"data_stored":371978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.171},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":0.67100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1359999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":1.121},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.024,"15min":1.0109999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.391,"5min":1.621,"15min":1.621},"last":0.88100000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.143,"15min":1.1579999999999999},"min":{"1min":0.66100000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.534,"5min":1.907,"15min":1.907},"last":1.228},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.018,"15min":1.046},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3740000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.022}]},{"osd":4,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.161,"15min":1.163},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.425,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.874,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6919999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.2929999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.83399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.653,"5min":2.661,"15min":2.661},"last":1.478},{"interface":"front","average":{"1min":1.258,"5min":1.3120000000000001,"15min":1.3360000000000001},"min":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.0820000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.788,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.409},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.343,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.7070000000000001,"5min":2.105,"15min":2.105},"last":1.3200000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4099999999999999,"15min":1.4039999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.393},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8600000000000001,"5min":2.008,"15min":2.008},"last":1.6830000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542189,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1478656,"data_stored":962346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1739999999999999,"5min":1.403,"15min":1.403},"last":0.93899999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95399999999999996},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2989999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.875}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.327},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5209999999999999,"5min":1.986,"15min":1.986},"last":1.081}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0489999999999999,"15min":1.0720000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3380000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.96099999999999997},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.23,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.171},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.641,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.137},{"interface":"front","average":{"1min":1.272,"5min":1.2869999999999999,"15min":1.29},"min":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.78,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.216}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.202,"15min":1.22},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4550000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8220000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.474}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.272},"min":{"1min":0.81299999999999994,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.546,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.284,"15min":1.286},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.742,"5min":2.149,"15min":2.149},"last":1.5669999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3979999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.865,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3839999999999999,"15min":1.4119999999999999},"min":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.603,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4119999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:12:19.967 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:12:23.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.710+0000 7fbf2acff640 1 Processor -- start 2023-11-24T08:12:23.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.710+0000 7fbf2acff640 1 -- start start 2023-11-24T08:12:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf2acff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbf240cf410 0x7fbf24165f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf2acff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2415c5c0 con 0x7fbf240b1de0 2023-11-24T08:12:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf2acff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf2415fb30 con 0x7fbf2413e110 2023-11-24T08:12:23.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf2acff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf240b2550 con 0x7fbf240cf410 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf294fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbf240b1de0 0x7fbf240ced00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:51866/0 (socket says 172.21.15.106:51866) 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf294fc640 1 -- 172.21.15.106:0/1709231323 learned_addr learned my addr 172.21.15.106:0/1709231323 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.712+0000 7fbf2a4fe640 1 --2- 172.21.15.106:0/1709231323 >> v2:172.21.15.156:3300/0 conn(0x7fbf240cf410 0x7fbf24165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2961549991 0 0) 0x7fbf2415c5c0 con 0x7fbf240b1de0 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf10003520 con 0x7fbf240b1de0 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3973354317 0 0) 0x7fbf10003520 con 0x7fbf240b1de0 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 >> v2:172.21.15.156:3300/0 conn(0x7fbf240cf410 msgr2=0x7fbf24165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 --2- 172.21.15.106:0/1709231323 >> v2:172.21.15.156:3300/0 conn(0x7fbf240cf410 0x7fbf24165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 >> v1:172.21.15.156:6789/0 conn(0x7fbf2413e110 legacy=0x7fbf2406ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2415ed70 con 0x7fbf240b1de0 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fbf14001670 con 0x7fbf240b1de0 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 >> v1:172.21.15.106:6789/0 conn(0x7fbf240b1de0 legacy=0x7fbf240ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf10003970 0x7fbf10003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf100047c0 0x7fbf10004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf1000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240b2550 con 0x7fbf10005180 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000c120 con 0x7fbf10003970 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.713+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000c2e0 con 0x7fbf100047c0 2023-11-24T08:12:23.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf2a4fe640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf1000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf294fc640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf100047c0 0x7fbf10004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf29cfd640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf10003970 0x7fbf10003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf2a4fe640 1 -- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf100047c0 msgr2=0x7fbf10004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf2a4fe640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf100047c0 0x7fbf10004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf2a4fe640 1 -- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf10003970 msgr2=0x7fbf10003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf2a4fe640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf10003970 0x7fbf10003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf2a4fe640 1 -- 172.21.15.106:0/1709231323 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10003520 con 0x7fbf10005180 2023-11-24T08:12:23.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.714+0000 7fbf29cfd640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf10003970 0x7fbf10003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.715+0000 7fbf2a4fe640 1 -- 172.21.15.106:0/1709231323 reap_dead start 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.715+0000 7fbf2a4fe640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf1000b980 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009f90 tx=0x7fbf100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c32149f4707f32e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.715+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c02cc20 con 0x7fbf10005180 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.715+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1c00f040 con 0x7fbf10005180 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.715+0000 7fbf28cfb640 1 -- 172.21.15.106:0/1709231323 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c003990 con 0x7fbf10005180 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 -- 172.21.15.106:0/1709231323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 msgr2=0x7fbf1000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf1000b980 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009f90 tx=0x7fbf100057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 -- 172.21.15.106:0/1709231323 shutdown_connections 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 --2- 172.21.15.106:0/1709231323 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf1000b980 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 -- 172.21.15.106:0/1709231323 >> 172.21.15.106:0/1709231323 conn(0x7fbf240cba10 msgr2=0x7fbf240cbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 -- 172.21.15.106:0/1709231323 shutdown_connections 2023-11-24T08:12:23.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.716+0000 7fbf2acff640 1 -- 172.21.15.106:0/1709231323 wait complete. 2023-11-24T08:12:23.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.718+0000 7fbf2acff640 1 Processor -- start 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.718+0000 7fbf2acff640 1 -- start start 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf2acff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf24140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf2acff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf24158130 0x7fbf241585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf2acff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf241f63b0 0x7fbf241f6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf2acff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241f6da0 con 0x7fbf10005180 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf2acff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2413f540 con 0x7fbf241f63b0 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf2acff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf241403a0 con 0x7fbf24158130 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf29cfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf24140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf29cfd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf24140830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:43684/0 (socket says 172.21.15.106:43684) 2023-11-24T08:12:23.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.719+0000 7fbf29cfd640 1 -- 172.21.15.106:0/4220244936 learned_addr learned my addr 172.21.15.106:0/4220244936 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf29cfd640 1 -- 172.21.15.106:0/4220244936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf24158130 msgr2=0x7fbf241585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf29cfd640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf24158130 0x7fbf241585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf29cfd640 1 -- 172.21.15.106:0/4220244936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf241f63b0 msgr2=0x7fbf241f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf29cfd640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf241f63b0 0x7fbf241f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf29cfd640 1 -- 172.21.15.106:0/4220244936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2413f790 con 0x7fbf10005180 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf29cfd640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf24140830 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d560 tx=0x7fbf2000da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20005480 con 0x7fbf10005180 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20003610 con 0x7fbf10005180 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20003980 con 0x7fbf10005180 2023-11-24T08:12:23.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2413fa70 con 0x7fbf10005180 2023-11-24T08:12:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.720+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf24140030 con 0x7fbf10005180 2023-11-24T08:12:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.721+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2404e840 con 0x7fbf10005180 2023-11-24T08:12:23.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.722+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fbf20003ae0 con 0x7fbf10005180 2023-11-24T08:12:23.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.723+0000 7fbf1a7fc640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf0806b260 0x7fbf0806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.723+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf20089ca0 con 0x7fbf10005180 2023-11-24T08:12:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.723+0000 7fbf294fc640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf0806b260 0x7fbf0806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:23.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.724+0000 7fbf294fc640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf0806b260 0x7fbf0806d720 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbf140017a0 tx=0x7fbf14009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:23.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.728+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbf20054660 con 0x7fbf10005180 2023-11-24T08:12:23.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.914+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbf2415f610 con 0x7fbf0806b260 2023-11-24T08:12:23.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.937+0000 7fbf1a7fc640 1 -- 172.21.15.106:0/4220244936 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286981 (secure 0 0 0) 0x7fbf14001c10 con 0x7fbf0806b260 2023-11-24T08:12:23.941 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:23.944 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf0806b260 msgr2=0x7fbf0806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf0806b260 0x7fbf0806d720 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fbf140017a0 tx=0x7fbf14009040 comp rx=0 tx=0).stop 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 msgr2=0x7fbf24140830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf24140830 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000d560 tx=0x7fbf2000da20 comp rx=0 tx=0).stop 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 shutdown_connections 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf241f63b0 0x7fbf241f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbf0806b260 0x7fbf0806d720 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbf24158130 0x7fbf241585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 --2- 172.21.15.106:0/4220244936 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbf10005180 0x7fbf24140830 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 >> 172.21.15.106:0/4220244936 conn(0x7fbf240cba10 msgr2=0x7fbf24072170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 shutdown_connections 2023-11-24T08:12:23.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:23.944+0000 7fbf2acff640 1 -- 172.21.15.106:0/4220244936 wait complete. 2023-11-24T08:12:23.966 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-11-24T08:12:23.339787+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":253636,"kb_used_data":8328,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754721084,"statfs":{"total":773094113280,"available":772834390016,"internally_reserved":0,"allocated":8527872,"data_stored":4265980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.792370"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873499+0000","last_change":"2023-11-24T08:12:10.569043+0000","last_active":"2023-11-24T08:12:11.873499+0000","last_peered":"2023-11-24T08:12:11.873499+0000","last_clean":"2023-11-24T08:12:11.873499+0000","last_became_active":"2023-11-24T08:12:10.568400+0000","last_became_peered":"2023-11-24T08:12:10.568400+0000","last_unstale":"2023-11-24T08:12:11.873499+0000","last_undegraded":"2023-11-24T08:12:11.873499+0000","last_fullsized":"2023-11-24T08:12:11.873499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:08:01.043925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554848+0000","last_change":"2023-11-24T08:12:10.554425+0000","last_active":"2023-11-24T08:12:11.554848+0000","last_peered":"2023-11-24T08:12:11.554848+0000","last_clean":"2023-11-24T08:12:11.554848+0000","last_became_active":"2023-11-24T08:12:10.554101+0000","last_became_peered":"2023-11-24T08:12:10.554101+0000","last_unstale":"2023-11-24T08:12:11.554848+0000","last_undegraded":"2023-11-24T08:12:11.554848+0000","last_fullsized":"2023-11-24T08:12:11.554848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:42.609564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872554+0000","last_change":"2023-11-24T08:12:10.559525+0000","last_active":"2023-11-24T08:12:11.872554+0000","last_peered":"2023-11-24T08:12:11.872554+0000","last_clean":"2023-11-24T08:12:11.872554+0000","last_became_active":"2023-11-24T08:12:10.558959+0000","last_became_peered":"2023-11-24T08:12:10.558959+0000","last_unstale":"2023-11-24T08:12:11.872554+0000","last_undegraded":"2023-11-24T08:12:11.872554+0000","last_fullsized":"2023-11-24T08:12:11.872554+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:38:30.747649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.548659+0000","last_change":"2023-11-24T08:12:10.547364+0000","last_active":"2023-11-24T08:12:11.548659+0000","last_peered":"2023-11-24T08:12:11.548659+0000","last_clean":"2023-11-24T08:12:11.548659+0000","last_became_active":"2023-11-24T08:12:10.546661+0000","last_became_peered":"2023-11-24T08:12:10.546661+0000","last_unstale":"2023-11-24T08:12:11.548659+0000","last_undegraded":"2023-11-24T08:12:11.548659+0000","last_fullsized":"2023-11-24T08:12:11.548659+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:33:07.880645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.870567+0000","last_change":"2023-11-24T08:12:10.557112+0000","last_active":"2023-11-24T08:12:11.870567+0000","last_peered":"2023-11-24T08:12:11.870567+0000","last_clean":"2023-11-24T08:12:11.870567+0000","last_became_active":"2023-11-24T08:12:10.556391+0000","last_became_peered":"2023-11-24T08:12:10.556391+0000","last_unstale":"2023-11-24T08:12:11.870567+0000","last_undegraded":"2023-11-24T08:12:11.870567+0000","last_fullsized":"2023-11-24T08:12:11.870567+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:23:05.471882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.552759+0000","last_change":"2023-11-24T08:12:10.553370+0000","last_active":"2023-11-24T08:12:11.552759+0000","last_peered":"2023-11-24T08:12:11.552759+0000","last_clean":"2023-11-24T08:12:11.552759+0000","last_became_active":"2023-11-24T08:12:10.553076+0000","last_became_peered":"2023-11-24T08:12:10.553076+0000","last_unstale":"2023-11-24T08:12:11.552759+0000","last_undegraded":"2023-11-24T08:12:11.552759+0000","last_fullsized":"2023-11-24T08:12:11.552759+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:43:12.647787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871097+0000","last_change":"2023-11-24T08:12:10.549320+0000","last_active":"2023-11-24T08:12:11.871097+0000","last_peered":"2023-11-24T08:12:11.871097+0000","last_clean":"2023-11-24T08:12:11.871097+0000","last_became_active":"2023-11-24T08:12:10.549039+0000","last_became_peered":"2023-11-24T08:12:10.549039+0000","last_unstale":"2023-11-24T08:12:11.871097+0000","last_undegraded":"2023-11-24T08:12:11.871097+0000","last_fullsized":"2023-11-24T08:12:11.871097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:15:41.290338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.552440+0000","last_change":"2023-11-24T08:12:10.550937+0000","last_active":"2023-11-24T08:12:11.552440+0000","last_peered":"2023-11-24T08:12:11.552440+0000","last_clean":"2023-11-24T08:12:11.552440+0000","last_became_active":"2023-11-24T08:12:10.550014+0000","last_became_peered":"2023-11-24T08:12:10.550014+0000","last_unstale":"2023-11-24T08:12:11.552440+0000","last_undegraded":"2023-11-24T08:12:11.552440+0000","last_fullsized":"2023-11-24T08:12:11.552440+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:06:33.302066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873138+0000","last_change":"2023-11-24T08:12:10.570392+0000","last_active":"2023-11-24T08:12:11.873138+0000","last_peered":"2023-11-24T08:12:11.873138+0000","last_clean":"2023-11-24T08:12:11.873138+0000","last_became_active":"2023-11-24T08:12:10.569744+0000","last_became_peered":"2023-11-24T08:12:10.569744+0000","last_unstale":"2023-11-24T08:12:11.873138+0000","last_undegraded":"2023-11-24T08:12:11.873138+0000","last_fullsized":"2023-11-24T08:12:11.873138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:04:15.053944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.555074+0000","last_change":"2023-11-24T08:12:10.566299+0000","last_active":"2023-11-24T08:12:11.555074+0000","last_peered":"2023-11-24T08:12:11.555074+0000","last_clean":"2023-11-24T08:12:11.555074+0000","last_became_active":"2023-11-24T08:12:10.565243+0000","last_became_peered":"2023-11-24T08:12:10.565243+0000","last_unstale":"2023-11-24T08:12:11.555074+0000","last_undegraded":"2023-11-24T08:12:11.555074+0000","last_fullsized":"2023-11-24T08:12:11.555074+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:26:01.385882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872695+0000","last_change":"2023-11-24T08:12:10.559144+0000","last_active":"2023-11-24T08:12:11.872695+0000","last_peered":"2023-11-24T08:12:11.872695+0000","last_clean":"2023-11-24T08:12:11.872695+0000","last_became_active":"2023-11-24T08:12:10.558801+0000","last_became_peered":"2023-11-24T08:12:10.558801+0000","last_unstale":"2023-11-24T08:12:11.872695+0000","last_undegraded":"2023-11-24T08:12:11.872695+0000","last_fullsized":"2023-11-24T08:12:11.872695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:22:47.665284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.548671+0000","last_change":"2023-11-24T08:12:10.550199+0000","last_active":"2023-11-24T08:12:11.548671+0000","last_peered":"2023-11-24T08:12:11.548671+0000","last_clean":"2023-11-24T08:12:11.548671+0000","last_became_active":"2023-11-24T08:12:10.549940+0000","last_became_peered":"2023-11-24T08:12:10.549940+0000","last_unstale":"2023-11-24T08:12:11.548671+0000","last_undegraded":"2023-11-24T08:12:11.548671+0000","last_fullsized":"2023-11-24T08:12:11.548671+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:07:11.162128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.868985+0000","last_change":"2023-11-24T08:12:10.567354+0000","last_active":"2023-11-24T08:12:11.868985+0000","last_peered":"2023-11-24T08:12:11.868985+0000","last_clean":"2023-11-24T08:12:11.868985+0000","last_became_active":"2023-11-24T08:12:10.567081+0000","last_became_peered":"2023-11-24T08:12:10.567081+0000","last_unstale":"2023-11-24T08:12:11.868985+0000","last_undegraded":"2023-11-24T08:12:11.868985+0000","last_fullsized":"2023-11-24T08:12:11.868985+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:51:48.259652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.870418+0000","last_change":"2023-11-24T08:12:10.550935+0000","last_active":"2023-11-24T08:12:11.870418+0000","last_peered":"2023-11-24T08:12:11.870418+0000","last_clean":"2023-11-24T08:12:11.870418+0000","last_became_active":"2023-11-24T08:12:10.550577+0000","last_became_peered":"2023-11-24T08:12:10.550577+0000","last_unstale":"2023-11-24T08:12:11.870418+0000","last_undegraded":"2023-11-24T08:12:11.870418+0000","last_fullsized":"2023-11-24T08:12:11.870418+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:42:01.247078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.868570+0000","last_change":"2023-11-24T08:12:10.555865+0000","last_active":"2023-11-24T08:12:11.868570+0000","last_peered":"2023-11-24T08:12:11.868570+0000","last_clean":"2023-11-24T08:12:11.868570+0000","last_became_active":"2023-11-24T08:12:10.555578+0000","last_became_peered":"2023-11-24T08:12:10.555578+0000","last_unstale":"2023-11-24T08:12:11.868570+0000","last_undegraded":"2023-11-24T08:12:11.868570+0000","last_fullsized":"2023-11-24T08:12:11.868570+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:14:14.040783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554596+0000","last_change":"2023-11-24T08:12:10.564799+0000","last_active":"2023-11-24T08:12:11.554596+0000","last_peered":"2023-11-24T08:12:11.554596+0000","last_clean":"2023-11-24T08:12:11.554596+0000","last_became_active":"2023-11-24T08:12:10.563390+0000","last_became_peered":"2023-11-24T08:12:10.563390+0000","last_unstale":"2023-11-24T08:12:11.554596+0000","last_undegraded":"2023-11-24T08:12:11.554596+0000","last_fullsized":"2023-11-24T08:12:11.554596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:42:42.609564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873409+0000","last_change":"2023-11-24T08:12:10.564744+0000","last_active":"2023-11-24T08:12:11.873409+0000","last_peered":"2023-11-24T08:12:11.873409+0000","last_clean":"2023-11-24T08:12:11.873409+0000","last_became_active":"2023-11-24T08:12:10.564235+0000","last_became_peered":"2023-11-24T08:12:10.564235+0000","last_unstale":"2023-11-24T08:12:11.873409+0000","last_undegraded":"2023-11-24T08:12:11.873409+0000","last_fullsized":"2023-11-24T08:12:11.873409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:41.056414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.548897+0000","last_change":"2023-11-24T08:12:10.546942+0000","last_active":"2023-11-24T08:12:11.548897+0000","last_peered":"2023-11-24T08:12:11.548897+0000","last_clean":"2023-11-24T08:12:11.548897+0000","last_became_active":"2023-11-24T08:12:10.546620+0000","last_became_peered":"2023-11-24T08:12:10.546620+0000","last_unstale":"2023-11-24T08:12:11.548897+0000","last_undegraded":"2023-11-24T08:12:11.548897+0000","last_fullsized":"2023-11-24T08:12:11.548897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:09:00.296920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.553912+0000","last_change":"2023-11-24T08:12:10.546461+0000","last_active":"2023-11-24T08:12:11.553912+0000","last_peered":"2023-11-24T08:12:11.553912+0000","last_clean":"2023-11-24T08:12:11.553912+0000","last_became_active":"2023-11-24T08:12:10.546163+0000","last_became_peered":"2023-11-24T08:12:10.546163+0000","last_unstale":"2023-11-24T08:12:11.553912+0000","last_undegraded":"2023-11-24T08:12:11.553912+0000","last_fullsized":"2023-11-24T08:12:11.553912+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:44.216570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.868672+0000","last_change":"2023-11-24T08:12:10.559687+0000","last_active":"2023-11-24T08:12:11.868672+0000","last_peered":"2023-11-24T08:12:11.868672+0000","last_clean":"2023-11-24T08:12:11.868672+0000","last_became_active":"2023-11-24T08:12:10.559425+0000","last_became_peered":"2023-11-24T08:12:10.559425+0000","last_unstale":"2023-11-24T08:12:11.868672+0000","last_undegraded":"2023-11-24T08:12:11.868672+0000","last_fullsized":"2023-11-24T08:12:11.868672+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:57:24.445257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554095+0000","last_change":"2023-11-24T08:12:10.559428+0000","last_active":"2023-11-24T08:12:11.554095+0000","last_peered":"2023-11-24T08:12:11.554095+0000","last_clean":"2023-11-24T08:12:11.554095+0000","last_became_active":"2023-11-24T08:12:10.558504+0000","last_became_peered":"2023-11-24T08:12:10.558504+0000","last_unstale":"2023-11-24T08:12:11.554095+0000","last_undegraded":"2023-11-24T08:12:11.554095+0000","last_fullsized":"2023-11-24T08:12:11.554095+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:59.711405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.553205+0000","last_change":"2023-11-24T08:12:10.558418+0000","last_active":"2023-11-24T08:12:11.553205+0000","last_peered":"2023-11-24T08:12:11.553205+0000","last_clean":"2023-11-24T08:12:11.553205+0000","last_became_active":"2023-11-24T08:12:10.558129+0000","last_became_peered":"2023-11-24T08:12:10.558129+0000","last_unstale":"2023-11-24T08:12:11.553205+0000","last_undegraded":"2023-11-24T08:12:11.553205+0000","last_fullsized":"2023-11-24T08:12:11.553205+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:19.996340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872954+0000","last_change":"2023-11-24T08:12:10.555799+0000","last_active":"2023-11-24T08:12:11.872954+0000","last_peered":"2023-11-24T08:12:11.872954+0000","last_clean":"2023-11-24T08:12:11.872954+0000","last_became_active":"2023-11-24T08:12:10.555538+0000","last_became_peered":"2023-11-24T08:12:10.555538+0000","last_unstale":"2023-11-24T08:12:11.872954+0000","last_undegraded":"2023-11-24T08:12:11.872954+0000","last_fullsized":"2023-11-24T08:12:11.872954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:43:46.347554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871206+0000","last_change":"2023-11-24T08:12:10.561781+0000","last_active":"2023-11-24T08:12:11.871206+0000","last_peered":"2023-11-24T08:12:11.871206+0000","last_clean":"2023-11-24T08:12:11.871206+0000","last_became_active":"2023-11-24T08:12:10.561507+0000","last_became_peered":"2023-11-24T08:12:10.561507+0000","last_unstale":"2023-11-24T08:12:11.871206+0000","last_undegraded":"2023-11-24T08:12:11.871206+0000","last_fullsized":"2023-11-24T08:12:11.871206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:14:47.593901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.555222+0000","last_change":"2023-11-24T08:12:10.554844+0000","last_active":"2023-11-24T08:12:11.555222+0000","last_peered":"2023-11-24T08:12:11.555222+0000","last_clean":"2023-11-24T08:12:11.555222+0000","last_became_active":"2023-11-24T08:12:10.554580+0000","last_became_peered":"2023-11-24T08:12:10.554580+0000","last_unstale":"2023-11-24T08:12:11.555222+0000","last_undegraded":"2023-11-24T08:12:11.555222+0000","last_fullsized":"2023-11-24T08:12:11.555222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:38:23.439017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872364+0000","last_change":"2023-11-24T08:12:10.548255+0000","last_active":"2023-11-24T08:12:11.872364+0000","last_peered":"2023-11-24T08:12:11.872364+0000","last_clean":"2023-11-24T08:12:11.872364+0000","last_became_active":"2023-11-24T08:12:10.547842+0000","last_became_peered":"2023-11-24T08:12:10.547842+0000","last_unstale":"2023-11-24T08:12:11.872364+0000","last_undegraded":"2023-11-24T08:12:11.872364+0000","last_fullsized":"2023-11-24T08:12:11.872364+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:21:05.743175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872826+0000","last_change":"2023-11-24T08:12:10.552434+0000","last_active":"2023-11-24T08:12:11.872826+0000","last_peered":"2023-11-24T08:12:11.872826+0000","last_clean":"2023-11-24T08:12:11.872826+0000","last_became_active":"2023-11-24T08:12:10.552111+0000","last_became_peered":"2023-11-24T08:12:10.552111+0000","last_unstale":"2023-11-24T08:12:11.872826+0000","last_undegraded":"2023-11-24T08:12:11.872826+0000","last_fullsized":"2023-11-24T08:12:11.872826+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:27:54.958861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554081+0000","last_change":"2023-11-24T08:12:10.562755+0000","last_active":"2023-11-24T08:12:11.554081+0000","last_peered":"2023-11-24T08:12:11.554081+0000","last_clean":"2023-11-24T08:12:11.554081+0000","last_became_active":"2023-11-24T08:12:10.562487+0000","last_became_peered":"2023-11-24T08:12:10.562487+0000","last_unstale":"2023-11-24T08:12:11.554081+0000","last_undegraded":"2023-11-24T08:12:11.554081+0000","last_fullsized":"2023-11-24T08:12:11.554081+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:21:44.216570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871455+0000","last_change":"2023-11-24T08:12:10.548756+0000","last_active":"2023-11-24T08:12:11.871455+0000","last_peered":"2023-11-24T08:12:11.871455+0000","last_clean":"2023-11-24T08:12:11.871455+0000","last_became_active":"2023-11-24T08:12:10.548060+0000","last_became_peered":"2023-11-24T08:12:10.548060+0000","last_unstale":"2023-11-24T08:12:11.871455+0000","last_undegraded":"2023-11-24T08:12:11.871455+0000","last_fullsized":"2023-11-24T08:12:11.871455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:22.602382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871499+0000","last_change":"2023-11-24T08:12:10.544012+0000","last_active":"2023-11-24T08:12:11.871499+0000","last_peered":"2023-11-24T08:12:11.871499+0000","last_clean":"2023-11-24T08:12:11.871499+0000","last_became_active":"2023-11-24T08:12:10.543658+0000","last_became_peered":"2023-11-24T08:12:10.543658+0000","last_unstale":"2023-11-24T08:12:11.871499+0000","last_undegraded":"2023-11-24T08:12:11.871499+0000","last_fullsized":"2023-11-24T08:12:11.871499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:18:54.097413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871507+0000","last_change":"2023-11-24T08:12:10.551277+0000","last_active":"2023-11-24T08:12:11.871507+0000","last_peered":"2023-11-24T08:12:11.871507+0000","last_clean":"2023-11-24T08:12:11.871507+0000","last_became_active":"2023-11-24T08:12:10.550960+0000","last_became_peered":"2023-11-24T08:12:10.550960+0000","last_unstale":"2023-11-24T08:12:11.871507+0000","last_undegraded":"2023-11-24T08:12:11.871507+0000","last_fullsized":"2023-11-24T08:12:11.871507+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:28:46.456451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.552812+0000","last_change":"2023-11-24T08:12:10.550673+0000","last_active":"2023-11-24T08:12:11.552812+0000","last_peered":"2023-11-24T08:12:11.552812+0000","last_clean":"2023-11-24T08:12:11.552812+0000","last_became_active":"2023-11-24T08:12:10.549624+0000","last_became_peered":"2023-11-24T08:12:10.549624+0000","last_unstale":"2023-11-24T08:12:11.552812+0000","last_undegraded":"2023-11-24T08:12:11.552812+0000","last_fullsized":"2023-11-24T08:12:11.552812+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:43.830381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873481+0000","last_change":"2023-11-24T08:12:10.563475+0000","last_active":"2023-11-24T08:12:11.873481+0000","last_peered":"2023-11-24T08:12:11.873481+0000","last_clean":"2023-11-24T08:12:11.873481+0000","last_became_active":"2023-11-24T08:12:10.563181+0000","last_became_peered":"2023-11-24T08:12:10.563181+0000","last_unstale":"2023-11-24T08:12:11.873481+0000","last_undegraded":"2023-11-24T08:12:11.873481+0000","last_fullsized":"2023-11-24T08:12:11.873481+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:59:01.852055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"74'4","reported_seq":16,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.568143+0000","last_change":"2023-11-24T08:12:09.535765+0000","last_active":"2023-11-24T08:12:10.568143+0000","last_peered":"2023-11-24T08:12:10.568143+0000","last_clean":"2023-11-24T08:12:10.568143+0000","last_became_active":"2023-11-24T08:12:09.535385+0000","last_became_peered":"2023-11-24T08:12:09.535385+0000","last_unstale":"2023-11-24T08:12:10.568143+0000","last_undegraded":"2023-11-24T08:12:10.568143+0000","last_fullsized":"2023-11-24T08:12:10.568143+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:39:48.699065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873471+0000","last_change":"2023-11-24T08:12:10.565063+0000","last_active":"2023-11-24T08:12:11.873471+0000","last_peered":"2023-11-24T08:12:11.873471+0000","last_clean":"2023-11-24T08:12:11.873471+0000","last_became_active":"2023-11-24T08:12:10.564387+0000","last_became_peered":"2023-11-24T08:12:10.564387+0000","last_unstale":"2023-11-24T08:12:11.873471+0000","last_undegraded":"2023-11-24T08:12:11.873471+0000","last_fullsized":"2023-11-24T08:12:11.873471+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:25:25.368675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.567539+0000","last_change":"2023-11-24T08:12:09.524544+0000","last_active":"2023-11-24T08:12:10.567539+0000","last_peered":"2023-11-24T08:12:10.567539+0000","last_clean":"2023-11-24T08:12:10.567539+0000","last_became_active":"2023-11-24T08:12:09.524188+0000","last_became_peered":"2023-11-24T08:12:09.524188+0000","last_unstale":"2023-11-24T08:12:10.567539+0000","last_undegraded":"2023-11-24T08:12:10.567539+0000","last_fullsized":"2023-11-24T08:12:10.567539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:44:30.832638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554287+0000","last_change":"2023-11-24T08:12:10.559969+0000","last_active":"2023-11-24T08:12:11.554287+0000","last_peered":"2023-11-24T08:12:11.554287+0000","last_clean":"2023-11-24T08:12:11.554287+0000","last_became_active":"2023-11-24T08:12:10.557733+0000","last_became_peered":"2023-11-24T08:12:10.557733+0000","last_unstale":"2023-11-24T08:12:11.554287+0000","last_undegraded":"2023-11-24T08:12:11.554287+0000","last_fullsized":"2023-11-24T08:12:11.554287+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:15.685869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.573607+0000","last_change":"2023-11-24T08:12:09.533833+0000","last_active":"2023-11-24T08:12:10.573607+0000","last_peered":"2023-11-24T08:12:10.573607+0000","last_clean":"2023-11-24T08:12:10.573607+0000","last_became_active":"2023-11-24T08:12:09.533263+0000","last_became_peered":"2023-11-24T08:12:09.533263+0000","last_unstale":"2023-11-24T08:12:10.573607+0000","last_undegraded":"2023-11-24T08:12:10.573607+0000","last_fullsized":"2023-11-24T08:12:10.573607+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:17:42.819752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872763+0000","last_change":"2023-11-24T08:12:10.562378+0000","last_active":"2023-11-24T08:12:11.872763+0000","last_peered":"2023-11-24T08:12:11.872763+0000","last_clean":"2023-11-24T08:12:11.872763+0000","last_became_active":"2023-11-24T08:12:10.562070+0000","last_became_peered":"2023-11-24T08:12:10.562070+0000","last_unstale":"2023-11-24T08:12:11.872763+0000","last_undegraded":"2023-11-24T08:12:11.872763+0000","last_fullsized":"2023-11-24T08:12:11.872763+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:27:41.799516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.569821+0000","last_change":"2023-11-24T08:12:09.529735+0000","last_active":"2023-11-24T08:12:10.569821+0000","last_peered":"2023-11-24T08:12:10.569821+0000","last_clean":"2023-11-24T08:12:10.569821+0000","last_became_active":"2023-11-24T08:12:09.529517+0000","last_became_peered":"2023-11-24T08:12:09.529517+0000","last_unstale":"2023-11-24T08:12:10.569821+0000","last_undegraded":"2023-11-24T08:12:10.569821+0000","last_fullsized":"2023-11-24T08:12:10.569821+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:31.878354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871890+0000","last_change":"2023-11-24T08:12:10.554779+0000","last_active":"2023-11-24T08:12:11.871890+0000","last_peered":"2023-11-24T08:12:11.871890+0000","last_clean":"2023-11-24T08:12:11.871890+0000","last_became_active":"2023-11-24T08:12:10.551612+0000","last_became_peered":"2023-11-24T08:12:10.551612+0000","last_unstale":"2023-11-24T08:12:11.871890+0000","last_undegraded":"2023-11-24T08:12:11.871890+0000","last_fullsized":"2023-11-24T08:12:11.871890+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:09:00.219825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.571370+0000","last_change":"2023-11-24T08:12:09.522576+0000","last_active":"2023-11-24T08:12:10.571370+0000","last_peered":"2023-11-24T08:12:10.571370+0000","last_clean":"2023-11-24T08:12:10.571370+0000","last_became_active":"2023-11-24T08:12:09.522114+0000","last_became_peered":"2023-11-24T08:12:09.522114+0000","last_unstale":"2023-11-24T08:12:10.571370+0000","last_undegraded":"2023-11-24T08:12:10.571370+0000","last_fullsized":"2023-11-24T08:12:10.571370+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:57:23.434629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554288+0000","last_change":"2023-11-24T08:12:10.552165+0000","last_active":"2023-11-24T08:12:11.554288+0000","last_peered":"2023-11-24T08:12:11.554288+0000","last_clean":"2023-11-24T08:12:11.554288+0000","last_became_active":"2023-11-24T08:12:10.551841+0000","last_became_peered":"2023-11-24T08:12:10.551841+0000","last_unstale":"2023-11-24T08:12:11.554288+0000","last_undegraded":"2023-11-24T08:12:11.554288+0000","last_fullsized":"2023-11-24T08:12:11.554288+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:45:15.685869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.571530+0000","last_change":"2023-11-24T08:12:09.531869+0000","last_active":"2023-11-24T08:12:10.571530+0000","last_peered":"2023-11-24T08:12:10.571530+0000","last_clean":"2023-11-24T08:12:10.571530+0000","last_became_active":"2023-11-24T08:12:09.531598+0000","last_became_peered":"2023-11-24T08:12:09.531598+0000","last_unstale":"2023-11-24T08:12:10.571530+0000","last_undegraded":"2023-11-24T08:12:10.571530+0000","last_fullsized":"2023-11-24T08:12:10.571530+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:26:27.517078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.870176+0000","last_change":"2023-11-24T08:12:10.557657+0000","last_active":"2023-11-24T08:12:11.870176+0000","last_peered":"2023-11-24T08:12:11.870176+0000","last_clean":"2023-11-24T08:12:11.870176+0000","last_became_active":"2023-11-24T08:12:10.557397+0000","last_became_peered":"2023-11-24T08:12:10.557397+0000","last_unstale":"2023-11-24T08:12:11.870176+0000","last_undegraded":"2023-11-24T08:12:11.870176+0000","last_fullsized":"2023-11-24T08:12:11.870176+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:21:13.644554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.521004+0000","last_change":"2023-11-24T08:12:09.513949+0000","last_active":"2023-11-24T08:12:10.521004+0000","last_peered":"2023-11-24T08:12:10.521004+0000","last_clean":"2023-11-24T08:12:10.521004+0000","last_became_active":"2023-11-24T08:12:09.513517+0000","last_became_peered":"2023-11-24T08:12:09.513517+0000","last_unstale":"2023-11-24T08:12:10.521004+0000","last_undegraded":"2023-11-24T08:12:10.521004+0000","last_fullsized":"2023-11-24T08:12:10.521004+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:42:00.236449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.868952+0000","last_change":"2023-11-24T08:12:10.558283+0000","last_active":"2023-11-24T08:12:11.868952+0000","last_peered":"2023-11-24T08:12:11.868952+0000","last_clean":"2023-11-24T08:12:11.868952+0000","last_became_active":"2023-11-24T08:12:10.557953+0000","last_became_peered":"2023-11-24T08:12:10.557953+0000","last_unstale":"2023-11-24T08:12:11.868952+0000","last_undegraded":"2023-11-24T08:12:11.868952+0000","last_fullsized":"2023-11-24T08:12:11.868952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:43.947914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.541797+0000","last_change":"2023-11-24T08:12:09.519208+0000","last_active":"2023-11-24T08:12:10.541797+0000","last_peered":"2023-11-24T08:12:10.541797+0000","last_clean":"2023-11-24T08:12:10.541797+0000","last_became_active":"2023-11-24T08:12:09.518952+0000","last_became_peered":"2023-11-24T08:12:09.518952+0000","last_unstale":"2023-11-24T08:12:10.541797+0000","last_undegraded":"2023-11-24T08:12:10.541797+0000","last_fullsized":"2023-11-24T08:12:10.541797+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:40:22.592432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872969+0000","last_change":"2023-11-24T08:12:10.542676+0000","last_active":"2023-11-24T08:12:11.872969+0000","last_peered":"2023-11-24T08:12:11.872969+0000","last_clean":"2023-11-24T08:12:11.872969+0000","last_became_active":"2023-11-24T08:12:10.542269+0000","last_became_peered":"2023-11-24T08:12:10.542269+0000","last_unstale":"2023-11-24T08:12:11.872969+0000","last_undegraded":"2023-11-24T08:12:11.872969+0000","last_fullsized":"2023-11-24T08:12:11.872969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:02:31.321354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.541106+0000","last_change":"2023-11-24T08:12:09.519666+0000","last_active":"2023-11-24T08:12:10.541106+0000","last_peered":"2023-11-24T08:12:10.541106+0000","last_clean":"2023-11-24T08:12:10.541106+0000","last_became_active":"2023-11-24T08:12:09.519348+0000","last_became_peered":"2023-11-24T08:12:09.519348+0000","last_unstale":"2023-11-24T08:12:10.541106+0000","last_undegraded":"2023-11-24T08:12:10.541106+0000","last_fullsized":"2023-11-24T08:12:10.541106+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:43:11.637158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873602+0000","last_change":"2023-11-24T08:12:10.569759+0000","last_active":"2023-11-24T08:12:11.873602+0000","last_peered":"2023-11-24T08:12:11.873602+0000","last_clean":"2023-11-24T08:12:11.873602+0000","last_became_active":"2023-11-24T08:12:10.568768+0000","last_became_peered":"2023-11-24T08:12:10.568768+0000","last_unstale":"2023-11-24T08:12:11.873602+0000","last_undegraded":"2023-11-24T08:12:11.873602+0000","last_fullsized":"2023-11-24T08:12:11.873602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:44:31.843267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.530455+0000","last_change":"2023-11-24T08:12:09.537826+0000","last_active":"2023-11-24T08:12:10.530455+0000","last_peered":"2023-11-24T08:12:10.530455+0000","last_clean":"2023-11-24T08:12:10.530455+0000","last_became_active":"2023-11-24T08:12:09.537555+0000","last_became_peered":"2023-11-24T08:12:09.537555+0000","last_unstale":"2023-11-24T08:12:10.530455+0000","last_undegraded":"2023-11-24T08:12:10.530455+0000","last_fullsized":"2023-11-24T08:12:10.530455+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:27.721452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.554129+0000","last_change":"2023-11-24T08:12:10.554484+0000","last_active":"2023-11-24T08:12:11.554129+0000","last_peered":"2023-11-24T08:12:11.554129+0000","last_clean":"2023-11-24T08:12:11.554129+0000","last_became_active":"2023-11-24T08:12:10.554135+0000","last_became_peered":"2023-11-24T08:12:10.554135+0000","last_unstale":"2023-11-24T08:12:11.554129+0000","last_undegraded":"2023-11-24T08:12:11.554129+0000","last_fullsized":"2023-11-24T08:12:11.554129+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:30:59.711405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"74'3","reported_seq":15,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.569381+0000","last_change":"2023-11-24T08:12:09.524384+0000","last_active":"2023-11-24T08:12:10.569381+0000","last_peered":"2023-11-24T08:12:10.569381+0000","last_clean":"2023-11-24T08:12:10.569381+0000","last_became_active":"2023-11-24T08:12:09.524038+0000","last_became_peered":"2023-11-24T08:12:09.524038+0000","last_unstale":"2023-11-24T08:12:10.569381+0000","last_undegraded":"2023-11-24T08:12:10.569381+0000","last_fullsized":"2023-11-24T08:12:10.569381+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:42.937285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871710+0000","last_change":"2023-11-24T08:12:10.549359+0000","last_active":"2023-11-24T08:12:11.871710+0000","last_peered":"2023-11-24T08:12:11.871710+0000","last_clean":"2023-11-24T08:12:11.871710+0000","last_became_active":"2023-11-24T08:12:10.548687+0000","last_became_peered":"2023-11-24T08:12:10.548687+0000","last_unstale":"2023-11-24T08:12:11.871710+0000","last_undegraded":"2023-11-24T08:12:11.871710+0000","last_fullsized":"2023-11-24T08:12:11.871710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:28:53.607752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"74'6","reported_seq":18,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.575616+0000","last_change":"2023-11-24T08:12:09.514045+0000","last_active":"2023-11-24T08:12:10.575616+0000","last_peered":"2023-11-24T08:12:10.575616+0000","last_clean":"2023-11-24T08:12:10.575616+0000","last_became_active":"2023-11-24T08:12:09.513337+0000","last_became_peered":"2023-11-24T08:12:09.513337+0000","last_unstale":"2023-11-24T08:12:10.575616+0000","last_undegraded":"2023-11-24T08:12:10.575616+0000","last_fullsized":"2023-11-24T08:12:10.575616+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:50:40.045785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872301+0000","last_change":"2023-11-24T08:12:10.564442+0000","last_active":"2023-11-24T08:12:11.872301+0000","last_peered":"2023-11-24T08:12:11.872301+0000","last_clean":"2023-11-24T08:12:11.872301+0000","last_became_active":"2023-11-24T08:12:10.564141+0000","last_became_peered":"2023-11-24T08:12:10.564141+0000","last_unstale":"2023-11-24T08:12:11.872301+0000","last_undegraded":"2023-11-24T08:12:11.872301+0000","last_fullsized":"2023-11-24T08:12:11.872301+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:19:39.670630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.568071+0000","last_change":"2023-11-24T08:12:09.528456+0000","last_active":"2023-11-24T08:12:10.568071+0000","last_peered":"2023-11-24T08:12:10.568071+0000","last_clean":"2023-11-24T08:12:10.568071+0000","last_became_active":"2023-11-24T08:12:09.528140+0000","last_became_peered":"2023-11-24T08:12:09.528140+0000","last_unstale":"2023-11-24T08:12:10.568071+0000","last_undegraded":"2023-11-24T08:12:10.568071+0000","last_fullsized":"2023-11-24T08:12:10.568071+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:17:08.890125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871700+0000","last_change":"2023-11-24T08:12:10.553797+0000","last_active":"2023-11-24T08:12:11.871700+0000","last_peered":"2023-11-24T08:12:11.871700+0000","last_clean":"2023-11-24T08:12:11.871700+0000","last_became_active":"2023-11-24T08:12:10.553537+0000","last_became_peered":"2023-11-24T08:12:10.553537+0000","last_unstale":"2023-11-24T08:12:11.871700+0000","last_undegraded":"2023-11-24T08:12:11.871700+0000","last_fullsized":"2023-11-24T08:12:11.871700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:53.827335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.529989+0000","last_change":"2023-11-24T08:12:09.537450+0000","last_active":"2023-11-24T08:12:10.529989+0000","last_peered":"2023-11-24T08:12:10.529989+0000","last_clean":"2023-11-24T08:12:10.529989+0000","last_became_active":"2023-11-24T08:12:09.537196+0000","last_became_peered":"2023-11-24T08:12:09.537196+0000","last_unstale":"2023-11-24T08:12:10.529989+0000","last_undegraded":"2023-11-24T08:12:10.529989+0000","last_fullsized":"2023-11-24T08:12:10.529989+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:20:10.605909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.868295+0000","last_change":"2023-11-24T08:12:10.566784+0000","last_active":"2023-11-24T08:12:11.868295+0000","last_peered":"2023-11-24T08:12:11.868295+0000","last_clean":"2023-11-24T08:12:11.868295+0000","last_became_active":"2023-11-24T08:12:10.566480+0000","last_became_peered":"2023-11-24T08:12:10.566480+0000","last_unstale":"2023-11-24T08:12:11.868295+0000","last_undegraded":"2023-11-24T08:12:11.868295+0000","last_fullsized":"2023-11-24T08:12:11.868295+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:24.907859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.542014+0000","last_change":"2023-11-24T08:12:09.530752+0000","last_active":"2023-11-24T08:12:10.542014+0000","last_peered":"2023-11-24T08:12:10.542014+0000","last_clean":"2023-11-24T08:12:10.542014+0000","last_became_active":"2023-11-24T08:12:09.529282+0000","last_became_peered":"2023-11-24T08:12:09.529282+0000","last_unstale":"2023-11-24T08:12:10.542014+0000","last_undegraded":"2023-11-24T08:12:10.542014+0000","last_fullsized":"2023-11-24T08:12:10.542014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:52:31.000935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.870951+0000","last_change":"2023-11-24T08:12:10.557209+0000","last_active":"2023-11-24T08:12:11.870951+0000","last_peered":"2023-11-24T08:12:11.870951+0000","last_clean":"2023-11-24T08:12:11.870951+0000","last_became_active":"2023-11-24T08:12:10.556907+0000","last_became_peered":"2023-11-24T08:12:10.556907+0000","last_unstale":"2023-11-24T08:12:11.870951+0000","last_undegraded":"2023-11-24T08:12:11.870951+0000","last_fullsized":"2023-11-24T08:12:11.870951+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:37.174928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"74'8","reported_seq":20,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.575430+0000","last_change":"2023-11-24T08:12:09.530860+0000","last_active":"2023-11-24T08:12:10.575430+0000","last_peered":"2023-11-24T08:12:10.575430+0000","last_clean":"2023-11-24T08:12:10.575430+0000","last_became_active":"2023-11-24T08:12:09.530167+0000","last_became_peered":"2023-11-24T08:12:09.530167+0000","last_unstale":"2023-11-24T08:12:10.575430+0000","last_undegraded":"2023-11-24T08:12:10.575430+0000","last_fullsized":"2023-11-24T08:12:10.575430+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:13.419418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.555077+0000","last_change":"2023-11-24T08:12:10.549074+0000","last_active":"2023-11-24T08:12:11.555077+0000","last_peered":"2023-11-24T08:12:11.555077+0000","last_clean":"2023-11-24T08:12:11.555077+0000","last_became_active":"2023-11-24T08:12:10.548728+0000","last_became_peered":"2023-11-24T08:12:10.548728+0000","last_unstale":"2023-11-24T08:12:11.555077+0000","last_undegraded":"2023-11-24T08:12:11.555077+0000","last_fullsized":"2023-11-24T08:12:11.555077+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:35:38.800185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.570626+0000","last_change":"2023-11-24T08:12:09.538063+0000","last_active":"2023-11-24T08:12:10.570626+0000","last_peered":"2023-11-24T08:12:10.570626+0000","last_clean":"2023-11-24T08:12:10.570626+0000","last_became_active":"2023-11-24T08:12:09.536932+0000","last_became_peered":"2023-11-24T08:12:09.536932+0000","last_unstale":"2023-11-24T08:12:10.570626+0000","last_undegraded":"2023-11-24T08:12:10.570626+0000","last_fullsized":"2023-11-24T08:12:10.570626+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:47:06.361251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873876+0000","last_change":"2023-11-24T08:12:10.560481+0000","last_active":"2023-11-24T08:12:11.873876+0000","last_peered":"2023-11-24T08:12:11.873876+0000","last_clean":"2023-11-24T08:12:11.873876+0000","last_became_active":"2023-11-24T08:12:10.559926+0000","last_became_peered":"2023-11-24T08:12:10.559926+0000","last_unstale":"2023-11-24T08:12:11.873876+0000","last_undegraded":"2023-11-24T08:12:11.873876+0000","last_fullsized":"2023-11-24T08:12:11.873876+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:57:24.445257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.573847+0000","last_change":"2023-11-24T08:12:09.533762+0000","last_active":"2023-11-24T08:12:10.573847+0000","last_peered":"2023-11-24T08:12:10.573847+0000","last_clean":"2023-11-24T08:12:10.573847+0000","last_became_active":"2023-11-24T08:12:09.533465+0000","last_became_peered":"2023-11-24T08:12:09.533465+0000","last_unstale":"2023-11-24T08:12:10.573847+0000","last_undegraded":"2023-11-24T08:12:10.573847+0000","last_fullsized":"2023-11-24T08:12:10.573847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:26:27.517078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.548854+0000","last_change":"2023-11-24T08:12:10.549648+0000","last_active":"2023-11-24T08:12:11.548854+0000","last_peered":"2023-11-24T08:12:11.548854+0000","last_clean":"2023-11-24T08:12:11.548854+0000","last_became_active":"2023-11-24T08:12:10.547708+0000","last_became_peered":"2023-11-24T08:12:10.547708+0000","last_unstale":"2023-11-24T08:12:11.548854+0000","last_undegraded":"2023-11-24T08:12:11.548854+0000","last_fullsized":"2023-11-24T08:12:11.548854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:07:50.317104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"74'1","reported_seq":13,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.573014+0000","last_change":"2023-11-24T08:12:09.530831+0000","last_active":"2023-11-24T08:12:10.573014+0000","last_peered":"2023-11-24T08:12:10.573014+0000","last_clean":"2023-11-24T08:12:10.573014+0000","last_became_active":"2023-11-24T08:12:09.530459+0000","last_became_peered":"2023-11-24T08:12:09.530459+0000","last_unstale":"2023-11-24T08:12:10.573014+0000","last_undegraded":"2023-11-24T08:12:10.573014+0000","last_fullsized":"2023-11-24T08:12:10.573014+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:12:53.972756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872049+0000","last_change":"2023-11-24T08:12:10.555591+0000","last_active":"2023-11-24T08:12:11.872049+0000","last_peered":"2023-11-24T08:12:11.872049+0000","last_clean":"2023-11-24T08:12:11.872049+0000","last_became_active":"2023-11-24T08:12:10.553690+0000","last_became_peered":"2023-11-24T08:12:10.553690+0000","last_unstale":"2023-11-24T08:12:11.872049+0000","last_undegraded":"2023-11-24T08:12:11.872049+0000","last_fullsized":"2023-11-24T08:12:11.872049+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:23:20.015987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.549829+0000","last_change":"2023-11-24T08:12:09.535763+0000","last_active":"2023-11-24T08:12:10.549829+0000","last_peered":"2023-11-24T08:12:10.549829+0000","last_clean":"2023-11-24T08:12:10.549829+0000","last_became_active":"2023-11-24T08:12:09.535048+0000","last_became_peered":"2023-11-24T08:12:09.535048+0000","last_unstale":"2023-11-24T08:12:10.549829+0000","last_undegraded":"2023-11-24T08:12:10.549829+0000","last_fullsized":"2023-11-24T08:12:10.549829+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:36:56.808696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871845+0000","last_change":"2023-11-24T08:12:10.543632+0000","last_active":"2023-11-24T08:12:11.871845+0000","last_peered":"2023-11-24T08:12:11.871845+0000","last_clean":"2023-11-24T08:12:11.871845+0000","last_became_active":"2023-11-24T08:12:10.543324+0000","last_became_peered":"2023-11-24T08:12:10.543324+0000","last_unstale":"2023-11-24T08:12:11.871845+0000","last_undegraded":"2023-11-24T08:12:11.871845+0000","last_fullsized":"2023-11-24T08:12:11.871845+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:05:24.549451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.530368+0000","last_change":"2023-11-24T08:12:09.536935+0000","last_active":"2023-11-24T08:12:10.530368+0000","last_peered":"2023-11-24T08:12:10.530368+0000","last_clean":"2023-11-24T08:12:10.530368+0000","last_became_active":"2023-11-24T08:12:09.535354+0000","last_became_peered":"2023-11-24T08:12:09.535354+0000","last_unstale":"2023-11-24T08:12:10.530368+0000","last_undegraded":"2023-11-24T08:12:10.530368+0000","last_fullsized":"2023-11-24T08:12:10.530368+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:48:22.663814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871268+0000","last_change":"2023-11-24T08:12:10.563210+0000","last_active":"2023-11-24T08:12:11.871268+0000","last_peered":"2023-11-24T08:12:11.871268+0000","last_clean":"2023-11-24T08:12:11.871268+0000","last_became_active":"2023-11-24T08:12:10.562004+0000","last_became_peered":"2023-11-24T08:12:10.562004+0000","last_unstale":"2023-11-24T08:12:11.871268+0000","last_undegraded":"2023-11-24T08:12:11.871268+0000","last_fullsized":"2023-11-24T08:12:11.871268+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:47:47.108050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.544895+0000","last_change":"2023-11-24T08:12:09.530273+0000","last_active":"2023-11-24T08:12:10.544895+0000","last_peered":"2023-11-24T08:12:10.544895+0000","last_clean":"2023-11-24T08:12:10.544895+0000","last_became_active":"2023-11-24T08:12:09.529967+0000","last_became_peered":"2023-11-24T08:12:09.529967+0000","last_unstale":"2023-11-24T08:12:10.544895+0000","last_undegraded":"2023-11-24T08:12:10.544895+0000","last_fullsized":"2023-11-24T08:12:10.544895+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:01:37.818727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872266+0000","last_change":"2023-11-24T08:12:10.552869+0000","last_active":"2023-11-24T08:12:11.872266+0000","last_peered":"2023-11-24T08:12:11.872266+0000","last_clean":"2023-11-24T08:12:11.872266+0000","last_became_active":"2023-11-24T08:12:10.547995+0000","last_became_peered":"2023-11-24T08:12:10.547995+0000","last_unstale":"2023-11-24T08:12:11.872266+0000","last_undegraded":"2023-11-24T08:12:11.872266+0000","last_fullsized":"2023-11-24T08:12:11.872266+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:16:12.826115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.547192+0000","last_change":"2023-11-24T08:12:09.530296+0000","last_active":"2023-11-24T08:12:10.547192+0000","last_peered":"2023-11-24T08:12:10.547192+0000","last_clean":"2023-11-24T08:12:10.547192+0000","last_became_active":"2023-11-24T08:12:09.529840+0000","last_became_peered":"2023-11-24T08:12:09.529840+0000","last_unstale":"2023-11-24T08:12:10.547192+0000","last_undegraded":"2023-11-24T08:12:10.547192+0000","last_fullsized":"2023-11-24T08:12:10.547192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:27:40.788887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.868787+0000","last_change":"2023-11-24T08:12:10.567058+0000","last_active":"2023-11-24T08:12:11.868787+0000","last_peered":"2023-11-24T08:12:11.868787+0000","last_clean":"2023-11-24T08:12:11.868787+0000","last_became_active":"2023-11-24T08:12:10.566453+0000","last_became_peered":"2023-11-24T08:12:10.566453+0000","last_unstale":"2023-11-24T08:12:11.868787+0000","last_undegraded":"2023-11-24T08:12:11.868787+0000","last_fullsized":"2023-11-24T08:12:11.868787+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:15:09.098387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.570690+0000","last_change":"2023-11-24T08:12:09.536994+0000","last_active":"2023-11-24T08:12:10.570690+0000","last_peered":"2023-11-24T08:12:10.570690+0000","last_clean":"2023-11-24T08:12:10.570690+0000","last_became_active":"2023-11-24T08:12:09.536664+0000","last_became_peered":"2023-11-24T08:12:09.536664+0000","last_unstale":"2023-11-24T08:12:10.570690+0000","last_undegraded":"2023-11-24T08:12:10.570690+0000","last_fullsized":"2023-11-24T08:12:10.570690+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:20:08.559029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872479+0000","last_change":"2023-11-24T08:12:10.548541+0000","last_active":"2023-11-24T08:12:11.872479+0000","last_peered":"2023-11-24T08:12:11.872479+0000","last_clean":"2023-11-24T08:12:11.872479+0000","last_became_active":"2023-11-24T08:12:10.547765+0000","last_became_peered":"2023-11-24T08:12:10.547765+0000","last_unstale":"2023-11-24T08:12:11.872479+0000","last_undegraded":"2023-11-24T08:12:11.872479+0000","last_fullsized":"2023-11-24T08:12:11.872479+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:58:16.666601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"75'1","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872679+0000","last_change":"2023-11-24T08:12:09.534163+0000","last_active":"2023-11-24T08:12:11.872679+0000","last_peered":"2023-11-24T08:12:11.872679+0000","last_clean":"2023-11-24T08:12:11.872679+0000","last_became_active":"2023-11-24T08:12:09.533384+0000","last_became_peered":"2023-11-24T08:12:09.533384+0000","last_unstale":"2023-11-24T08:12:11.872679+0000","last_undegraded":"2023-11-24T08:12:11.872679+0000","last_fullsized":"2023-11-24T08:12:11.872679+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:40:39.337647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.872710+0000","last_change":"2023-11-24T08:12:10.567951+0000","last_active":"2023-11-24T08:12:11.872710+0000","last_peered":"2023-11-24T08:12:11.872710+0000","last_clean":"2023-11-24T08:12:11.872710+0000","last_became_active":"2023-11-24T08:12:10.567670+0000","last_became_peered":"2023-11-24T08:12:10.567670+0000","last_unstale":"2023-11-24T08:12:11.872710+0000","last_undegraded":"2023-11-24T08:12:11.872710+0000","last_fullsized":"2023-11-24T08:12:11.872710+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:59:43.811170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.545377+0000","last_change":"2023-11-24T08:12:09.524480+0000","last_active":"2023-11-24T08:12:10.545377+0000","last_peered":"2023-11-24T08:12:10.545377+0000","last_clean":"2023-11-24T08:12:10.545377+0000","last_became_active":"2023-11-24T08:12:09.524212+0000","last_became_peered":"2023-11-24T08:12:09.524212+0000","last_unstale":"2023-11-24T08:12:10.545377+0000","last_undegraded":"2023-11-24T08:12:10.545377+0000","last_fullsized":"2023-11-24T08:12:10.545377+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:17.346075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.873881+0000","last_change":"2023-11-24T08:12:10.559962+0000","last_active":"2023-11-24T08:12:11.873881+0000","last_peered":"2023-11-24T08:12:11.873881+0000","last_clean":"2023-11-24T08:12:11.873881+0000","last_became_active":"2023-11-24T08:12:10.559660+0000","last_became_peered":"2023-11-24T08:12:10.559660+0000","last_unstale":"2023-11-24T08:12:11.873881+0000","last_undegraded":"2023-11-24T08:12:11.873881+0000","last_fullsized":"2023-11-24T08:12:11.873881+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:02:24.907859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.552539+0000","last_change":"2023-11-24T08:12:09.530037+0000","last_active":"2023-11-24T08:12:10.552539+0000","last_peered":"2023-11-24T08:12:10.552539+0000","last_clean":"2023-11-24T08:12:10.552539+0000","last_became_active":"2023-11-24T08:12:09.529803+0000","last_became_peered":"2023-11-24T08:12:09.529803+0000","last_unstale":"2023-11-24T08:12:10.552539+0000","last_undegraded":"2023-11-24T08:12:10.552539+0000","last_fullsized":"2023-11-24T08:12:10.552539+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:08:00.033297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.870607+0000","last_change":"2023-11-24T08:12:10.552530+0000","last_active":"2023-11-24T08:12:11.870607+0000","last_peered":"2023-11-24T08:12:11.870607+0000","last_clean":"2023-11-24T08:12:11.870607+0000","last_became_active":"2023-11-24T08:12:10.552147+0000","last_became_peered":"2023-11-24T08:12:10.552147+0000","last_unstale":"2023-11-24T08:12:11.870607+0000","last_undegraded":"2023-11-24T08:12:11.870607+0000","last_fullsized":"2023-11-24T08:12:11.870607+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:17:09.900754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.544569+0000","last_change":"2023-11-24T08:12:09.522451+0000","last_active":"2023-11-24T08:12:10.544569+0000","last_peered":"2023-11-24T08:12:10.544569+0000","last_clean":"2023-11-24T08:12:10.544569+0000","last_became_active":"2023-11-24T08:12:09.521531+0000","last_became_peered":"2023-11-24T08:12:09.521531+0000","last_unstale":"2023-11-24T08:12:10.544569+0000","last_undegraded":"2023-11-24T08:12:10.544569+0000","last_fullsized":"2023-11-24T08:12:10.544569+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:28:51.436372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.553421+0000","last_change":"2023-11-24T08:12:10.556747+0000","last_active":"2023-11-24T08:12:11.553421+0000","last_peered":"2023-11-24T08:12:11.553421+0000","last_clean":"2023-11-24T08:12:11.553421+0000","last_became_active":"2023-11-24T08:12:10.556527+0000","last_became_peered":"2023-11-24T08:12:10.556527+0000","last_unstale":"2023-11-24T08:12:11.553421+0000","last_undegraded":"2023-11-24T08:12:11.553421+0000","last_fullsized":"2023-11-24T08:12:11.553421+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:03:19.996340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.521132+0000","last_change":"2023-11-24T08:12:09.521396+0000","last_active":"2023-11-24T08:12:10.521132+0000","last_peered":"2023-11-24T08:12:10.521132+0000","last_clean":"2023-11-24T08:12:10.521132+0000","last_became_active":"2023-11-24T08:12:09.520385+0000","last_became_peered":"2023-11-24T08:12:09.520385+0000","last_unstale":"2023-11-24T08:12:10.521132+0000","last_undegraded":"2023-11-24T08:12:10.521132+0000","last_fullsized":"2023-11-24T08:12:10.521132+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:23:04.461254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.548760+0000","last_change":"2023-11-24T08:12:10.539946+0000","last_active":"2023-11-24T08:12:11.548760+0000","last_peered":"2023-11-24T08:12:11.548760+0000","last_clean":"2023-11-24T08:12:11.548760+0000","last_became_active":"2023-11-24T08:12:10.539589+0000","last_became_peered":"2023-11-24T08:12:10.539589+0000","last_unstale":"2023-11-24T08:12:11.548760+0000","last_undegraded":"2023-11-24T08:12:11.548760+0000","last_fullsized":"2023-11-24T08:12:11.548760+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:02:32.601987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.536948+0000","last_change":"2023-11-24T08:12:09.529444+0000","last_active":"2023-11-24T08:12:10.536948+0000","last_peered":"2023-11-24T08:12:10.536948+0000","last_clean":"2023-11-24T08:12:10.536948+0000","last_became_active":"2023-11-24T08:12:09.528128+0000","last_became_peered":"2023-11-24T08:12:09.528128+0000","last_unstale":"2023-11-24T08:12:10.536948+0000","last_undegraded":"2023-11-24T08:12:10.536948+0000","last_fullsized":"2023-11-24T08:12:10.536948+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:56:11.013386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.552478+0000","last_change":"2023-11-24T08:12:10.564428+0000","last_active":"2023-11-24T08:12:11.552478+0000","last_peered":"2023-11-24T08:12:11.552478+0000","last_clean":"2023-11-24T08:12:11.552478+0000","last_became_active":"2023-11-24T08:12:10.563926+0000","last_became_peered":"2023-11-24T08:12:10.563926+0000","last_unstale":"2023-11-24T08:12:11.552478+0000","last_undegraded":"2023-11-24T08:12:11.552478+0000","last_fullsized":"2023-11-24T08:12:11.552478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:11:14.430047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.526806+0000","last_change":"2023-11-24T08:12:09.535030+0000","last_active":"2023-11-24T08:12:10.526806+0000","last_peered":"2023-11-24T08:12:10.526806+0000","last_clean":"2023-11-24T08:12:10.526806+0000","last_became_active":"2023-11-24T08:12:09.530814+0000","last_became_peered":"2023-11-24T08:12:09.530814+0000","last_unstale":"2023-11-24T08:12:10.526806+0000","last_undegraded":"2023-11-24T08:12:10.526806+0000","last_fullsized":"2023-11-24T08:12:10.526806+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:29:05.991585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-11-24T08:12:11.871895+0000","last_change":"2023-11-24T08:12:10.547515+0000","last_active":"2023-11-24T08:12:11.871895+0000","last_peered":"2023-11-24T08:12:11.871895+0000","last_clean":"2023-11-24T08:12:11.871895+0000","last_became_active":"2023-11-24T08:12:10.547222+0000","last_became_peered":"2023-11-24T08:12:10.547222+0000","last_unstale":"2023-11-24T08:12:11.871895+0000","last_undegraded":"2023-11-24T08:12:11.871895+0000","last_fullsized":"2023-11-24T08:12:11.871895+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:09.487651+0000","last_clean_scrub_stamp":"2023-11-24T08:12:09.487651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:57:21.836948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"74'2","reported_seq":14,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.569754+0000","last_change":"2023-11-24T08:12:09.535433+0000","last_active":"2023-11-24T08:12:10.569754+0000","last_peered":"2023-11-24T08:12:10.569754+0000","last_clean":"2023-11-24T08:12:10.569754+0000","last_became_active":"2023-11-24T08:12:09.534941+0000","last_became_peered":"2023-11-24T08:12:09.534941+0000","last_unstale":"2023-11-24T08:12:10.569754+0000","last_undegraded":"2023-11-24T08:12:10.569754+0000","last_fullsized":"2023-11-24T08:12:10.569754+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:08.477022+0000","last_clean_scrub_stamp":"2023-11-24T08:12:08.477022+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:15:31.532846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":207,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-11-24T08:12:10.542859+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:10.542859+0000","last_peered":"2023-11-24T08:12:10.542859+0000","last_clean":"2023-11-24T08:12:10.542859+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:10.542859+0000","last_undegraded":"2023-11-24T08:12:10.542859+0000","last_fullsized":"2023-11-24T08:12:10.542859+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32656,"kb_used_data":1476,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339184,"statfs":{"total":96636764160,"available":96603324416,"internally_reserved":0,"allocated":1511424,"data_stored":976037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98799999999999999,"5min":1.014,"15min":1.012},"min":{"1min":0.64300000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.456,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.411},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.03,"15min":0.998},"min":{"1min":0.67800000000000005,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.5329999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.117}]},{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.1299999999999999,"15min":1.125},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.5149999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1739999999999999,"15min":1.1659999999999999},"min":{"1min":0.85199999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.454,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.3620000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.1120000000000001,"15min":1.0660000000000001},"min":{"1min":0.68000000000000005,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.534,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.214,"15min":1.206},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.631,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.1739999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.163,"5min":1.246,"15min":1.236},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.3959999999999999,"5min":3.387,"15min":3.387},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.246,"15min":1.22},"min":{"1min":0.90500000000000003,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.583,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.069}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.234,"15min":1.212},"min":{"1min":0.97399999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.448,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.226},{"interface":"front","average":{"1min":1.341,"5min":1.391,"15min":1.3759999999999999},"min":{"1min":0.96599999999999997,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.6259999999999999,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.6240000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.4039999999999999,"15min":1.427},"min":{"1min":0.89800000000000002,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6399999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.256},{"interface":"front","average":{"1min":1.391,"5min":1.379,"15min":1.3580000000000001},"min":{"1min":1.129,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.698,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.456}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.5069999999999999,"15min":1.4830000000000001},"min":{"1min":1.3759999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.806,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.585},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.492,"15min":1.4790000000000001},"min":{"1min":1.1100000000000001,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.728,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.496}]}]},{"osd":6,"up_from":14,"seq":60129542191,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32084,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339756,"statfs":{"total":96636764160,"available":96603910144,"internally_reserved":0,"allocated":917504,"data_stored":385145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":1.0780000000000001,"15min":1.095},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.472,"5min":2.085,"15min":2.085},"last":1.304},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.214,"15min":1.2370000000000001},"min":{"1min":0.66700000000000004,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4850000000000001,"5min":1.794,"15min":1.794},"last":1.218}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.70999999999999996,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.6599999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.1919999999999999,"5min":1.1699999999999999,"15min":1.171},"min":{"1min":0.80400000000000005,"5min":0.749,"15min":0.749},"max":{"1min":1.7370000000000001,"5min":1.98,"15min":1.98},"last":0.94299999999999995}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.248,"5min":1.2250000000000001,"15min":1.2210000000000001},"min":{"1min":0.84899999999999998,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6950000000000001,"5min":1.778,"15min":1.778},"last":1.1619999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.1659999999999999,"15min":1.1930000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6899999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.3839999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.3009999999999999,"15min":1.343},"min":{"1min":0.72499999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7849999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.508},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.446},"min":{"1min":0.98299999999999998,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.847,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.0640000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.343,"15min":1.369},"min":{"1min":0.77600000000000002,"5min":0.622,"15min":0.622},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5309999999999999,"5min":1.556,"15min":1.587},"min":{"1min":1.242,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.4359999999999999,"5min":2.661,"15min":2.661},"last":1.7090000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.625,"15min":1.708},"min":{"1min":1.151,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":1.776,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.345},{"interface":"front","average":{"1min":1.456,"5min":1.5900000000000001,"15min":1.651},"min":{"1min":1.0249999999999999,"5min":0.505,"15min":0.505},"max":{"1min":1.921,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.256}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.581,"5min":1.726,"15min":1.78},"min":{"1min":1.2949999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.016,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.8480000000000001},{"interface":"front","average":{"1min":1.605,"5min":1.7250000000000001,"15min":1.7849999999999999},"min":{"1min":1.2050000000000001,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.1499999999999999,"5min":2.657,"15min":2.657},"last":1.472}]}]},{"osd":5,"up_from":14,"seq":60129542190,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":913408,"data_stored":385143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0529999999999999,"15min":1.0249999999999999},"min":{"1min":0.72399999999999998,"5min":0.627,"15min":0.627},"max":{"1min":1.6100000000000001,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.133,"15min":1.157},"min":{"1min":0.77700000000000002,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.6850000000000001,"5min":1.837,"15min":1.837},"last":0.751}]},{"osd":1,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.121,"15min":1.1160000000000001},"min":{"1min":0.69099999999999995,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":1.302},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1890000000000001,"15min":1.198},"min":{"1min":0.82799999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.4850000000000001,"5min":1.722,"15min":1.722},"last":1.0669999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.2649999999999999,"15min":1.3129999999999999},"min":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.4970000000000001,"5min":2.085,"15min":2.085},"last":1.0049999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.3320000000000001,"15min":1.3620000000000001},"min":{"1min":0.84499999999999997,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.478,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.413}]},{"osd":3,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.399,"15min":1.4339999999999999},"min":{"1min":0.95399999999999996,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.573,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.1659999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.468,"15min":1.5169999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.7130000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.5109999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4490000000000001,"15min":1.4590000000000001},"min":{"1min":1.1259999999999999,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.8500000000000001,"5min":3.577,"15min":3.577},"last":1.23},{"interface":"front","average":{"1min":1.49,"5min":1.645,"15min":1.7030000000000001},"min":{"1min":1.0620000000000001,"5min":0.92600000000000005,"15min":0.92600000000000005},"max":{"1min":1.958,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.3560000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5780000000000001,"15min":1.6060000000000001},"min":{"1min":1.0209999999999999,"5min":0.64200000000000002,"15min":0.64200000000000002},"max":{"1min":1.869,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.554,"5min":1.5489999999999999,"15min":1.5409999999999999},"min":{"1min":1.0089999999999999,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":2.0030000000000001,"5min":2.512,"15min":2.512},"last":1.448}]},{"osd":7,"last update":"Fri Nov 24 08:11:45 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.6839999999999999,"15min":1.716},"min":{"1min":1.21,"5min":0.96699999999999997,"15min":0.96699999999999997},"max":{"1min":1.8839999999999999,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.708,"15min":1.7370000000000001},"min":{"1min":1.2949999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.9390000000000001,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.5720000000000001}]}]},{"osd":4,"up_from":14,"seq":60129542191,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339760,"statfs":{"total":96636764160,"available":96603914240,"internally_reserved":0,"allocated":913408,"data_stored":385123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.01,"15min":0.98699999999999999},"min":{"1min":0.64700000000000002,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.6240000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.0449999999999999},{"interface":"front","average":{"1min":1.145,"5min":1.099,"15min":1.093},"min":{"1min":0.71999999999999997,"5min":0.626,"15min":0.626},"max":{"1min":2.0099999999999998,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.3700000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1459999999999999},"min":{"1min":0.63700000000000001,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6140000000000001,"5min":1.859,"15min":1.859},"last":1.4079999999999999},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.1919999999999999,"15min":1.1639999999999999},"min":{"1min":0.79400000000000004,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.1230000000000002,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.99099999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2689999999999999,"15min":1.2749999999999999},"min":{"1min":0.81899999999999995,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.7,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.115,"5min":1.099,"15min":1.105},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.8879999999999999,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.278,"15min":1.2889999999999999},"min":{"1min":0.78900000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.659,"5min":2.25,"15min":2.25},"last":1.155},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.86599999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8360000000000001,"5min":2.746,"15min":2.746},"last":1.2370000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.298,"15min":1.3180000000000001},"min":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.2},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.452,"15min":1.458},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.8440000000000001,"5min":2.4209999999999998,"15min":2.4209999999999998},"last":1.5680000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4259999999999999,"15min":1.446},"min":{"1min":0.73799999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.4809999999999999,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.306},{"interface":"front","average":{"1min":1.47,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.628,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.8500000000000001,"5min":2.641,"15min":2.641},"last":1.806}]},{"osd":7,"last update":"Fri Nov 24 08:11:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5,"5min":1.5149999999999999,"15min":1.5129999999999999},"min":{"1min":0.69599999999999995,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.2269999999999999,"5min":2.548,"15min":2.548},"last":1.8779999999999999},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.534,"15min":1.5489999999999999},"min":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.8359999999999999,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.615}]}]},{"osd":3,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32076,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339764,"statfs":{"total":96636764160,"available":96603918336,"internally_reserved":0,"allocated":917504,"data_stored":385165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0489999999999999,"15min":1.145},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3180000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0589999999999999,"15min":1.143},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.085,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.96099999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.95899999999999996,"15min":0.995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4630000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.534,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.1180000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.1559999999999999,"15min":1.22},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5269999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.1839999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.385,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.631,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.337,"15min":1.373},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4670000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.0660000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.5069999999999999,"15min":1.5680000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.486,"15min":1.536},"min":{"1min":0.872,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8700000000000001,"5min":3.339,"15min":3.339},"last":1.2410000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.5609999999999999,"15min":1.6659999999999999},"min":{"1min":0.746,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.742,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.151},{"interface":"front","average":{"1min":1.379,"5min":1.522,"15min":1.5580000000000001},"min":{"1min":1.0249999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.681,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.222}]},{"osd":7,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.605,"15min":1.675},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.603,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.355,"5min":1.8320000000000001,"15min":2.016},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7629999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.208}]}]},{"osd":2,"up_from":14,"seq":60129542191,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343920,"statfs":{"total":96636764160,"available":96608174080,"internally_reserved":0,"allocated":921600,"data_stored":386665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.75700000000000001,"5min":0.77300000000000002,"15min":0.76500000000000001},"min":{"1min":0.47899999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.21,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.81699999999999995,"15min":0.77700000000000002},"min":{"1min":0.435,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6950000000000001,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.98199999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.85899999999999999,"15min":0.873},"min":{"1min":0.41499999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.244,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.95299999999999996,"15min":0.92800000000000005},"min":{"1min":0.58999999999999997,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.514,"5min":1.671,"15min":1.671},"last":1.3}]},{"osd":3,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.98599999999999999,"15min":0.97399999999999998},"min":{"1min":0.56799999999999995,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.387,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.363},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.98499999999999999,"15min":1.0009999999999999},"min":{"1min":0.53200000000000003,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.232,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.1339999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2130000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.81499999999999995,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":1.6679999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.3939999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.23,"15min":1.224},"min":{"1min":0.876,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":1.623}]},{"osd":5,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1899999999999999,"5min":1.3620000000000001,"15min":1.431},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.629,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.266},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.294,"15min":1.286},"min":{"1min":0.872,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.839,"5min":1.839,"15min":1.839},"last":1.5069999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3839999999999999,"15min":1.4159999999999999},"min":{"1min":0.94799999999999995,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.895,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.3260000000000001,"15min":1.337},"min":{"1min":0.93500000000000005,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9490000000000001,"5min":1.958,"15min":1.958},"last":1.3320000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3480000000000001,"5min":1.4339999999999999,"15min":1.4770000000000001},"min":{"1min":0.91000000000000003,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.99,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4239999999999999,"15min":1.4419999999999999},"min":{"1min":0.87,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":2.0739999999999998,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.8200000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542191,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339744,"statfs":{"total":96636764160,"available":96603897856,"internally_reserved":0,"allocated":929792,"data_stored":387281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.83099999999999996,"15min":0.83699999999999997},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.86299999999999999,"15min":0.88300000000000001},"min":{"1min":0.373,"5min":0.37,"15min":0.37},"max":{"1min":1.145,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.1879999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.85199999999999998,"15min":0.85299999999999998},"min":{"1min":0.504,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1359999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.0229999999999999,"5min":1.024,"15min":1.0109999999999999},"min":{"1min":0.59899999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.391,"5min":1.621,"15min":1.621},"last":0.97699999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.143,"15min":1.1579999999999999},"min":{"1min":0.66100000000000003,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.534,"5min":1.907,"15min":1.907},"last":1.143},{"interface":"front","average":{"1min":0.90800000000000003,"5min":1.018,"15min":1.046},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3740000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.458}]},{"osd":4,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.161,"15min":1.163},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":1.425,"5min":1.9199999999999999,"15min":1.9199999999999999},"last":1.083},{"interface":"front","average":{"1min":1.2649999999999999,"5min":1.3240000000000001,"15min":1.3300000000000001},"min":{"1min":0.874,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.6919999999999999,"5min":2.0920000000000001,"15min":2.0920000000000001},"last":1.347}]},{"osd":5,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.2669999999999999,"15min":1.3029999999999999},"min":{"1min":0.83399999999999996,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.653,"5min":2.661,"15min":2.661},"last":1.5660000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.3120000000000001,"15min":1.3360000000000001},"min":{"1min":0.72999999999999998,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.514,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.4179999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.3480000000000001,"15min":1.3580000000000001},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.788,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.633},{"interface":"front","average":{"1min":1.3640000000000001,"5min":1.343,"15min":1.335},"min":{"1min":0.88800000000000001,"5min":0.81499999999999995,"15min":0.81499999999999995},"max":{"1min":1.7070000000000001,"5min":2.105,"15min":2.105},"last":1.605}]},{"osd":7,"last update":"Fri Nov 24 08:11:43 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4099999999999999,"15min":1.4039999999999999},"min":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"max":{"1min":1.7669999999999999,"5min":2.1400000000000001,"15min":2.1400000000000001},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3600000000000001,"15min":1.3660000000000001},"min":{"1min":0.88300000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8600000000000001,"5min":2.008,"15min":2.008},"last":1.6910000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1503232,"data_stored":975421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.80100000000000005,"15min":0.80300000000000005},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.1739999999999999,"5min":1.403,"15min":1.403},"last":0.92700000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.94799999999999995,"15min":0.95399999999999996},"min":{"1min":0.5,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.2989999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.337}]},{"osd":2,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.86499999999999999,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.47199999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.0569999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.91900000000000004,"15min":0.93100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.5209999999999999,"5min":1.986,"15min":1.986},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.0489999999999999,"15min":1.0720000000000001},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.3380000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.91600000000000004,"15min":0.96099999999999997},"min":{"1min":0.42899999999999999,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.23,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.1579999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.165,"15min":1.171},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.641,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.2869999999999999,"15min":1.29},"min":{"1min":0.88300000000000001,"5min":0.81100000000000005,"15min":0.81100000000000005},"max":{"1min":1.78,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.5880000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.202,"15min":1.22},"min":{"1min":0.82099999999999995,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.4550000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.429},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.2569999999999999,"15min":1.268},"min":{"1min":0.80100000000000005,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.8220000000000001,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.5}]},{"osd":6,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.2669999999999999,"15min":1.272},"min":{"1min":0.81299999999999994,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.546,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.2709999999999999,"5min":1.284,"15min":1.286},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.742,"5min":2.149,"15min":2.149},"last":1.298}]},{"osd":7,"last update":"Fri Nov 24 08:11:42 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.3979999999999999,"15min":1.4319999999999999},"min":{"1min":0.83599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.865,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.3839999999999999,"15min":1.4119999999999999},"min":{"1min":0.78400000000000003,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.603,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.5369999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:12:23.973 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:12:23.973 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:12:24.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.231+0000 7f433bad4640 1 Processor -- start 2023-11-24T08:12:24.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.232+0000 7f433bad4640 1 -- start start 2023-11-24T08:12:24.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.233+0000 7f433bad4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.233+0000 7f433bad4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43340d3870 0x7f43340b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.233+0000 7f433bad4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f43340b1400 0x7f43340b1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.233+0000 7f433bad4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f433415db00 con 0x7f43340d3870 2023-11-24T08:12:24.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.233+0000 7f433bad4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43340d3cf0 con 0x7f43340b1400 2023-11-24T08:12:24.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.233+0000 7f433bad4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43340d0ce0 con 0x7f433415d720 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43340d3870 0x7f43340b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43340d3870 0x7f43340b0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46354/0 (socket says 172.21.15.106:46354) 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 -- 172.21.15.106:0/3108565532 learned_addr learned my addr 172.21.15.106:0/3108565532 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f4339870640 1 --2- 172.21.15.106:0/3108565532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433a071640 1 --2- 172.21.15.106:0/3108565532 >> v2:172.21.15.156:3300/0 conn(0x7f43340b1400 0x7f43340b1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 -- 172.21.15.106:0/3108565532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 msgr2=0x7f43340d3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:24.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 --2- 172.21.15.106:0/3108565532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340d3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 -- 172.21.15.106:0/3108565532 >> v2:172.21.15.156:3300/0 conn(0x7f43340b1400 msgr2=0x7f43340b1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:24.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 --2- 172.21.15.106:0/3108565532 >> v2:172.21.15.156:3300/0 conn(0x7f43340b1400 0x7f43340b1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433906f640 1 -- 172.21.15.106:0/3108565532 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f433406d400 con 0x7f43340d3870 2023-11-24T08:12:24.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.234+0000 7f433a071640 1 --2- 172.21.15.106:0/3108565532 >> v2:172.21.15.156:3300/0 conn(0x7f43340b1400 0x7f43340b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.235+0000 7f433906f640 1 --2- 172.21.15.106:0/3108565532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340b0ec0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f433408c760 tx=0x7f432802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ebca3c7aa577377 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433886e640 1 -- 172.21.15.106:0/3108565532 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43280317d0 con 0x7f43340d3870 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433886e640 1 -- 172.21.15.106:0/3108565532 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4328031930 con 0x7f43340d3870 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433886e640 1 -- 172.21.15.106:0/3108565532 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328031cc0 con 0x7f43340d3870 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 -- 172.21.15.106:0/3108565532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 msgr2=0x7f43340b0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 --2- 172.21.15.106:0/3108565532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340b0ec0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f433408c760 tx=0x7f432802b870 comp rx=0 tx=0).stop 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 -- 172.21.15.106:0/3108565532 shutdown_connections 2023-11-24T08:12:24.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 --2- 172.21.15.106:0/3108565532 >> v2:172.21.15.156:3300/0 conn(0x7f43340b1400 0x7f43340b1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 --2- 172.21.15.106:0/3108565532 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340b0ec0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 --2- 172.21.15.106:0/3108565532 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340d3330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 -- 172.21.15.106:0/3108565532 >> 172.21.15.106:0/3108565532 conn(0x7f433408af60 msgr2=0x7f433409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:24.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.236+0000 7f433bad4640 1 -- 172.21.15.106:0/3108565532 shutdown_connections 2023-11-24T08:12:24.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.237+0000 7f433bad4640 1 -- 172.21.15.106:0/3108565532 wait complete. 2023-11-24T08:12:24.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.238+0000 7f433bad4640 1 Processor -- start 2023-11-24T08:12:24.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.238+0000 7f433bad4640 1 -- start start 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.239+0000 7f433bad4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340b1400 0x7f433414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.239+0000 7f433bad4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.239+0000 7f433bad4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340caa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.239+0000 7f433bad4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4334149ee0 con 0x7f43340d3870 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.239+0000 7f433bad4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433415fcb0 con 0x7f43340b1400 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.239+0000 7f433bad4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433415ff90 con 0x7f433415d720 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46358/0 (socket says 172.21.15.106:46358) 2023-11-24T08:12:24.241 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 -- 172.21.15.106:0/1846887979 learned_addr learned my addr 172.21.15.106:0/1846887979 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433a071640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f4339870640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340b1400 0x7f433414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 -- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 msgr2=0x7f43340caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 -- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340b1400 msgr2=0x7f433414a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340b1400 0x7f433414a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f433906f640 1 -- 172.21.15.106:0/1846887979 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280094d0 con 0x7f43340d3870 2023-11-24T08:12:24.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.240+0000 7f4339870640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340b1400 0x7f433414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.241+0000 7f433906f640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340ca4f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f432802b8b0 tx=0x7f43280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.241+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328032870 con 0x7f43340d3870 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.241+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43280329f0 con 0x7f43340d3870 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.241+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328032da0 con 0x7f43340d3870 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.241+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4334160270 con 0x7f43340d3870 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.241+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4334147b60 con 0x7f43340d3870 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.242+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43341600d0 con 0x7f43340d3870 2023-11-24T08:12:24.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.243+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f4328030070 con 0x7f43340d3870 2023-11-24T08:12:24.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.244+0000 7f43327fc640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f432006b260 0x7f432006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.244+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f43280abef0 con 0x7f43340d3870 2023-11-24T08:12:24.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.244+0000 7f4339870640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f432006b260 0x7f432006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.245+0000 7f4339870640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f432006b260 0x7f432006d720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4324009f70 tx=0x7f432400b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:24.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.248+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43341600d0 con 0x7f43340d3870 2023-11-24T08:12:24.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.507+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f43341600d0 con 0x7f43340d3870 2023-11-24T08:12:24.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.511+0000 7f43327fc640 1 -- 172.21.15.106:0/1846887979 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7f43341600d0 con 0x7f43340d3870 2023-11-24T08:12:24.513 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2023-11-24T08:12:24.514 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:12:24.514 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2023-11-24T08:12:24.514 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2023-11-24T08:12:24.514 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:12:24.515 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.515+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f432006b260 msgr2=0x7f432006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.515+0000 7f433bad4640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f432006b260 0x7f432006d720 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4324009f70 tx=0x7f432400b040 comp rx=0 tx=0).stop 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.515+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 msgr2=0x7f43340ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.515+0000 7f433bad4640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340ca4f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f432802b8b0 tx=0x7f43280046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 shutdown_connections 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f432006b260 0x7f432006d720 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f433415d720 0x7f43340caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43340d3870 0x7f43340ca4f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 --2- 172.21.15.106:0/1846887979 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340b1400 0x7f433414a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 >> 172.21.15.106:0/1846887979 conn(0x7f433408af60 msgr2=0x7f433406c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:24.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 shutdown_connections 2023-11-24T08:12:24.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.516+0000 7f433bad4640 1 -- 172.21.15.106:0/1846887979 wait complete. 2023-11-24T08:12:24.535 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.745+0000 7fe81acb4640 1 Processor -- start 2023-11-24T08:12:24.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.746+0000 7fe81acb4640 1 -- start start 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe81acb4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe81415d4e0 0x7fe8140d1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe81acb4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8140d38e0 0x7fe8140cfb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe81acb4640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe8140d0070 0x7fe8140d0da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe81acb4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8140d4110 con 0x7fe81415d4e0 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe81acb4640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8140d04e0 con 0x7fe8140d0070 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe81acb4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8140ce990 con 0x7fe8140d38e0 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe819cb2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe81415d4e0 0x7fe8140d1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe819cb2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe81415d4e0 0x7fe8140d1e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46370/0 (socket says 172.21.15.106:46370) 2023-11-24T08:12:24.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.747+0000 7fe819cb2640 1 -- 172.21.15.106:0/4038223515 learned_addr learned my addr 172.21.15.106:0/4038223515 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe81a4b3640 1 --2- 172.21.15.106:0/4038223515 >> v2:172.21.15.156:3300/0 conn(0x7fe8140d0070 0x7fe8140d0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe8194b1640 1 --2- 172.21.15.106:0/4038223515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8140d38e0 0x7fe8140cfb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe819cb2640 1 -- 172.21.15.106:0/4038223515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8140d38e0 msgr2=0x7fe8140cfb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe819cb2640 1 --2- 172.21.15.106:0/4038223515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8140d38e0 0x7fe8140cfb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe819cb2640 1 -- 172.21.15.106:0/4038223515 >> v2:172.21.15.156:3300/0 conn(0x7fe8140d0070 msgr2=0x7fe8140d0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe819cb2640 1 --2- 172.21.15.106:0/4038223515 >> v2:172.21.15.156:3300/0 conn(0x7fe8140d0070 0x7fe8140d0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe819cb2640 1 -- 172.21.15.106:0/4038223515 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8140cec70 con 0x7fe81415d4e0 2023-11-24T08:12:24.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.748+0000 7fe81a4b3640 1 --2- 172.21.15.106:0/4038223515 >> v2:172.21.15.156:3300/0 conn(0x7fe8140d0070 0x7fe8140d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.749+0000 7fe819cb2640 1 --2- 172.21.15.106:0/4038223515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415d4e0 0x7fe8140d1e20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe8140b2a00 tx=0x7fe80802b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a170e3f36f6b466 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe818cb0640 1 -- 172.21.15.106:0/4038223515 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808031860 con 0x7fe81415d4e0 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe818cb0640 1 -- 172.21.15.106:0/4038223515 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8080319c0 con 0x7fe81415d4e0 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe818cb0640 1 -- 172.21.15.106:0/4038223515 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe808031d50 con 0x7fe81415d4e0 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 -- 172.21.15.106:0/4038223515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415d4e0 msgr2=0x7fe8140d1e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 --2- 172.21.15.106:0/4038223515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415d4e0 0x7fe8140d1e20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fe8140b2a00 tx=0x7fe80802b900 comp rx=0 tx=0).stop 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 -- 172.21.15.106:0/4038223515 shutdown_connections 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 --2- 172.21.15.106:0/4038223515 >> v2:172.21.15.156:3300/0 conn(0x7fe8140d0070 0x7fe8140d0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 --2- 172.21.15.106:0/4038223515 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe8140d38e0 0x7fe8140cfb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 --2- 172.21.15.106:0/4038223515 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe81415d4e0 0x7fe8140d1e20 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 -- 172.21.15.106:0/4038223515 >> 172.21.15.106:0/4038223515 conn(0x7fe81409c510 msgr2=0x7fe8140d31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:24.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 -- 172.21.15.106:0/4038223515 shutdown_connections 2023-11-24T08:12:24.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.750+0000 7fe81acb4640 1 -- 172.21.15.106:0/4038223515 wait complete. 2023-11-24T08:12:24.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.752+0000 7fe81acb4640 1 Processor -- start 2023-11-24T08:12:24.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.752+0000 7fe81acb4640 1 -- start start 2023-11-24T08:12:24.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe81acb4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8140d0070 0x7fe8141526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe81acb4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 0x7fe8140b7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe81acb4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe81415d4e0 0x7fe81415ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe81acb4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415f240 con 0x7fe8140d38e0 2023-11-24T08:12:24.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe81acb4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415f520 con 0x7fe8140d0070 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe81acb4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81415f800 con 0x7fe81415d4e0 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe8194b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 0x7fe8140b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe8194b1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 0x7fe8140b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46384/0 (socket says 172.21.15.106:46384) 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.753+0000 7fe8194b1640 1 -- 172.21.15.106:0/2415303447 learned_addr learned my addr 172.21.15.106:0/2415303447 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8194b1640 1 -- 172.21.15.106:0/2415303447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe81415d4e0 msgr2=0x7fe81415ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8194b1640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe81415d4e0 0x7fe81415ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8194b1640 1 -- 172.21.15.106:0/2415303447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8140d0070 msgr2=0x7fe8141526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8194b1640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8140d0070 0x7fe8141526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8194b1640 1 -- 172.21.15.106:0/2415303447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8080094d0 con 0x7fe8140d38e0 2023-11-24T08:12:24.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8194b1640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 0x7fe8140b7800 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ce10 tx=0x7fe81000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:24.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810003b90 con 0x7fe8140d38e0 2023-11-24T08:12:24.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe810003cf0 con 0x7fe8140d38e0 2023-11-24T08:12:24.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe810003ea0 con 0x7fe8140d38e0 2023-11-24T08:12:24.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.754+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81415fae0 con 0x7fe8140d38e0 2023-11-24T08:12:24.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.755+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe814152c10 con 0x7fe8140d38e0 2023-11-24T08:12:24.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.756+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe814160000 con 0x7fe8140d38e0 2023-11-24T08:12:24.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.757+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fe810004240 con 0x7fe8140d38e0 2023-11-24T08:12:24.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.758+0000 7fe8067fc640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe7f806b260 0x7fe7f806d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:24.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.758+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe810012070 con 0x7fe8140d38e0 2023-11-24T08:12:24.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.758+0000 7fe819cb2640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe7f806b260 0x7fe7f806d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:24.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.759+0000 7fe819cb2640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe7f806b260 0x7fe7f806d720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe808036000 tx=0x7fe808030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:24.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.762+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe814160000 con 0x7fe8140d38e0 2023-11-24T08:12:24.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.987+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe814160000 con 0x7fe8140d38e0 2023-11-24T08:12:24.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.988+0000 7fe8067fc640 1 -- 172.21.15.106:0/2415303447 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+4927 (secure 0 0 0) 0x7fe814160000 con 0x7fe8140d38e0 2023-11-24T08:12:24.990 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:24.990 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":43,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:09.955346+0000","modified":"2023-11-24T08:12:13.029327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/4067546353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4067546353},{"type":"v1","addr":"172.21.15.106:6837","nonce":4067546353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:12:24.990 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 43 2023-11-24T08:12:24.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.990+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe7f806b260 msgr2=0x7fe7f806d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.990+0000 7fe81acb4640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe7f806b260 0x7fe7f806d720 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe808036000 tx=0x7fe808030000 comp rx=0 tx=0).stop 2023-11-24T08:12:24.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.990+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 msgr2=0x7fe8140b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:24.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.990+0000 7fe81acb4640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 0x7fe8140b7800 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ce10 tx=0x7fe81000a660 comp rx=0 tx=0).stop 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 shutdown_connections 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe7f806b260 0x7fe7f806d720 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe81415d4e0 0x7fe81415ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe8140d38e0 0x7fe8140b7800 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 --2- 172.21.15.106:0/2415303447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe8140d0070 0x7fe8141526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 >> 172.21.15.106:0/2415303447 conn(0x7fe81409c510 msgr2=0x7fe8140b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 shutdown_connections 2023-11-24T08:12:24.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:24.991+0000 7fe81acb4640 1 -- 172.21.15.106:0/2415303447 wait complete. 2023-11-24T08:12:25.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 43, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:12:09.955346+0000', 'modified': '2023-11-24T08:12:13.029327+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6516}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6516': {'gid': 6516, 'name': 'c', 'rank': 0, 'incarnation': 39, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/4067546353', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 4067546353}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 4067546353}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:12:25.414 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:12:25.414 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-24T08:12:25.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.633+0000 7f8055716640 1 Processor -- start 2023-11-24T08:12:25.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.633+0000 7f8055716640 1 -- start start 2023-11-24T08:12:25.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.634+0000 7f8055716640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f805015b440 0x7f80500cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.634+0000 7f8055716640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f80500cbf20 0x7f80500c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.634+0000 7f8055716640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f805015eea0 con 0x7f80500c57e0 2023-11-24T08:12:25.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f8055716640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80500cc3d0 con 0x7f805015b440 2023-11-24T08:12:25.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f8055716640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f805015b820 con 0x7f80500cbf20 2023-11-24T08:12:25.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f805015b440 0x7f80500cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f805015b440 0x7f80500cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46392/0 (socket says 172.21.15.106:46392) 2023-11-24T08:12:25.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 -- 172.21.15.106:0/1686011701 learned_addr learned my addr 172.21.15.106:0/1686011701 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 -- 172.21.15.106:0/1686011701 >> v2:172.21.15.156:3300/0 conn(0x7f80500cbf20 msgr2=0x7f80500c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1686011701 >> v2:172.21.15.156:3300/0 conn(0x7f80500cbf20 0x7f80500c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 --2- 172.21.15.106:0/1686011701 >> v2:172.21.15.156:3300/0 conn(0x7f80500cbf20 0x7f80500c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 -- 172.21.15.106:0/1686011701 >> v1:172.21.15.156:6789/0 conn(0x7f80500c57e0 legacy=0x7f80500c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804effd640 1 -- 172.21.15.106:0/1686011701 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f805006d600 con 0x7f805015b440 2023-11-24T08:12:25.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.635+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1686011701 >> v2:172.21.15.156:3300/0 conn(0x7f80500cbf20 0x7f80500c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:25.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.637+0000 7f804effd640 1 --2- 172.21.15.106:0/1686011701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805015b440 0x7f80500cb9e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f805006e890 tx=0x7f804002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5f007604b64ac1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.637+0000 7f804dffb640 1 -- 172.21.15.106:0/1686011701 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804002c9f0 con 0x7f805015b440 2023-11-24T08:12:25.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.637+0000 7f804dffb640 1 -- 172.21.15.106:0/1686011701 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804002cb50 con 0x7f805015b440 2023-11-24T08:12:25.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.637+0000 7f804dffb640 1 -- 172.21.15.106:0/1686011701 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80400343f0 con 0x7f805015b440 2023-11-24T08:12:25.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 -- 172.21.15.106:0/1686011701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805015b440 msgr2=0x7f80500cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 --2- 172.21.15.106:0/1686011701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805015b440 0x7f80500cb9e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f805006e890 tx=0x7f804002be50 comp rx=0 tx=0).stop 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 -- 172.21.15.106:0/1686011701 shutdown_connections 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 --2- 172.21.15.106:0/1686011701 >> v2:172.21.15.156:3300/0 conn(0x7f80500cbf20 0x7f80500c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 --2- 172.21.15.106:0/1686011701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f805015b440 0x7f80500cb9e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 -- 172.21.15.106:0/1686011701 >> 172.21.15.106:0/1686011701 conn(0x7f80500b0c50 msgr2=0x7f80500b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 -- 172.21.15.106:0/1686011701 shutdown_connections 2023-11-24T08:12:25.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.638+0000 7f8055716640 1 -- 172.21.15.106:0/1686011701 wait complete. 2023-11-24T08:12:25.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.640+0000 7f8055716640 1 Processor -- start 2023-11-24T08:12:25.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.640+0000 7f8055716640 1 -- start start 2023-11-24T08:12:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f8055716640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 0x7f80501f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f8055716640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80500cbf20 0x7f8050156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f8055716640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f805015b440 0x7f80500c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f8055716640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500c60d0 con 0x7f80500c57e0 2023-11-24T08:12:25.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f8055716640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80500c8060 con 0x7f805015b440 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f8055716640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80500c6320 con 0x7f80500cbf20 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 0x7f80501f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 0x7f80501f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46402/0 (socket says 172.21.15.106:46402) 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 -- 172.21.15.106:0/3823969188 learned_addr learned my addr 172.21.15.106:0/3823969188 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 -- 172.21.15.106:0/3823969188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80500cbf20 msgr2=0x7f8050156c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80500cbf20 0x7f8050156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 -- 172.21.15.106:0/3823969188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f805015b440 msgr2=0x7f80500c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f805015b440 0x7f80500c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.641+0000 7f804effd640 1 -- 172.21.15.106:0/3823969188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8040009b30 con 0x7f80500c57e0 2023-11-24T08:12:25.645 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.642+0000 7f804effd640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 0x7f80501f4e80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8040036040 tx=0x7f8040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.642+0000 7f80477fe640 1 -- 172.21.15.106:0/3823969188 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804003e450 con 0x7f80500c57e0 2023-11-24T08:12:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.642+0000 7f80477fe640 1 -- 172.21.15.106:0/3823969188 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804003e5f0 con 0x7f80500c57e0 2023-11-24T08:12:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.642+0000 7f80477fe640 1 -- 172.21.15.106:0/3823969188 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804003e9a0 con 0x7f80500c57e0 2023-11-24T08:12:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.642+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80500c6600 con 0x7f80500c57e0 2023-11-24T08:12:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.642+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80500c6bc0 con 0x7f80500c57e0 2023-11-24T08:12:25.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.644+0000 7f80477fe640 1 -- 172.21.15.106:0/3823969188 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f804003eb00 con 0x7f80500c57e0 2023-11-24T08:12:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.645+0000 7f80477fe640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f803006b230 0x7f803006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.645+0000 7f80477fe640 1 -- 172.21.15.106:0/3823969188 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80400acc40 con 0x7f80500c57e0 2023-11-24T08:12:25.647 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.645+0000 7f804e7fc640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f803006b230 0x7f803006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.646+0000 7f8058383740 1 Processor -- start 2023-11-24T08:12:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.646+0000 7f804e7fc640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f803006b230 0x7f803006d6f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8038006fd0 tx=0x7f8038008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.648 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.646+0000 7f8058383740 1 -- start start 2023-11-24T08:12:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f8058383740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x5622029e3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029e24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a26970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f8058383740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220293d880 con 0x5622029e32b0 2023-11-24T08:12:25.650 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56220294cae0 con 0x5622029e29f0 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56220294a1d0 con 0x5622029ddfb0 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.647+0000 7f804effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x5622029e3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x5622029e3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46412/0 (socket says 172.21.15.106:46412) 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 -- 172.21.15.106:0/224638380 learned_addr learned my addr 172.21.15.106:0/224638380 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 -- 172.21.15.106:0/224638380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 msgr2=0x5622029e24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029e24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 -- 172.21.15.106:0/224638380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 msgr2=0x562202a26970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a26970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 -- 172.21.15.106:0/224638380 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56220290d860 con 0x5622029e32b0 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f804effd640 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x5622029e3690 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f80400093e0 tx=0x7f804003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e81b017de995a8a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f8044ff9640 1 -- 172.21.15.106:0/224638380 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804007f690 con 0x5622029e32b0 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f8044ff9640 1 -- 172.21.15.106:0/224638380 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040083840 con 0x5622029e32b0 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.648+0000 7f8044ff9640 1 -- 172.21.15.106:0/224638380 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804007aae0 con 0x5622029e32b0 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 -- 172.21.15.106:0/224638380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 msgr2=0x5622029e3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.652 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x5622029e3690 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f80400093e0 tx=0x7f804003f000 comp rx=0 tx=0).stop 2023-11-24T08:12:25.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 -- 172.21.15.106:0/224638380 shutdown_connections 2023-11-24T08:12:25.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a26970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.657 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029e24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 --2- 172.21.15.106:0/224638380 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x5622029e3690 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 -- 172.21.15.106:0/224638380 >> 172.21.15.106:0/224638380 conn(0x5622029a0c70 msgr2=0x562202a28da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 -- 172.21.15.106:0/224638380 shutdown_connections 2023-11-24T08:12:25.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.649+0000 7f8058383740 1 -- 172.21.15.106:0/224638380 wait complete. 2023-11-24T08:12:25.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.650+0000 7f8058383740 1 -- start start 2023-11-24T08:12:25.658 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.650+0000 7f8058383740 1 Processor -- start 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a01fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x562202a2ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x562202a2b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x562202945ac0 con 0x5622029e29f0 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622028e22f0 con 0x5622029ddfb0 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622028ffc70 con 0x5622029e32b0 2023-11-24T08:12:25.659 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.651+0000 7f8058383740 10 client.0 _ll_register_callbacks cb 0x5622029043b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x562202a2ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x562202a2ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46424/0 (socket says 172.21.15.106:46424) 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 -- 172.21.15.106:0/3740197090 learned_addr learned my addr 172.21.15.106:0/3740197090 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804effd640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a01fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 -- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 msgr2=0x562202a2b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804f7fe640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x562202a2b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x562202a2b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.660 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 -- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 msgr2=0x562202a01fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a01fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 -- 172.21.15.106:0/3740197090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80500c6bc0 con 0x5622029e29f0 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804effd640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a01fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f804e7fc640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x562202a2ab80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f803800ce60 tx=0x7f80380137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beb7c3672a1f433d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.652+0000 7f802dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f802dffb640 1 -- 172.21.15.106:0/3740197090 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803801d3f0 con 0x5622029e29f0 2023-11-24T08:12:25.661 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f802dffb640 1 -- 172.21.15.106:0/3740197090 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f803801d550 con 0x5622029e29f0 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56220293eaa0 con 0x5622029e29f0 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f802dffb640 1 -- 172.21.15.106:0/3740197090 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f803801d710 con 0x5622029e29f0 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 10 client.6684 fetch_fsmap learned FSMap version 43 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x562202944140 con 0x5622029e29f0 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f802dffb640 1 -- 172.21.15.106:0/3740197090 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56220293eaa0 con 0x5622029e29f0 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f802dffb640 1 -- 172.21.15.106:0/3740197090 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f803801c070 con 0x5622029e29f0 2023-11-24T08:12:25.662 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 10 client.6684 fetch_fsmap finished waiting for FSMap version 43 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 10 client.6684 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 20 client.6684 populate_metadata read hostname 'smithi106' 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a2dc00 0x562202a279d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 4 client.6684 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.653+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 --> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56220293e7c0 con 0x562202a2dc00 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.654+0000 7f804effd640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a2dc00 0x562202a279d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.663 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.654+0000 7f804effd640 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a2dc00 0x562202a279d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f9fc529e615eb6b server_cookie=a89318d8003e708b in_seq=0 out_seq=0 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.654+0000 7f802dffb640 10 client.6684 ms_handle_connect on v2:172.21.15.106:6835/4067546353 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f802dffb640 1 -- 172.21.15.106:0/3740197090 <== mds.0 v2:172.21.15.106:6835/4067546353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56220293e7c0 con 0x562202a2dc00 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f802dffb640 10 client.6684 handle_command_reply: tid=0 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 client.6684 shutdown 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 msgr2=0x562202a2ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x562202a2ab80 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f803800ce60 tx=0x7f80380137d0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 shutdown_connections 2023-11-24T08:12:25.664 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a2dc00 0x562202a279d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x562202a2b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x562202a2ab80 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 --2- 172.21.15.106:0/3740197090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a01fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f802dffb640 0 client.6684 ms_handle_reset on v2:172.21.15.106:6835/4067546353 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 >> 172.21.15.106:0/3740197090 conn(0x5622029a0c70 msgr2=0x5622029d9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 shutdown_connections 2023-11-24T08:12:25.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.655+0000 7f8058383740 1 -- 172.21.15.106:0/3740197090 wait complete. 2023-11-24T08:12:25.666 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.656+0000 7f8058383740 20 client.6684 trim_cache size 0 max 16384 2023-11-24T08:12:25.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.673+0000 7f8058383740 1 Processor -- start 2023-11-24T08:12:25.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.673+0000 7f8058383740 1 -- start start 2023-11-24T08:12:25.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029d9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a02cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f8058383740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x562202a03200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f8058383740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622026a8f20 con 0x5622029e32b0 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622028f9e30 con 0x5622029e29f0 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562202a01dd0 con 0x5622029ddfb0 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f804f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x562202a03200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f804f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x562202a03200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46438/0 (socket says 172.21.15.106:46438) 2023-11-24T08:12:25.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.674+0000 7f804f7fe640 1 -- 172.21.15.106:0/1468666598 learned_addr learned my addr 172.21.15.106:0/1468666598 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804effd640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029d9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a02cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804f7fe640 1 -- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 msgr2=0x5622029d9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804f7fe640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029d9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804f7fe640 1 -- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 msgr2=0x562202a02cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804f7fe640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a02cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804f7fe640 1 -- 172.21.15.106:0/1468666598 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622028e1b80 con 0x5622029e32b0 2023-11-24T08:12:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804effd640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029d9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:25.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a02cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f804f7fe640 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x562202a03200 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f80480014e0 tx=0x7f80480019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d14fc2145177cac server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f80457fa640 1 -- 172.21.15.106:0/1468666598 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804800b070 con 0x5622029e32b0 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f80457fa640 1 -- 172.21.15.106:0/1468666598 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8048007030 con 0x5622029e32b0 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.675+0000 7f80457fa640 1 -- 172.21.15.106:0/1468666598 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80480073e0 con 0x5622029e32b0 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 -- 172.21.15.106:0/1468666598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 msgr2=0x562202a03200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x562202a03200 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f80480014e0 tx=0x7f80480019a0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 -- 172.21.15.106:0/1468666598 shutdown_connections 2023-11-24T08:12:25.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e32b0 0x562202a03200 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029e29f0 0x562202a02cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 --2- 172.21.15.106:0/1468666598 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029ddfb0 0x5622029d9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 -- 172.21.15.106:0/1468666598 >> 172.21.15.106:0/1468666598 conn(0x5622029a0c70 msgr2=0x5622029e1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 -- 172.21.15.106:0/1468666598 shutdown_connections 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.676+0000 7f8058383740 1 -- 172.21.15.106:0/1468666598 wait complete. 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.677+0000 7f8058383740 1 -- start start 2023-11-24T08:12:25.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.677+0000 7f8058383740 1 Processor -- start 2023-11-24T08:12:25.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a7b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x5622029de4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x5622029de9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x562202a7c8a0 con 0x5622029e29f0 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562202a07b80 con 0x5622029ddfb0 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x562202a7ba40 con 0x5622029e32b0 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f8058383740 10 client.0 _ll_register_callbacks cb 0x7f803800cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x5622029de4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x5622029de4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46442/0 (socket says 172.21.15.106:46442) 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 -- 172.21.15.106:0/1004297999 learned_addr learned my addr 172.21.15.106:0/1004297999 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 -- 172.21.15.106:0/1004297999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 msgr2=0x5622029de9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x5622029de9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 -- 172.21.15.106:0/1004297999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 msgr2=0x562202a7b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a7b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.678+0000 7f804e7fc640 1 -- 172.21.15.106:0/1004297999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8048001190 con 0x5622029e29f0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f804e7fc640 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x5622029de4b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f803800c8b0 tx=0x7f8038013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c41a05080035078 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f802dffb640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f802dffb640 1 -- 172.21.15.106:0/1004297999 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803802e600 con 0x5622029e29f0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f802dffb640 1 -- 172.21.15.106:0/1004297999 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f803802e760 con 0x5622029e29f0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8040083840 con 0x5622029e29f0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f802dffb640 1 -- 172.21.15.106:0/1004297999 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f803802e920 con 0x5622029e29f0 2023-11-24T08:12:25.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f8058383740 10 client.6690 fetch_fsmap learned FSMap version 43 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=2+}) v3 -- 0x562202907170 con 0x5622029e29f0 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.679+0000 7f802dffb640 1 -- 172.21.15.106:0/1004297999 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8040083840 con 0x5622029e29f0 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f802dffb640 1 -- 172.21.15.106:0/1004297999 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 43) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f803801c040 con 0x5622029e29f0 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f8058383740 10 client.6690 fetch_fsmap finished waiting for FSMap version 43 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f8058383740 10 client.6690 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f8058383740 20 client.6690 populate_metadata read hostname 'smithi106' 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f8058383740 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a3ac20 0x562202a1ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f8058383740 4 client.6690 mds_command: new command op to 6516 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 --> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5622026466c0 con 0x562202a3ac20 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f804effd640 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a3ac20 0x562202a1ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f804effd640 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a3ac20 0x562202a1ea60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edd60856c6267299 server_cookie=be081b0d14328a85 in_seq=0 out_seq=0 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.680+0000 7f802dffb640 10 client.6690 ms_handle_connect on v2:172.21.15.106:6835/4067546353 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f802dffb640 1 -- 172.21.15.106:0/1004297999 <== mds.0 v2:172.21.15.106:6835/4067546353 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5622026466c0 con 0x562202a3ac20 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f802dffb640 10 client.6690 handle_command_reply: tid=0 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 client.6690 shutdown 2023-11-24T08:12:25.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 msgr2=0x5622029de4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x5622029de4b0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f803800c8b0 tx=0x7f8038013000 comp rx=0 tx=0).stop 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 shutdown_connections 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:6835/4067546353,v1:172.21.15.106:6837/4067546353] conn(0x562202a3ac20 0x562202a1ea60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x5622029e32b0 0x5622029de9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x5622029e29f0 0x5622029de4b0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 --2- 172.21.15.106:0/1004297999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5622029ddfb0 0x562202a7b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f802dffb640 0 client.6690 ms_handle_reset on v2:172.21.15.106:6835/4067546353 2023-11-24T08:12:25.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 >> 172.21.15.106:0/1004297999 conn(0x5622029a0c70 msgr2=0x5622029d9360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 shutdown_connections 2023-11-24T08:12:25.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 1 -- 172.21.15.106:0/1004297999 wait complete. 2023-11-24T08:12:25.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.681+0000 7f8058383740 20 client.6690 trim_cache size 0 max 16384 2023-11-24T08:12:25.691 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:12:25.691 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d1a897c2-15ce-4272-ad66-4b3723970cb1", 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 6516, 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 15.143449819000001, 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 43, 2023-11-24T08:12:25.692 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 75, 2023-11-24T08:12:25.693 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 75, 2023-11-24T08:12:25.693 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 20.545355992000001 2023-11-24T08:12:25.693 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f803006b230 msgr2=0x7f803006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f803006b230 0x7f803006d6f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f8038006fd0 tx=0x7f8038008040 comp rx=0 tx=0).stop 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 msgr2=0x7f80501f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 0x7f80501f4e80 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f8040036040 tx=0x7f8040004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 shutdown_connections 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f805015b440 0x7f80500c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f803006b230 0x7f803006d6f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80500cbf20 0x7f8050156c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 --2- 172.21.15.106:0/3823969188 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80500c57e0 0x7f80501f4e80 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 >> 172.21.15.106:0/3823969188 conn(0x7f80500b0c50 msgr2=0x7f80500b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 shutdown_connections 2023-11-24T08:12:25.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.682+0000 7f8055716640 1 -- 172.21.15.106:0/3823969188 wait complete. 2023-11-24T08:12:25.700 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:12:25.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.898+0000 7fd37ff53640 1 Processor -- start 2023-11-24T08:12:25.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.899+0000 7fd37ff53640 1 -- start start 2023-11-24T08:12:25.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.900+0000 7fd37ff53640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 0x7fd3780ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.900+0000 7fd37ff53640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd3780cd470 0x7fd37815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.900+0000 7fd37ff53640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37815ab80 con 0x7fd3780b2250 2023-11-24T08:12:25.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.900+0000 7fd37ff53640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd37815a990 con 0x7fd3780cd470 2023-11-24T08:12:25.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.900+0000 7fd37ff53640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd37815e3c0 con 0x7fd37815df80 2023-11-24T08:12:25.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.901+0000 7fd37e4f0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd3780cd470 0x7fd37815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.901+0000 7fd37d4ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 0x7fd3780ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.901+0000 7fd37e4f0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd3780cd470 0x7fd37815b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:48078/0 (socket says 172.21.15.106:48078) 2023-11-24T08:12:25.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.901+0000 7fd37d4ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 0x7fd3780ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:59528/0 (socket says 172.21.15.106:59528) 2023-11-24T08:12:25.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.901+0000 7fd37e4f0640 1 -- 172.21.15.106:0/3025445125 learned_addr learned my addr 172.21.15.106:0/3025445125 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.902+0000 7fd37e4f0640 1 -- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 msgr2=0x7fd3780ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:25.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.902+0000 7fd37e4f0640 1 --2- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 0x7fd3780ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.902+0000 7fd37e4f0640 1 -- 172.21.15.106:0/3025445125 >> v1:172.21.15.156:6789/0 conn(0x7fd3780b2250 legacy=0x7fd3780b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.902+0000 7fd37e4f0640 1 -- 172.21.15.106:0/3025445125 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3780b2d40 con 0x7fd3780cd470 2023-11-24T08:12:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.902+0000 7fd37d4ee640 1 --2- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 0x7fd3780ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:12:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.903+0000 7fd37e4f0640 1 --2- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780cd470 0x7fd37815b080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd37002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71b1d962dfd07f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.904+0000 7fd37cced640 1 -- 172.21.15.106:0/3025445125 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37002cb00 con 0x7fd3780cd470 2023-11-24T08:12:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.904+0000 7fd37cced640 1 -- 172.21.15.106:0/3025445125 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37002cc60 con 0x7fd3780cd470 2023-11-24T08:12:25.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.904+0000 7fd37cced640 1 -- 172.21.15.106:0/3025445125 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3700343f0 con 0x7fd3780cd470 2023-11-24T08:12:25.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 -- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780cd470 msgr2=0x7fd37815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 --2- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780cd470 0x7fd37815b080 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd37002bf60 comp rx=0 tx=0).stop 2023-11-24T08:12:25.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 -- 172.21.15.106:0/3025445125 shutdown_connections 2023-11-24T08:12:25.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 --2- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780cd470 0x7fd37815b080 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 --2- 172.21.15.106:0/3025445125 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd37815df80 0x7fd3780ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 -- 172.21.15.106:0/3025445125 >> 172.21.15.106:0/3025445125 conn(0x7fd37806a920 msgr2=0x7fd37806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 -- 172.21.15.106:0/3025445125 shutdown_connections 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.905+0000 7fd37ff53640 1 -- 172.21.15.106:0/3025445125 wait complete. 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.907+0000 7fd37ff53640 1 Processor -- start 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.907+0000 7fd37ff53640 1 -- start start 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.908+0000 7fd37ff53640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780b2250 0x7fd3781f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.908+0000 7fd37ff53640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd3780cd470 0x7fd378150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.908+0000 7fd37ff53640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 0x7fd378071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.908+0000 7fd37ff53640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781e4d00 con 0x7fd37815df80 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.908+0000 7fd37ff53640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781e3da0 con 0x7fd3780b2250 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.908+0000 7fd37ff53640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3781e4080 con 0x7fd3780cd470 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 0x7fd378071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 0x7fd378071e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46446/0 (socket says 172.21.15.106:46446) 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 -- 172.21.15.106:0/2512084117 learned_addr learned my addr 172.21.15.106:0/2512084117 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:25.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37d4ee640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd3780cd470 0x7fd378150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37dcef640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780b2250 0x7fd3781f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 -- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd3780cd470 msgr2=0x7fd378150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd3780cd470 0x7fd378150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 -- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780b2250 msgr2=0x7fd3781f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780b2250 0x7fd3781f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37e4f0640 1 -- 172.21.15.106:0/2512084117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36800b6a0 con 0x7fd37815df80 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.909+0000 7fd37d4ee640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd3780cd470 0x7fd378150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:25.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.910+0000 7fd37e4f0640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 0x7fd378071e40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd370004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.910+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd370009510 con 0x7fd37815df80 2023-11-24T08:12:25.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.910+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37003e450 con 0x7fd37815df80 2023-11-24T08:12:25.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.910+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37003e800 con 0x7fd37815df80 2023-11-24T08:12:25.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.910+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd370009b30 con 0x7fd37815df80 2023-11-24T08:12:25.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.910+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3781e4640 con 0x7fd37815df80 2023-11-24T08:12:25.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.912+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd378067470 con 0x7fd37815df80 2023-11-24T08:12:25.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.912+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fd37003e960 con 0x7fd37815df80 2023-11-24T08:12:25.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.913+0000 7fd36e7fc640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd35c06b230 0x7fd35c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:25.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.913+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3700ac8d0 con 0x7fd37815df80 2023-11-24T08:12:25.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.914+0000 7fd37dcef640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd35c06b230 0x7fd35c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:25.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.914+0000 7fd37dcef640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd35c06b230 0x7fd35c06d6f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd374000f60 tx=0x7fd374009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:25.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:25.918+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd370077430 con 0x7fd37815df80 2023-11-24T08:12:26.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.146+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd3780cea00 con 0x7fd37815df80 2023-11-24T08:12:26.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.147+0000 7fd36e7fc640 1 -- 172.21.15.106:0/2512084117 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd37007aa50 con 0x7fd37815df80 2023-11-24T08:12:26.148 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:12:26.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.149+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd35c06b230 msgr2=0x7fd35c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:26.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.149+0000 7fd37ff53640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd35c06b230 0x7fd35c06d6f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fd374000f60 tx=0x7fd374009040 comp rx=0 tx=0).stop 2023-11-24T08:12:26.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.149+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 msgr2=0x7fd378071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:26.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.149+0000 7fd37ff53640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 0x7fd378071e40 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fd370009e50 tx=0x7fd370004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:26.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 shutdown_connections 2023-11-24T08:12:26.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd37815df80 0x7fd378071e40 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:26.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd35c06b230 0x7fd35c06d6f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:26.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd3780cd470 0x7fd378150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:26.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 --2- 172.21.15.106:0/2512084117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3780b2250 0x7fd3781f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:26.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 >> 172.21.15.106:0/2512084117 conn(0x7fd37806a920 msgr2=0x7fd3780cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:26.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 shutdown_connections 2023-11-24T08:12:26.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:26.150+0000 7fd37ff53640 1 -- 172.21.15.106:0/2512084117 wait complete. 2023-11-24T08:12:26.170 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:12:26.170 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:12:26.171 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:12:26.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.172 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:12:26.223 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:12:26.224 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:12:26.224 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:12:26.224 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.224 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:12:26.224 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.225 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:12:26.225 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:26.225 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:12:26.225 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:12:26.225 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:26.226 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:12:26.226 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:12:26.226 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:26.226 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:12:26.226 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:26.227 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:12:26.227 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42170sec preferred_lft 42170sec 2023-11-24T08:12:26.227 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:12:26.227 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.227 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:12:26.228 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:26.228 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:12:26.228 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.228 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a407:f7ff:fe84:63d0/64 scope link 2023-11-24T08:12:26.228 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.228 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:12:26.229 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:12:26.229 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9079:d8ff:febb:c08c/64 scope link 2023-11-24T08:12:26.229 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.230 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:12:26.230 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:12:26.230 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::20e3:f2ff:fe93:487e/64 scope link 2023-11-24T08:12:26.230 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:26.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.231 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:12:26.320 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:12:26.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.324 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:12:26.379 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:12:26.379 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:12:26.380 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:12:26.380 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:12:26.380 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:12:26.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.380 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:12:26.380 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:12:26.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:12:26.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:12:26.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:12:26.381 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:12:26.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.566 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:12:26.615 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:12:26.660 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:12:26.661 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:12:26.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.661 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2023-11-24T08:12:26.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:12:26.763 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:12:26.817 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:12:26.818 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:12:26.818 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:12:26.869 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:12:26.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:26.869 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:12:27.008 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:12:27.008 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:12:27.038 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:12:27.038 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:12:27.038 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:12:27.038 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:12:27.039 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:12:27.039 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-24T08:12:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:12:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:12:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:12:27.040 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:12:27.044 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:12:27.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:27.044 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:12:27.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:27.131 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:12:27.200 INFO:teuthology.orchestra.run.smithi106.stdout:client6696 2023-11-24T08:12:27.200 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:27.200 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:27.253 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:12:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.513+0000 7f2909f61640 1 Processor -- start 2023-11-24T08:12:27.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.513+0000 7f2909f61640 1 -- start start 2023-11-24T08:12:27.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2909f61640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cf450 0x7f29040cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:27.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2909f61640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29040cdfb0 0x7f29040ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2909f61640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29040cece0 con 0x7f290415c7f0 2023-11-24T08:12:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2909f61640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2904156010 con 0x7f29040cdfb0 2023-11-24T08:12:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2909f61640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290406ce30 con 0x7f29040cf450 2023-11-24T08:12:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2903fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29040cdfb0 0x7f29040ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:27.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2903fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f29040cdfb0 0x7f29040ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46454/0 (socket says 172.21.15.106:46454) 2023-11-24T08:12:27.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f2903fff640 1 -- 172.21.15.106:0/1175996740 learned_addr learned my addr 172.21.15.106:0/1175996740 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:27.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f29037fe640 1 --1- 172.21.15.106:0/1175996740 >> v1:172.21.15.106:6789/0 conn(0x7f290415c7f0 0x7f290406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38106/0 (socket says 172.21.15.106:38106) 2023-11-24T08:12:27.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.515+0000 7f28faffd640 1 --2- 172.21.15.106:0/1175996740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cf450 0x7f29040cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.516+0000 7f2903fff640 1 -- 172.21.15.106:0/1175996740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cf450 msgr2=0x7f29040cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.516+0000 7f2903fff640 1 --2- 172.21.15.106:0/1175996740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cf450 0x7f29040cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.516+0000 7f2903fff640 1 -- 172.21.15.106:0/1175996740 >> v1:172.21.15.106:6789/0 conn(0x7f290415c7f0 legacy=0x7f290406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:27.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.516+0000 7f2903fff640 1 -- 172.21.15.106:0/1175996740 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290406d040 con 0x7f29040cdfb0 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.517+0000 7f2903fff640 1 --2- 172.21.15.106:0/1175996740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29040cdfb0 0x7f29040ce440 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e50 tx=0x7f28fc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf8d49af4b987a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.517+0000 7f2902ffd640 1 -- 172.21.15.106:0/1175996740 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc02cb00 con 0x7f29040cdfb0 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.517+0000 7f2902ffd640 1 -- 172.21.15.106:0/1175996740 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc02cc60 con 0x7f29040cdfb0 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.517+0000 7f2902ffd640 1 -- 172.21.15.106:0/1175996740 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0343f0 con 0x7f29040cdfb0 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 -- 172.21.15.106:0/1175996740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29040cdfb0 msgr2=0x7f29040ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 --2- 172.21.15.106:0/1175996740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29040cdfb0 0x7f29040ce440 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e50 tx=0x7f28fc02bf60 comp rx=0 tx=0).stop 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 -- 172.21.15.106:0/1175996740 shutdown_connections 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 --2- 172.21.15.106:0/1175996740 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f29040cdfb0 0x7f29040ce440 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 --2- 172.21.15.106:0/1175996740 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cf450 0x7f29040cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 -- 172.21.15.106:0/1175996740 >> 172.21.15.106:0/1175996740 conn(0x7f290406a920 msgr2=0x7f290406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:27.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 -- 172.21.15.106:0/1175996740 shutdown_connections 2023-11-24T08:12:27.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.518+0000 7f2909f61640 1 -- 172.21.15.106:0/1175996740 wait complete. 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.520+0000 7f2909f61640 1 Processor -- start 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.520+0000 7f2909f61640 1 -- start start 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2909f61640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cdfb0 0x7f29041f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2909f61640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29040cf450 0x7f29040c8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2909f61640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 0x7f290415a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2909f61640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29040ad8e0 con 0x7f290415c7f0 2023-11-24T08:12:27.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2909f61640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29041f33b0 con 0x7f29040cf450 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2909f61640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040ac4c0 con 0x7f29040cdfb0 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 0x7f290415a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 0x7f290415a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46458/0 (socket says 172.21.15.106:46458) 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 -- 172.21.15.106:0/102745638 learned_addr learned my addr 172.21.15.106:0/102745638 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f29037fe640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cdfb0 0x7f29041f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f28faffd640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29040cf450 0x7f29040c8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 -- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cdfb0 msgr2=0x7f29041f4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cdfb0 0x7f29041f4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 -- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29040cf450 msgr2=0x7f29040c8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29040cf450 0x7f29040c8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f2903fff640 1 -- 172.21.15.106:0/102745638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009b30 con 0x7f290415c7f0 2023-11-24T08:12:27.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.521+0000 7f29037fe640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cdfb0 0x7f29041f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.522+0000 7f2903fff640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 0x7f290415a990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e50 tx=0x7f28fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.522+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc009510 con 0x7f290415c7f0 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.522+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc03e450 con 0x7f290415c7f0 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.522+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc03e800 con 0x7f290415c7f0 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.522+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29040ac7a0 con 0x7f290415c7f0 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.522+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29040acd60 con 0x7f290415c7f0 2023-11-24T08:12:27.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.523+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29041f3520 con 0x7f290415c7f0 2023-11-24T08:12:27.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.524+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f28fc03e960 con 0x7f290415c7f0 2023-11-24T08:12:27.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.525+0000 7f2900ff9640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f28e406b260 0x7f28e406d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:27.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.525+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28fc0ac9c0 con 0x7f290415c7f0 2023-11-24T08:12:27.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.525+0000 7f29037fe640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f28e406b260 0x7f28e406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:27.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.526+0000 7f29037fe640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f28e406b260 0x7f28e406d720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000f60 tx=0x7f28f4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:27.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.530+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f28fc0774e0 con 0x7f290415c7f0 2023-11-24T08:12:27.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.758+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2904067470 con 0x7f290415c7f0 2023-11-24T08:12:27.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.758+0000 7f2900ff9640 1 -- 172.21.15.106:0/102745638 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v43) v1 ==== 53+0+73 (secure 0 0 0) 0x7f28fc07ab00 con 0x7f290415c7f0 2023-11-24T08:12:27.761 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:12:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f28e406b260 msgr2=0x7f28e406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:27.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f28e406b260 0x7f28e406d720 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000f60 tx=0x7f28f4009040 comp rx=0 tx=0).stop 2023-11-24T08:12:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 msgr2=0x7f290415a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 0x7f290415a990 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f28fc009e50 tx=0x7f28fc004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 shutdown_connections 2023-11-24T08:12:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f290415c7f0 0x7f290415a990 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f29040cf450 0x7f29040c8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f28e406b260 0x7f28e406d720 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 --2- 172.21.15.106:0/102745638 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f29040cdfb0 0x7f29041f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 >> 172.21.15.106:0/102745638 conn(0x7f290406a920 msgr2=0x7f290415cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.761+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 shutdown_connections 2023-11-24T08:12:27.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:27.762+0000 7f2909f61640 1 -- 172.21.15.106:0/102745638 wait complete. 2023-11-24T08:12:27.781 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:12:27.781 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:12:27.781 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:12:27.782 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:12:27.782 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:12:27.782 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:12:27.782 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:12:27.782 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:12:27.782 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:12:27.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:27.783 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.843 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42184sec preferred_lft 42184sec 2023-11-24T08:12:27.844 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::1831:a6ff:fe65:552d/64 scope link 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:12:27.845 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::683b:9fff:fe67:7152/64 scope link 2023-11-24T08:12:27.846 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.846 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:12:27.846 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether a2:a3:a7:89:e5:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:12:27.846 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a0a3:a7ff:fe89:e507/64 scope link 2023-11-24T08:12:27.847 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:12:27.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:27.848 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:12:27.942 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:12:27.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:27.946 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:12:28.000 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:12:28.000 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:12:28.000 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:12:28.000 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:12:28.000 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:12:28.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:28.000 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:12:28.001 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:12:28.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:12:28.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:12:28.001 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:12:28.001 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:12:28.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:28.180 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:12:28.232 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:12:28.232 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:12:28.233 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:12:28.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:28.233 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-24T08:12:28.326 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:12:28.326 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:12:28.375 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:12:28.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:12:28.375 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:12:28.419 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:12:28.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:28.420 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:12:28.606 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:12:28.606 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:12:28.641 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:12:28.641 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:12:28.641 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:12:28.641 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 1@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:12:28.641 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:12:28.642 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-24T08:12:28.642 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:12:28.643 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:12:28.643 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:12:28.643 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:12:28.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:12:28.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:28.646 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:12:28.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:28.731 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:12:28.791 INFO:teuthology.orchestra.run.smithi156.stdout:client5332 2023-11-24T08:12:28.791 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:28.791 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:28.848 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:12:29.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.072+0000 7f6333bec640 1 Processor -- start 2023-11-24T08:12:29.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.072+0000 7f6333bec640 1 -- start start 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.073+0000 7f6333bec640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c15d4f0 0x7f632c0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.073+0000 7f6333bec640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f632c15d8d0 0x7f632c0d17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.073+0000 7f6333bec640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f632c0cdf30 con 0x7f632c0d0a50 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.073+0000 7f6333bec640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f632c0d3e30 con 0x7f632c15d8d0 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.073+0000 7f6333bec640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c0d3b50 con 0x7f632c15d4f0 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6332189640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f632c15d8d0 0x7f632c0d17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6331187640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f632c0d0a50 0x7f632c0d30f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38120/0 (socket says 172.21.15.106:38120) 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6332189640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f632c15d8d0 0x7f632c0d17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46472/0 (socket says 172.21.15.106:46472) 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6331187640 1 -- 172.21.15.106:0/970476781 learned_addr learned my addr 172.21.15.106:0/970476781 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:29.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6331988640 1 --2- 172.21.15.106:0/970476781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c15d4f0 0x7f632c0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6332189640 1 -- 172.21.15.106:0/970476781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c15d4f0 msgr2=0x7f632c0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6332189640 1 --2- 172.21.15.106:0/970476781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c15d4f0 0x7f632c0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6332189640 1 -- 172.21.15.106:0/970476781 >> v1:172.21.15.106:6789/0 conn(0x7f632c0d0a50 legacy=0x7f632c0d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6330986640 1 -- 172.21.15.106:0/970476781 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321100809 0 0) 0x7f632c0cdf30 con 0x7f632c0d0a50 2023-11-24T08:12:29.078 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.074+0000 7f6332189640 1 -- 172.21.15.106:0/970476781 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c0ce1d0 con 0x7f632c15d8d0 2023-11-24T08:12:29.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6332189640 1 --2- 172.21.15.106:0/970476781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d8d0 0x7f632c0d17d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f63240097a0 tx=0x7f632402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173b20d7f8e65feb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:29.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6330986640 1 -- 172.21.15.106:0/970476781 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632402d040 con 0x7f632c15d8d0 2023-11-24T08:12:29.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6330986640 1 -- 172.21.15.106:0/970476781 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324031810 con 0x7f632c15d8d0 2023-11-24T08:12:29.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6330986640 1 -- 172.21.15.106:0/970476781 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324031ba0 con 0x7f632c15d8d0 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6333bec640 1 -- 172.21.15.106:0/970476781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d8d0 msgr2=0x7f632c0d17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6333bec640 1 --2- 172.21.15.106:0/970476781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d8d0 0x7f632c0d17d0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f63240097a0 tx=0x7f632402b8b0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6333bec640 1 -- 172.21.15.106:0/970476781 shutdown_connections 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6333bec640 1 --2- 172.21.15.106:0/970476781 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d8d0 0x7f632c0d17d0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6333bec640 1 --2- 172.21.15.106:0/970476781 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c15d4f0 0x7f632c0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.076+0000 7f6333bec640 1 -- 172.21.15.106:0/970476781 >> 172.21.15.106:0/970476781 conn(0x7f632c061950 msgr2=0x7f632c08af60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.077+0000 7f6333bec640 1 -- 172.21.15.106:0/970476781 shutdown_connections 2023-11-24T08:12:29.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.077+0000 7f6333bec640 1 -- 172.21.15.106:0/970476781 wait complete. 2023-11-24T08:12:29.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.078+0000 7f6333bec640 1 Processor -- start 2023-11-24T08:12:29.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.078+0000 7f6333bec640 1 -- start start 2023-11-24T08:12:29.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.079+0000 7f6333bec640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c0d0a50 0x7f632c15b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.079+0000 7f6333bec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 0x7f632c15feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.079+0000 7f6333bec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15d8d0 0x7f632c1603f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.079+0000 7f6333bec640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1f3e10 con 0x7f632c15d4f0 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.079+0000 7f6333bec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1f4230 con 0x7f632c15d8d0 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.079+0000 7f6333bec640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f632c1f3470 con 0x7f632c0d0a50 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 0x7f632c15feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 0x7f632c15feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46486/0 (socket says 172.21.15.106:46486) 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 -- 172.21.15.106:0/1076032012 learned_addr learned my addr 172.21.15.106:0/1076032012 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331988640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c0d0a50 0x7f632c15b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 -- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c0d0a50 msgr2=0x7f632c15b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c0d0a50 0x7f632c15b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6332189640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15d8d0 0x7f632c1603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 -- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15d8d0 msgr2=0x7f632c1603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15d8d0 0x7f632c1603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 -- 172.21.15.106:0/1076032012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009400 con 0x7f632c15d4f0 2023-11-24T08:12:29.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.080+0000 7f6331187640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 0x7f632c15feb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4d0 tx=0x7f632000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:29.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.081+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63200074e0 con 0x7f632c15d4f0 2023-11-24T08:12:29.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.081+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6320007660 con 0x7f632c15d4f0 2023-11-24T08:12:29.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.081+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6320004a20 con 0x7f632c15d4f0 2023-11-24T08:12:29.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.081+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f632c1f3750 con 0x7f632c15d4f0 2023-11-24T08:12:29.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.081+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f632c1f7760 con 0x7f632c15d4f0 2023-11-24T08:12:29.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.082+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f632c1f3f50 con 0x7f632c15d4f0 2023-11-24T08:12:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.083+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f6320004b80 con 0x7f632c15d4f0 2023-11-24T08:12:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.084+0000 7f632a7fc640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f631806b230 0x7f631806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.084+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6320088e70 con 0x7f632c15d4f0 2023-11-24T08:12:29.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.084+0000 7f6331988640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f631806b230 0x7f631806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.085+0000 7f6331988640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f631806b230 0x7f631806d6f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f632c0b7ec0 tx=0x7f631c0039b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:29.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.089+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f632c1f3f50 con 0x7f632c15d4f0 2023-11-24T08:12:29.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.314+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f632c1f3f50 con 0x7f632c15d4f0 2023-11-24T08:12:29.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.362+0000 7f632a7fc640 1 -- 172.21.15.106:0/1076032012 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v44) v1 ==== 136+0+0 (secure 0 0 0) 0x7f632c1f3f50 con 0x7f632c15d4f0 2023-11-24T08:12:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f631806b230 msgr2=0x7f631806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f631806b230 0x7f631806d6f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f632c0b7ec0 tx=0x7f631c0039b0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 msgr2=0x7f632c15feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 0x7f632c15feb0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f632000c4d0 tx=0x7f632000c990 comp rx=0 tx=0).stop 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 shutdown_connections 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f632c15d8d0 0x7f632c1603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f632c15d4f0 0x7f632c15feb0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f631806b230 0x7f631806d6f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 --2- 172.21.15.106:0/1076032012 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f632c0d0a50 0x7f632c15b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 >> 172.21.15.106:0/1076032012 conn(0x7f632c061950 msgr2=0x7f632c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 shutdown_connections 2023-11-24T08:12:29.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.366+0000 7f6333bec640 1 -- 172.21.15.106:0/1076032012 wait complete. 2023-11-24T08:12:29.386 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:12:29.586 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.584+0000 7fa86674c640 1 Processor -- start 2023-11-24T08:12:29.587 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.584+0000 7fa86674c640 1 -- start start 2023-11-24T08:12:29.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.585+0000 7fa86674c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa86015b440 0x7fa8600cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.585+0000 7fa86674c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8600cbf20 0x7fa8600c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.585+0000 7fa86674c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600c57e0 0x7fa8600c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.585+0000 7fa86674c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8600cc3d0 con 0x7fa8600cbf20 2023-11-24T08:12:29.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.585+0000 7fa86674c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa86015b820 con 0x7fa86015b440 2023-11-24T08:12:29.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.585+0000 7fa86674c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86015eea0 con 0x7fa8600c57e0 2023-11-24T08:12:29.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8600cbf20 0x7fa8600c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8600cbf20 0x7fa8600c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46498/0 (socket says 172.21.15.106:46498) 2023-11-24T08:12:29.589 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 -- 172.21.15.106:0/1969321699 learned_addr learned my addr 172.21.15.106:0/1969321699 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:29.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85ffff640 1 --2- 172.21.15.106:0/1969321699 >> v2:172.21.15.156:3300/0 conn(0x7fa86015b440 0x7fa8600cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 -- 172.21.15.106:0/1969321699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600c57e0 msgr2=0x7fa8600c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:29.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa864ce9640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600c57e0 0x7fa8600c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600c57e0 0x7fa8600c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 -- 172.21.15.106:0/1969321699 >> v2:172.21.15.156:3300/0 conn(0x7fa86015b440 msgr2=0x7fa8600cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 --2- 172.21.15.106:0/1969321699 >> v2:172.21.15.156:3300/0 conn(0x7fa86015b440 0x7fa8600cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85f7fe640 1 -- 172.21.15.106:0/1969321699 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600abf30 con 0x7fa8600cbf20 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa864ce9640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600c57e0 0x7fa8600c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.586+0000 7fa85ffff640 1 --2- 172.21.15.106:0/1969321699 >> v2:172.21.15.156:3300/0 conn(0x7fa86015b440 0x7fa8600cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.587+0000 7fa85f7fe640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8600cbf20 0x7fa8600c52a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa86006e770 tx=0x7fa85402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2574ed0f291bdd0e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:29.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa85effd640 1 -- 172.21.15.106:0/1969321699 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8540317e0 con 0x7fa8600cbf20 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa85effd640 1 -- 172.21.15.106:0/1969321699 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa854031940 con 0x7fa8600cbf20 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa85effd640 1 -- 172.21.15.106:0/1969321699 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa854031cd0 con 0x7fa8600cbf20 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 -- 172.21.15.106:0/1969321699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8600cbf20 msgr2=0x7fa8600c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8600cbf20 0x7fa8600c52a0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fa86006e770 tx=0x7fa85402b880 comp rx=0 tx=0).stop 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 -- 172.21.15.106:0/1969321699 shutdown_connections 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600c57e0 0x7fa8600c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 --2- 172.21.15.106:0/1969321699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8600cbf20 0x7fa8600c52a0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 --2- 172.21.15.106:0/1969321699 >> v2:172.21.15.156:3300/0 conn(0x7fa86015b440 0x7fa8600cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 -- 172.21.15.106:0/1969321699 >> 172.21.15.106:0/1969321699 conn(0x7fa8600b0c50 msgr2=0x7fa8600b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:29.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 -- 172.21.15.106:0/1969321699 shutdown_connections 2023-11-24T08:12:29.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.588+0000 7fa86674c640 1 -- 172.21.15.106:0/1969321699 wait complete. 2023-11-24T08:12:29.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.590+0000 7fa86674c640 1 Processor -- start 2023-11-24T08:12:29.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.590+0000 7fa86674c640 1 -- start start 2023-11-24T08:12:29.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa86674c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8600c57e0 0x7fa86014beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa86674c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600cbf20 0x7fa8600c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa86674c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 0x7fa8600b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa86674c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c70e0 con 0x7fa86015b440 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa86674c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c73c0 con 0x7fa8600c57e0 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa86674c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8600c76a0 con 0x7fa8600cbf20 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa864ce9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 0x7fa8600b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa864ce9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 0x7fa8600b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46512/0 (socket says 172.21.15.106:46512) 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.591+0000 7fa864ce9640 1 -- 172.21.15.106:0/2333316099 learned_addr learned my addr 172.21.15.106:0/2333316099 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa864ce9640 1 -- 172.21.15.106:0/2333316099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600cbf20 msgr2=0x7fa8600c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa864ce9640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600cbf20 0x7fa8600c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa864ce9640 1 -- 172.21.15.106:0/2333316099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8600c57e0 msgr2=0x7fa86014beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa864ce9640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8600c57e0 0x7fa86014beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa864ce9640 1 -- 172.21.15.106:0/2333316099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540094d0 con 0x7fa86015b440 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa864ce9640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 0x7fa8600b0100 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa85800cef0 tx=0x7fa85800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800d840 con 0x7fa86015b440 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85800d9a0 con 0x7fa86015b440 2023-11-24T08:12:29.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85800dd50 con 0x7fa86015b440 2023-11-24T08:12:29.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8600c7980 con 0x7fa86015b440 2023-11-24T08:12:29.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.592+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8600c6d90 con 0x7fa86015b440 2023-11-24T08:12:29.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.593+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8600c7ea0 con 0x7fa86015b440 2023-11-24T08:12:29.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.594+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fa85801e3b0 con 0x7fa86015b440 2023-11-24T08:12:29.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.596+0000 7fa85cff9640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa84c06b260 0x7fa84c06d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:29.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.596+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa85808a830 con 0x7fa86015b440 2023-11-24T08:12:29.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.596+0000 7fa85ffff640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa84c06b260 0x7fa84c06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:29.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.597+0000 7fa85ffff640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa84c06b260 0x7fa84c06d720 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa850006fd0 tx=0x7fa850008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:29.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.603+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8600c7ea0 con 0x7fa86015b440 2023-11-24T08:12:29.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.819+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa86015f6b0 con 0x7fa84c06b260 2023-11-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.822+0000 7fa85cff9640 1 -- 172.21.15.106:0/2333316099 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa85000ae00 con 0x7fa84c06b260 2023-11-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:12:29.824 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2023-11-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:29.825 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:12:29.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa84c06b260 msgr2=0x7fa84c06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa84c06b260 0x7fa84c06d720 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa850006fd0 tx=0x7fa850008040 comp rx=0 tx=0).stop 2023-11-24T08:12:29.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 msgr2=0x7fa8600b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:29.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 0x7fa8600b0100 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fa85800cef0 tx=0x7fa85800a710 comp rx=0 tx=0).stop 2023-11-24T08:12:29.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 shutdown_connections 2023-11-24T08:12:29.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa86015b440 0x7fa8600b0100 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa84c06b260 0x7fa84c06d720 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8600cbf20 0x7fa8600c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.824+0000 7fa86674c640 1 --2- 172.21.15.106:0/2333316099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8600c57e0 0x7fa86014beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.825+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 >> 172.21.15.106:0/2333316099 conn(0x7fa8600b0c50 msgr2=0x7fa8600b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.825+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 shutdown_connections 2023-11-24T08:12:29.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:29.825+0000 7fa86674c640 1 -- 172.21.15.106:0/2333316099 wait complete. 2023-11-24T08:12:29.844 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:12:30.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.045+0000 7fed0b9a9640 1 Processor -- start 2023-11-24T08:12:30.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.045+0000 7fed0b9a9640 1 -- start start 2023-11-24T08:12:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.046+0000 7fed0b9a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed0415d7f0 0x7fed040c5990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.046+0000 7fed0b9a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed0406d600 0x7fed040cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.046+0000 7fed0b9a9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed0415e400 con 0x7fed040c4220 2023-11-24T08:12:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.046+0000 7fed0b9a9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed040ce880 con 0x7fed0415d7f0 2023-11-24T08:12:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.046+0000 7fed0b9a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040c5ed0 con 0x7fed0406d600 2023-11-24T08:12:30.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed0415d7f0 0x7fed040c5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed0415d7f0 0x7fed040c5990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46514/0 (socket says 172.21.15.106:46514) 2023-11-24T08:12:30.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 -- 172.21.15.106:0/3958361860 learned_addr learned my addr 172.21.15.106:0/3958361860 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:30.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09f46640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed0406d600 0x7fed040cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 -- 172.21.15.106:0/3958361860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed0406d600 msgr2=0x7fed040cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:30.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed0406d600 0x7fed040cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 -- 172.21.15.106:0/3958361860 >> v1:172.21.15.156:6789/0 conn(0x7fed040c4220 legacy=0x7fed040cabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:30.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09745640 1 -- 172.21.15.106:0/3958361860 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed040c46e0 con 0x7fed0415d7f0 2023-11-24T08:12:30.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.047+0000 7fed09f46640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed0406d600 0x7fed040cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.048+0000 7fed09745640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0415d7f0 0x7fed040c5990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fed0406ea50 tx=0x7fed0002e0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e722cfd4206d4e81 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fecfbfff640 1 -- 172.21.15.106:0/3958361860 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0002ee50 con 0x7fed0415d7f0 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fecfbfff640 1 -- 172.21.15.106:0/3958361860 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed00036440 con 0x7fed0415d7f0 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fecfbfff640 1 -- 172.21.15.106:0/3958361860 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed000367d0 con 0x7fed0415d7f0 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 -- 172.21.15.106:0/3958361860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0415d7f0 msgr2=0x7fed040c5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0415d7f0 0x7fed040c5990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fed0406ea50 tx=0x7fed0002e0a0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 -- 172.21.15.106:0/3958361860 shutdown_connections 2023-11-24T08:12:30.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed0406d600 0x7fed040cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/3958361860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0415d7f0 0x7fed040c5990 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 -- 172.21.15.106:0/3958361860 >> 172.21.15.106:0/3958361860 conn(0x7fed040b0c50 msgr2=0x7fed040b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:30.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 -- 172.21.15.106:0/3958361860 shutdown_connections 2023-11-24T08:12:30.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.049+0000 7fed0b9a9640 1 -- 172.21.15.106:0/3958361860 wait complete. 2023-11-24T08:12:30.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.051+0000 7fed0b9a9640 1 Processor -- start 2023-11-24T08:12:30.053 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.051+0000 7fed0b9a9640 1 -- start start 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed0b9a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 0x7fed0414de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed0b9a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed040c4220 0x7fed0409c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed0b9a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0415d7f0 0x7fed0409a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed0b9a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed04099cd0 con 0x7fed0406d600 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed0b9a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed040ca890 con 0x7fed0415d7f0 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed0b9a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed040cab70 con 0x7fed040c4220 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed09745640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 0x7fed0414de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed09745640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 0x7fed0414de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46524/0 (socket says 172.21.15.106:46524) 2023-11-24T08:12:30.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.052+0000 7fed09745640 1 -- 172.21.15.106:0/2744854474 learned_addr learned my addr 172.21.15.106:0/2744854474 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:30.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed09745640 1 -- 172.21.15.106:0/2744854474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed040c4220 msgr2=0x7fed0409c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed09745640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed040c4220 0x7fed0409c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed09745640 1 -- 172.21.15.106:0/2744854474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0415d7f0 msgr2=0x7fed0409a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed09745640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0415d7f0 0x7fed0409a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed09745640 1 -- 172.21.15.106:0/2744854474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed0000b770 con 0x7fed0406d600 2023-11-24T08:12:30.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed09745640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 0x7fed0414de60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fed0002e0e0 tx=0x7fed000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0003f750 con 0x7fed0406d600 2023-11-24T08:12:30.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0003f8d0 con 0x7fed0406d600 2023-11-24T08:12:30.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0003fc80 con 0x7fed0406d600 2023-11-24T08:12:30.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed040cae50 con 0x7fed0406d600 2023-11-24T08:12:30.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.053+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0414e3a0 con 0x7fed0406d600 2023-11-24T08:12:30.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.054+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed040cb370 con 0x7fed0406d600 2023-11-24T08:12:30.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.055+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7fed000403e0 con 0x7fed0406d600 2023-11-24T08:12:30.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.056+0000 7fecf9ffb640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fece806b230 0x7fece806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.056+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fed000abe20 con 0x7fed0406d600 2023-11-24T08:12:30.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.057+0000 7fed08f44640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fece806b230 0x7fece806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.057+0000 7fed08f44640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fece806b230 0x7fece806d6f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed040c5ea0 tx=0x7fecf40079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.061+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed040cb370 con 0x7fed0406d600 2023-11-24T08:12:30.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.249+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fed040cb370 con 0x7fed0406d600 2023-11-24T08:12:30.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.250+0000 7fecf9ffb640 1 -- 172.21.15.106:0/2744854474 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fed040cb370 con 0x7fed0406d600 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fece806b230 msgr2=0x7fece806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fece806b230 0x7fece806d6f0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fed040c5ea0 tx=0x7fecf40079b0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 msgr2=0x7fed0414de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 0x7fed0414de60 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fed0002e0e0 tx=0x7fed000046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 shutdown_connections 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed0415d7f0 0x7fed0409a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fece806b230 0x7fece806d6f0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed040c4220 0x7fed0409c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 --2- 172.21.15.106:0/2744854474 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed0406d600 0x7fed0414de60 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 >> 172.21.15.106:0/2744854474 conn(0x7fed040b0c50 msgr2=0x7fed040cee40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.253+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 shutdown_connections 2023-11-24T08:12:30.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.254+0000 7fed0b9a9640 1 -- 172.21.15.106:0/2744854474 wait complete. 2023-11-24T08:12:30.275 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp596 2023-11-24T08:12:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.490+0000 7f05293ec640 1 Processor -- start 2023-11-24T08:12:30.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.490+0000 7f05293ec640 1 -- start start 2023-11-24T08:12:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05293ec640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f052415b440 0x7f05240cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05293ec640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05240c59b0 0x7f052406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05293ec640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05240c5dc0 con 0x7f05240cbf20 2023-11-24T08:12:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05293ec640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05240cc3d0 con 0x7f05240c59b0 2023-11-24T08:12:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05293ec640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f052415b820 con 0x7f052415b440 2023-11-24T08:12:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05237fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05240c59b0 0x7f052406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05237fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f05240c59b0 0x7f052406c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46540/0 (socket says 172.21.15.106:46540) 2023-11-24T08:12:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.492+0000 7f05237fe640 1 -- 172.21.15.106:0/616536190 learned_addr learned my addr 172.21.15.106:0/616536190 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.493+0000 7f05237fe640 1 -- 172.21.15.106:0/616536190 >> v2:172.21.15.156:3300/0 conn(0x7f052415b440 msgr2=0x7f05240cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.493+0000 7f05237fe640 1 --2- 172.21.15.106:0/616536190 >> v2:172.21.15.156:3300/0 conn(0x7f052415b440 0x7f05240cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.493+0000 7f05237fe640 1 -- 172.21.15.106:0/616536190 >> v1:172.21.15.156:6789/0 conn(0x7f05240cbf20 legacy=0x7f05240c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:30.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.493+0000 7f05237fe640 1 -- 172.21.15.106:0/616536190 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052415eea0 con 0x7f05240c59b0 2023-11-24T08:12:30.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.494+0000 7f05237fe640 1 --2- 172.21.15.106:0/616536190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f052406c5f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f052406e770 tx=0x7f051c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8b9f67d2f210d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.494+0000 7f0521ffb640 1 -- 172.21.15.106:0/616536190 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c02c9f0 con 0x7f05240c59b0 2023-11-24T08:12:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.494+0000 7f0521ffb640 1 -- 172.21.15.106:0/616536190 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c02cb50 con 0x7f05240c59b0 2023-11-24T08:12:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.494+0000 7f0521ffb640 1 -- 172.21.15.106:0/616536190 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c0343f0 con 0x7f05240c59b0 2023-11-24T08:12:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 -- 172.21.15.106:0/616536190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 msgr2=0x7f052406c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 --2- 172.21.15.106:0/616536190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f052406c5f0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f052406e770 tx=0x7f051c02be50 comp rx=0 tx=0).stop 2023-11-24T08:12:30.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 -- 172.21.15.106:0/616536190 shutdown_connections 2023-11-24T08:12:30.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 --2- 172.21.15.106:0/616536190 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f052406c5f0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 --2- 172.21.15.106:0/616536190 >> v2:172.21.15.156:3300/0 conn(0x7f052415b440 0x7f05240cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 -- 172.21.15.106:0/616536190 >> 172.21.15.106:0/616536190 conn(0x7f05240b0c50 msgr2=0x7f05240b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:30.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 -- 172.21.15.106:0/616536190 shutdown_connections 2023-11-24T08:12:30.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.495+0000 7f05293ec640 1 -- 172.21.15.106:0/616536190 wait complete. 2023-11-24T08:12:30.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.496+0000 7f05293ec640 1 Processor -- start 2023-11-24T08:12:30.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.497+0000 7f05293ec640 1 -- start start 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.497+0000 7f05293ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f05241554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05293ec640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05240cbf20 0x7f05241e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05293ec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f052415b440 0x7f0524153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05293ec640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05241567c0 con 0x7f05240c59b0 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05293ec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0524154f90 con 0x7f052415b440 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05293ec640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0524155bb0 con 0x7f05240cbf20 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f05241554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f05241554b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46552/0 (socket says 172.21.15.106:46552) 2023-11-24T08:12:30.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 -- 172.21.15.106:0/1957160157 learned_addr learned my addr 172.21.15.106:0/1957160157 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05237fe640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f052415b440 0x7f0524153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05227fc640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05240cbf20 0x7f05241e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 -- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05240cbf20 msgr2=0x7f05241e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05240cbf20 0x7f05241e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 -- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f052415b440 msgr2=0x7f0524153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f052415b440 0x7f0524153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f0522ffd640 1 -- 172.21.15.106:0/1957160157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c009b30 con 0x7f05240c59b0 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05237fe640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f052415b440 0x7f0524153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.498+0000 7f05227fc640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05240cbf20 0x7f05241e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.499+0000 7f0522ffd640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f05241554b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f051400b410 tx=0x7f051400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.499+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514011830 con 0x7f05240c59b0 2023-11-24T08:12:30.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.499+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05140119b0 con 0x7f05240c59b0 2023-11-24T08:12:30.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.499+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0514011d60 con 0x7f05240c59b0 2023-11-24T08:12:30.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.499+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0524155e90 con 0x7f05240c59b0 2023-11-24T08:12:30.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.499+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0524156450 con 0x7f05240c59b0 2023-11-24T08:12:30.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.500+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f052404e8b0 con 0x7f05240c59b0 2023-11-24T08:12:30.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.501+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f051401a400 con 0x7f05240c59b0 2023-11-24T08:12:30.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.502+0000 7f050b7fe640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f051006b260 0x7f051006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.502+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0514089f00 con 0x7f05240c59b0 2023-11-24T08:12:30.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.502+0000 7f05227fc640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f051006b260 0x7f051006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.503+0000 7f05227fc640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f051006b260 0x7f051006d720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0518007fd0 tx=0x7f0518009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.507+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0514054990 con 0x7f05240c59b0 2023-11-24T08:12:30.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.705+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp596", "target": ["mon-mgr", ""]}) v1 -- 0x7f052415f3d0 con 0x7f051006b260 2023-11-24T08:12:30.721 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:30.719+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:30.722 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:30.719+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:30.722 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:30.719+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:30.722 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:30.719+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:30.722 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:30.719+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:30.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.738+0000 7f050b7fe640 1 -- 172.21.15.106:0/1957160157 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f05180012c0 con 0x7f051006b260 2023-11-24T08:12:30.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f051006b260 msgr2=0x7f051006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f051006b260 0x7f051006d720 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0518007fd0 tx=0x7f0518009040 comp rx=0 tx=0).stop 2023-11-24T08:12:30.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 msgr2=0x7f05241554b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f05241554b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f051400b410 tx=0x7f051400b8d0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 shutdown_connections 2023-11-24T08:12:30.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f052415b440 0x7f0524153af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f051006b260 0x7f051006d720 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f05240cbf20 0x7f05241e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 --2- 172.21.15.106:0/1957160157 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f05240c59b0 0x7f05241554b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 >> 172.21.15.106:0/1957160157 conn(0x7f05240b0c50 msgr2=0x7f05240ac750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:30.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 shutdown_connections 2023-11-24T08:12:30.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.741+0000 7f05293ec640 1 -- 172.21.15.106:0/1957160157 wait complete. 2023-11-24T08:12:30.760 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol5440 --group_name subvol_grp596 2023-11-24T08:12:30.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.967+0000 7f1f974f8640 1 Processor -- start 2023-11-24T08:12:30.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.968+0000 7f1f974f8640 1 -- start start 2023-11-24T08:12:30.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.969+0000 7f1f974f8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f90159f70 0x7f1f9015a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.969+0000 7f1f974f8640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1f900b3f50 0x7f1f900b4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.969+0000 7f1f974f8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f9015b560 con 0x7f1f900aec80 2023-11-24T08:12:30.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.969+0000 7f1f974f8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b3b30 con 0x7f1f90159f70 2023-11-24T08:12:30.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.969+0000 7f1f974f8640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900b4890 con 0x7f1f900b3f50 2023-11-24T08:12:30.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f90159f70 0x7f1f9015a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1f90159f70 0x7f1f9015a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46576/0 (socket says 172.21.15.106:46576) 2023-11-24T08:12:30.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 -- 172.21.15.106:0/2005892838 learned_addr learned my addr 172.21.15.106:0/2005892838 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:30.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f96cf7640 1 --2- 172.21.15.106:0/2005892838 >> v2:172.21.15.156:3300/0 conn(0x7f1f900b3f50 0x7f1f900b4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 -- 172.21.15.106:0/2005892838 >> v2:172.21.15.156:3300/0 conn(0x7f1f900b3f50 msgr2=0x7f1f900b4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:30.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 --2- 172.21.15.106:0/2005892838 >> v2:172.21.15.156:3300/0 conn(0x7f1f900b3f50 0x7f1f900b4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 -- 172.21.15.106:0/2005892838 >> v1:172.21.15.106:6789/0 conn(0x7f1f900aec80 legacy=0x7f1f9006ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.970+0000 7f1f964f6640 1 -- 172.21.15.106:0/2005892838 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f9015acc0 con 0x7f1f90159f70 2023-11-24T08:12:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.971+0000 7f1f964f6640 1 --2- 172.21.15.106:0/2005892838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015a350 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1f9006f800 tx=0x7f1f8c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=971c10220a60765f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f954f4640 1 -- 172.21.15.106:0/2005892838 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c031870 con 0x7f1f90159f70 2023-11-24T08:12:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f954f4640 1 -- 172.21.15.106:0/2005892838 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8c0319d0 con 0x7f1f90159f70 2023-11-24T08:12:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f954f4640 1 -- 172.21.15.106:0/2005892838 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8c031d60 con 0x7f1f90159f70 2023-11-24T08:12:30.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 -- 172.21.15.106:0/2005892838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 msgr2=0x7f1f9015a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 --2- 172.21.15.106:0/2005892838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015a350 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f1f9006f800 tx=0x7f1f8c02b910 comp rx=0 tx=0).stop 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 -- 172.21.15.106:0/2005892838 shutdown_connections 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 --2- 172.21.15.106:0/2005892838 >> v2:172.21.15.156:3300/0 conn(0x7f1f900b3f50 0x7f1f900b4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 --2- 172.21.15.106:0/2005892838 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015a350 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 -- 172.21.15.106:0/2005892838 >> 172.21.15.106:0/2005892838 conn(0x7f1f90070c80 msgr2=0x7f1f90071060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 -- 172.21.15.106:0/2005892838 shutdown_connections 2023-11-24T08:12:30.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.972+0000 7f1f974f8640 1 -- 172.21.15.106:0/2005892838 wait complete. 2023-11-24T08:12:30.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.974+0000 7f1f974f8640 1 Processor -- start 2023-11-24T08:12:30.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.974+0000 7f1f974f8640 1 -- start start 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f974f8640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f900aec80 0x7f1f90152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f974f8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f900b3f50 0x7f1f901f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f974f8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f974f8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90154340 con 0x7f1f90159f70 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f974f8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f900aa190 con 0x7f1f900b3f50 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f974f8640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f90153cc0 con 0x7f1f900aec80 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f96cf7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.975+0000 7f1f96cf7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46590/0 (socket says 172.21.15.106:46590) 2023-11-24T08:12:30.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 -- 172.21.15.106:0/1887388214 learned_addr learned my addr 172.21.15.106:0/1887388214 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f964f6640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f900aec80 0x7f1f90152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f95cf5640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f900b3f50 0x7f1f901f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 -- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f900aec80 msgr2=0x7f1f90152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f900aec80 0x7f1f90152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 -- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f900b3f50 msgr2=0x7f1f901f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f900b3f50 0x7f1f901f2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 -- 172.21.15.106:0/1887388214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c0094d0 con 0x7f1f90159f70 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f95cf5640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f900b3f50 0x7f1f901f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f96cf7640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015c760 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1f88007e70 tx=0x7f1f8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.976+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88014620 con 0x7f1f90159f70 2023-11-24T08:12:30.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.977+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f880147a0 con 0x7f1f90159f70 2023-11-24T08:12:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.977+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f901565d0 con 0x7f1f90159f70 2023-11-24T08:12:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.977+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f88014b50 con 0x7f1f90159f70 2023-11-24T08:12:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.977+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f901568b0 con 0x7f1f90159f70 2023-11-24T08:12:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.978+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9004e840 con 0x7f1f90159f70 2023-11-24T08:12:30.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.979+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f1f88014cb0 con 0x7f1f90159f70 2023-11-24T08:12:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.980+0000 7f1f86ffd640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f7406b230 0x7f1f7406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:30.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.980+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f8808a6a0 con 0x7f1f90159f70 2023-11-24T08:12:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.980+0000 7f1f964f6640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f7406b230 0x7f1f7406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:30.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.981+0000 7f1f964f6640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f7406b230 0x7f1f7406d6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c036000 tx=0x7f1f8c03c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:30.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:30.984+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1f88055220 con 0x7f1f90159f70 2023-11-24T08:12:31.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.205+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol5440", "group_name": "subvol_grp596", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f9006f070 con 0x7f1f7406b230 2023-11-24T08:12:31.251 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.248+0000 7f1f86ffd640 1 -- 172.21.15.106:0/1887388214 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1f8c031bc0 con 0x7f1f7406b230 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f7406b230 msgr2=0x7f1f7406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f7406b230 0x7f1f7406d6f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c036000 tx=0x7f1f8c03c040 comp rx=0 tx=0).stop 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 msgr2=0x7f1f9015c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015c760 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f1f88007e70 tx=0x7f1f8800e4b0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 shutdown_connections 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1f90159f70 0x7f1f9015c760 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1f900b3f50 0x7f1f901f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1f7406b230 0x7f1f7406d6f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 --2- 172.21.15.106:0/1887388214 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1f900aec80 0x7f1f90152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 >> 172.21.15.106:0/1887388214 conn(0x7f1f90070c80 msgr2=0x7f1f9006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:31.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 shutdown_connections 2023-11-24T08:12:31.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.252+0000 7f1f974f8640 1 -- 172.21.15.106:0/1887388214 wait complete. 2023-11-24T08:12:31.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:31.275 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp596/.snap/ancestral_snap_1) 2023-11-24T08:12:31.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:31.405 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%i ./volumes/subvol_grp596) 2023-11-24T08:12:31.460 INFO:teuthology.orchestra.run.smithi106.stdout:1099511627777 2023-11-24T08:12:31.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:31.461 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls ./volumes/subvol_grp596/subvol5440/.snap/_ancestral_snap_1_1099511627777) 2023-11-24T08:12:31.525 INFO:teuthology.orchestra.run.smithi106.stdout:6dda659e-8758-4b0f-bac8-a414048230c4 2023-11-24T08:12:31.527 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol5440 _ancestral_snap_1_1099511627777 subvol_grp596 2023-11-24T08:12:31.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.741+0000 7f3b89da9640 1 Processor -- start 2023-11-24T08:12:31.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.741+0000 7f3b89da9640 1 -- start start 2023-11-24T08:12:31.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b89da9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3b840cc0f0 0x7f3b840c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:31.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b89da9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f3b840c57e0 0x7f3b8406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:31.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b89da9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b8415eea0 con 0x7f3b8415b440 2023-11-24T08:12:31.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b89da9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b8415b820 con 0x7f3b840cc0f0 2023-11-24T08:12:31.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b89da9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b840abf30 con 0x7f3b840c57e0 2023-11-24T08:12:31.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b82ffd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f3b840cc0f0 0x7f3b840c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:31.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b837fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3b8415b440 0x7f3b840cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38138/0 (socket says 172.21.15.106:38138) 2023-11-24T08:12:31.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.743+0000 7f3b837fe640 1 -- 172.21.15.106:0/2231115941 learned_addr learned my addr 172.21.15.106:0/2231115941 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:31.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.744+0000 7f3b83fff640 1 --2- 172.21.15.106:0/2231115941 >> v2:172.21.15.156:3300/0 conn(0x7f3b840c57e0 0x7f3b8406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:31.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.744+0000 7f3b82ffd640 1 -- 172.21.15.106:0/2231115941 >> v2:172.21.15.156:3300/0 conn(0x7f3b840c57e0 msgr2=0x7f3b8406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:31.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.744+0000 7f3b82ffd640 1 --2- 172.21.15.106:0/2231115941 >> v2:172.21.15.156:3300/0 conn(0x7f3b840c57e0 0x7f3b8406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.744+0000 7f3b82ffd640 1 -- 172.21.15.106:0/2231115941 >> v1:172.21.15.106:6789/0 conn(0x7f3b8415b440 legacy=0x7f3b840cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:31.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.744+0000 7f3b827fc640 1 -- 172.21.15.106:0/2231115941 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3694927220 0 0) 0x7f3b8415eea0 con 0x7f3b8415b440 2023-11-24T08:12:31.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.744+0000 7f3b82ffd640 1 -- 172.21.15.106:0/2231115941 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b8415f0f0 con 0x7f3b840cc0f0 2023-11-24T08:12:31.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.745+0000 7f3b82ffd640 1 --2- 172.21.15.106:0/2231115941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b840cc0f0 0x7f3b840c52a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3b840c95e0 tx=0x7f3b6c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547ef0da61f2cbee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.745+0000 7f3b827fc640 1 -- 172.21.15.106:0/2231115941 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c02e070 con 0x7f3b840cc0f0 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.745+0000 7f3b827fc640 1 -- 172.21.15.106:0/2231115941 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b6c0315a0 con 0x7f3b840cc0f0 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.745+0000 7f3b827fc640 1 -- 172.21.15.106:0/2231115941 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6c031930 con 0x7f3b840cc0f0 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 -- 172.21.15.106:0/2231115941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b840cc0f0 msgr2=0x7f3b840c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 --2- 172.21.15.106:0/2231115941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b840cc0f0 0x7f3b840c52a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f3b840c95e0 tx=0x7f3b6c02b840 comp rx=0 tx=0).stop 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 -- 172.21.15.106:0/2231115941 shutdown_connections 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 --2- 172.21.15.106:0/2231115941 >> v2:172.21.15.156:3300/0 conn(0x7f3b840c57e0 0x7f3b8406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 --2- 172.21.15.106:0/2231115941 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b840cc0f0 0x7f3b840c52a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 -- 172.21.15.106:0/2231115941 >> 172.21.15.106:0/2231115941 conn(0x7f3b840b0c50 msgr2=0x7f3b840b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 -- 172.21.15.106:0/2231115941 shutdown_connections 2023-11-24T08:12:31.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.746+0000 7f3b89da9640 1 -- 172.21.15.106:0/2231115941 wait complete. 2023-11-24T08:12:31.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.748+0000 7f3b89da9640 1 Processor -- start 2023-11-24T08:12:31.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.748+0000 7f3b89da9640 1 -- start start 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b89da9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b840c57e0 0x7f3b84154cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b89da9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b840cc0f0 0x7f3b84159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b89da9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b8415b440 0x7f3b84159fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b89da9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840ccbd0 con 0x7f3b8415b440 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b89da9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840cceb0 con 0x7f3b840c57e0 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b89da9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b840cd190 con 0x7f3b840cc0f0 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b8415b440 0x7f3b84159fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b8415b440 0x7f3b84159fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46602/0 (socket says 172.21.15.106:46602) 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 -- 172.21.15.106:0/257053608 learned_addr learned my addr 172.21.15.106:0/257053608 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b837fe640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b840c57e0 0x7f3b84154cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b82ffd640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b840cc0f0 0x7f3b84159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:31.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 -- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b840cc0f0 msgr2=0x7f3b84159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b840cc0f0 0x7f3b84159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 -- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b840c57e0 msgr2=0x7f3b84154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b840c57e0 0x7f3b84154cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b83fff640 1 -- 172.21.15.106:0/257053608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c009400 con 0x7f3b8415b440 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b82ffd640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3b840cc0f0 0x7f3b84159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.749+0000 7f3b837fe640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b840c57e0 0x7f3b84154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.750+0000 7f3b83fff640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3b8415b440 0x7f3b84159fc0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00d800 tx=0x7f3b7c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.750+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00b6c0 con 0x7f3b8415b440 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.750+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b7c00b840 con 0x7f3b8415b440 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.750+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c00bbf0 con 0x7f3b8415b440 2023-11-24T08:12:31.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.750+0000 7f3b89da9640 1 -- 172.21.15.106:0/257053608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b840cd470 con 0x7f3b8415b440 2023-11-24T08:12:31.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.750+0000 7f3b89da9640 1 -- 172.21.15.106:0/257053608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b84155230 con 0x7f3b8415b440 2023-11-24T08:12:31.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.751+0000 7f3b89da9640 1 -- 172.21.15.106:0/257053608 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b8404e8b0 con 0x7f3b8415b440 2023-11-24T08:12:31.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.753+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f3b7c00bd50 con 0x7f3b8415b440 2023-11-24T08:12:31.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.754+0000 7f3b7bfff640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3b6406b230 0x7f3b6406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:31.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.754+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b7c08a2f0 con 0x7f3b8415b440 2023-11-24T08:12:31.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.754+0000 7f3b837fe640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3b6406b230 0x7f3b6406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:31.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.755+0000 7f3b837fe640 1 --2- 172.21.15.106:0/257053608 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3b6406b230 0x7f3b6406d6f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3b74007fd0 tx=0x7f3b74009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:31.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.759+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3b7c054dc0 con 0x7f3b8415b440 2023-11-24T08:12:31.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.981+0000 7f3b89da9640 1 -- 172.21.15.106:0/257053608 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol5440", "snap_name": "_ancestral_snap_1_1099511627777", "group_name": "subvol_grp596", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b8415f990 con 0x7f3b6406b230 2023-11-24T08:12:31.994 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:31.992+0000 7f67aa1bf640 -1 mgr.server reply reply (22) Invalid argument snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T08:12:31.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:31.992+0000 7f3b7bfff640 1 -- 172.21.15.106:0/257053608 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: -22 snapshot name '_ancestral_snap_1_1099511627777' is invalid) v1 ==== 66+0+0 (secure 0 0 0) 0x7f3b7400d920 con 0x7f3b6406b230 2023-11-24T08:12:31.995 INFO:teuthology.orchestra.run.smithi106.stderr:Error EINVAL: snapshot name '_ancestral_snap_1_1099511627777' is invalid 2023-11-24T08:12:32.008 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-11-24T08:12:32.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:32.009 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo rmdir ./volumes/subvol_grp596/.snap/ancestral_snap_1) 2023-11-24T08:12:32.113 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol5440 --group_name subvol_grp596 2023-11-24T08:12:32.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.328+0000 7f0913d2c640 1 Processor -- start 2023-11-24T08:12:32.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.328+0000 7f0913d2c640 1 -- start start 2023-11-24T08:12:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.330+0000 7f0913d2c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f090c0cf450 0x7f090c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.330+0000 7f0913d2c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c0cece0 con 0x7f090c15c7f0 2023-11-24T08:12:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.330+0000 7f0913d2c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090c06d040 con 0x7f090c0cdfb0 2023-11-24T08:12:32.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.330+0000 7f0913d2c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f090c0cfe40 con 0x7f090c0cf450 2023-11-24T08:12:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.330+0000 7f0911ac8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f090c15c7f0 0x7f090c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38154/0 (socket says 172.21.15.106:38154) 2023-11-24T08:12:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.330+0000 7f0911ac8640 1 -- 172.21.15.106:0/2310254259 learned_addr learned my addr 172.21.15.106:0/2310254259 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f09112c7640 1 --2- 172.21.15.106:0/2310254259 >> v2:172.21.15.156:3300/0 conn(0x7f090c0cf450 0x7f090c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238752637 0 0) 0x7f090c0cece0 con 0x7f090c15c7f0 2023-11-24T08:12:32.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08f8003520 con 0x7f090c15c7f0 2023-11-24T08:12:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102949042 0 0) 0x7f08f8003520 con 0x7f090c15c7f0 2023-11-24T08:12:32.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 >> v2:172.21.15.156:3300/0 conn(0x7f090c0cf450 msgr2=0x7f090c0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 --2- 172.21.15.106:0/2310254259 >> v2:172.21.15.156:3300/0 conn(0x7f090c0cf450 0x7f090c0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 >> v1:172.21.15.156:6789/0 conn(0x7f090c0cdfb0 legacy=0x7f090c0ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c06d320 con 0x7f090c15c7f0 2023-11-24T08:12:32.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f0908001670 con 0x7f090c15c7f0 2023-11-24T08:12:32.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 >> v1:172.21.15.106:6789/0 conn(0x7f090c15c7f0 legacy=0x7f090c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f80047c0 0x7f08f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f8005180 0x7f08f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0cfe40 con 0x7f08f8005180 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08f800c020 con 0x7f08f8003970 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.331+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08f800c1e0 con 0x7f08f80047c0 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09122c9640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f8005180 0x7f08f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09112c7640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f80047c0 0x7f08f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09122c9640 1 -- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f80047c0 msgr2=0x7f08f8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09122c9640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f80047c0 0x7f08f8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f0911ac8640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09122c9640 1 -- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8003970 msgr2=0x7f08f8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09122c9640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f09122c9640 1 -- 172.21.15.106:0/2310254259 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8003520 con 0x7f08f8005180 2023-11-24T08:12:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.332+0000 7f0911ac8640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f08f8003970 0x7f08f8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.333+0000 7f09122c9640 1 -- 172.21.15.106:0/2310254259 reap_dead start 2023-11-24T08:12:32.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.333+0000 7f09122c9640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f8005180 0x7f08f800b980 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0904001e50 tx=0x7f08f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2d3dcdd7b4f61c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:32.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904003b70 con 0x7f08f8005180 2023-11-24T08:12:32.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0904003d10 con 0x7f08f8005180 2023-11-24T08:12:32.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0910ac6640 1 -- 172.21.15.106:0/2310254259 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0904005310 con 0x7f08f8005180 2023-11-24T08:12:32.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 -- 172.21.15.106:0/2310254259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f8005180 msgr2=0x7f08f800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.345 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f8005180 0x7f08f800b980 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f0904001e50 tx=0x7f08f80057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 -- 172.21.15.106:0/2310254259 shutdown_connections 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 --2- 172.21.15.106:0/2310254259 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f08f8005180 0x7f08f800b980 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 -- 172.21.15.106:0/2310254259 >> 172.21.15.106:0/2310254259 conn(0x7f090c06a920 msgr2=0x7f090c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 -- 172.21.15.106:0/2310254259 shutdown_connections 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.334+0000 7f0913d2c640 1 -- 172.21.15.106:0/2310254259 wait complete. 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.336+0000 7f0913d2c640 1 Processor -- start 2023-11-24T08:12:32.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.336+0000 7f0913d2c640 1 -- start start 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f0913d2c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f8005180 0x7f090c071970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f0913d2c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 0x7f090c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f0913d2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0b3770 0x7f090c1596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f0913d2c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c072330 con 0x7f090c071eb0 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f0913d2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f090c152e40 con 0x7f090c0b3770 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f0913d2c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f090c0b2f40 con 0x7f08f8005180 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f09112c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 0x7f090c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f09112c7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 0x7f090c0b3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46612/0 (socket says 172.21.15.106:46612) 2023-11-24T08:12:32.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.337+0000 7f09112c7640 1 -- 172.21.15.106:0/3932908398 learned_addr learned my addr 172.21.15.106:0/3932908398 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:32.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f0911ac8640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f8005180 0x7f090c071970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09122c9640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0b3770 0x7f090c1596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09112c7640 1 -- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f8005180 msgr2=0x7f090c071970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09112c7640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f8005180 0x7f090c071970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09112c7640 1 -- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0b3770 msgr2=0x7f090c1596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.348 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09112c7640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0b3770 0x7f090c1596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09112c7640 1 -- 172.21.15.106:0/3932908398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f090c0b3fb0 con 0x7f090c071eb0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f0911ac8640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f8005180 0x7f090c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09112c7640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 0x7f090c0b3230 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f08fc011540 tx=0x7f08fc011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc015930 con 0x7f090c071eb0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08fc015a90 con 0x7f090c071eb0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08fc015e60 con 0x7f090c071eb0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.338+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f090c0b2180 con 0x7f090c071eb0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.339+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f090c0b2740 con 0x7f090c071eb0 2023-11-24T08:12:32.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.340+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090c067470 con 0x7f090c071eb0 2023-11-24T08:12:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.341+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 10) v1 ==== 89588+0+0 (secure 0 0 0) 0x7f08fc01d400 con 0x7f090c071eb0 2023-11-24T08:12:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.342+0000 7f09027fc640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08f006b260 0x7f08f006d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.342+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08fc08a4c0 con 0x7f090c071eb0 2023-11-24T08:12:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.342+0000 7f0911ac8640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08f006b260 0x7f08f006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.343+0000 7f0911ac8640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08f006b260 0x7f08f006d720 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0908001eb0 tx=0x7f0908009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:32.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.346+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f08fc054f90 con 0x7f090c071eb0 2023-11-24T08:12:32.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.395+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f08fc055360 con 0x7f090c071eb0 2023-11-24T08:12:32.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.567+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol5440", "group_name": "subvol_grp596", "target": ["mon-mgr", ""]}) v1 -- 0x7f090c0cd730 con 0x7f08f006b260 2023-11-24T08:12:32.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.603+0000 7f09027fc640 1 -- 172.21.15.106:0/3932908398 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f090c0cd730 con 0x7f08f006b260 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08f006b260 msgr2=0x7f08f006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08f006b260 0x7f08f006d720 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0908001eb0 tx=0x7f0908009040 comp rx=0 tx=0).stop 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 msgr2=0x7f090c0b3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 0x7f090c0b3230 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f08fc011540 tx=0x7f08fc011a00 comp rx=0 tx=0).stop 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 shutdown_connections 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f090c0b3770 0x7f090c1596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f090c071eb0 0x7f090c0b3230 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f08f006b260 0x7f08f006d720 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 --2- 172.21.15.106:0/3932908398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f08f8005180 0x7f090c071970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 >> 172.21.15.106:0/3932908398 conn(0x7f090c06a920 msgr2=0x7f090c15db90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:32.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 shutdown_connections 2023-11-24T08:12:32.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.606+0000 7f0913d2c640 1 -- 172.21.15.106:0/3932908398 wait complete. 2023-11-24T08:12:32.615 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.613+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.615 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.613+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.615 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.613+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.615 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.613+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.616 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.613+0000 7f67ac1c3640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:32.629 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2023-11-24T08:12:32.634 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.631+0000 7f67ab9c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.635 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.631+0000 7f67ab9c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.635 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.631+0000 7f67ab9c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.635 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.631+0000 7f67ab9c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.635 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:12:32.631+0000 7f67ab9c2640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:12:32.678 INFO:teuthology.orchestra.run.smithi106.stdout:2 2023-11-24T08:12:32.681 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp596 2023-11-24T08:12:32.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.871+0000 7efe15516640 1 Processor -- start 2023-11-24T08:12:32.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.872+0000 7efe15516640 1 -- start start 2023-11-24T08:12:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.873+0000 7efe15516640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efe1015b440 0x7efe100cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.873+0000 7efe15516640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe100c5dc0 con 0x7efe100cbf20 2023-11-24T08:12:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.873+0000 7efe15516640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe1015eea0 con 0x7efe100c59b0 2023-11-24T08:12:32.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.873+0000 7efe15516640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe100cc3d0 con 0x7efe1015b440 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe07fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7efe100cbf20 0x7efe100c52a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38166/0 (socket says 172.21.15.106:38166) 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe07fff640 1 -- 172.21.15.106:0/702461162 learned_addr learned my addr 172.21.15.106:0/702461162 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe0effd640 1 --2- 172.21.15.106:0/702461162 >> v2:172.21.15.106:3300/0 conn(0x7efe1015b440 0x7efe100cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe0e7fc640 1 -- 172.21.15.106:0/702461162 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 701227650 0 0) 0x7efe100c5dc0 con 0x7efe100cbf20 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe0e7fc640 1 -- 172.21.15.106:0/702461162 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdfc003520 con 0x7efe100cbf20 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe0effd640 1 -- 172.21.15.106:0/702461162 >> v1:172.21.15.156:6789/0 conn(0x7efe100c59b0 legacy=0x7efe1006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.877 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe0effd640 1 -- 172.21.15.106:0/702461162 >> v1:172.21.15.106:6789/0 conn(0x7efe100cbf20 legacy=0x7efe100c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.874+0000 7efe0effd640 1 -- 172.21.15.106:0/702461162 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe100abf30 con 0x7efe1015b440 2023-11-24T08:12:32.878 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.876+0000 7efe0effd640 1 --2- 172.21.15.106:0/702461162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe1015b440 0x7efe100cb9e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efe000097a0 tx=0x7efe0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fcf1f31ef119c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.876+0000 7efe0e7fc640 1 -- 172.21.15.106:0/702461162 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00031830 con 0x7efe1015b440 2023-11-24T08:12:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.876+0000 7efe0e7fc640 1 -- 172.21.15.106:0/702461162 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe00031990 con 0x7efe1015b440 2023-11-24T08:12:32.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.876+0000 7efe0e7fc640 1 -- 172.21.15.106:0/702461162 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00031d20 con 0x7efe1015b440 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 -- 172.21.15.106:0/702461162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe1015b440 msgr2=0x7efe100cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 --2- 172.21.15.106:0/702461162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe1015b440 0x7efe100cb9e0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7efe000097a0 tx=0x7efe0002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 -- 172.21.15.106:0/702461162 shutdown_connections 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 --2- 172.21.15.106:0/702461162 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe1015b440 0x7efe100cb9e0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 -- 172.21.15.106:0/702461162 >> 172.21.15.106:0/702461162 conn(0x7efe100b0c50 msgr2=0x7efe100b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 -- 172.21.15.106:0/702461162 shutdown_connections 2023-11-24T08:12:32.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.877+0000 7efe15516640 1 -- 172.21.15.106:0/702461162 wait complete. 2023-11-24T08:12:32.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.879+0000 7efe15516640 1 Processor -- start 2023-11-24T08:12:32.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.879+0000 7efe15516640 1 -- start start 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe15516640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 0x7efe100c8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe15516640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe100cbf20 0x7efe101f2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe15516640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe1015b440 0x7efe101f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe15516640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1015dca0 con 0x7efe100c59b0 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe15516640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c64c0 con 0x7efe100cbf20 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe15516640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe100c67a0 con 0x7efe1015b440 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe0effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 0x7efe100c8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe0effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 0x7efe100c8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46664/0 (socket says 172.21.15.106:46664) 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.880+0000 7efe0effd640 1 -- 172.21.15.106:0/2333142663 learned_addr learned my addr 172.21.15.106:0/2333142663 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe0effd640 1 -- 172.21.15.106:0/2333142663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe1015b440 msgr2=0x7efe101f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe0effd640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe1015b440 0x7efe101f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe0effd640 1 -- 172.21.15.106:0/2333142663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe100cbf20 msgr2=0x7efe101f2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe0effd640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe100cbf20 0x7efe101f2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe0effd640 1 -- 172.21.15.106:0/2333142663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe00009400 con 0x7efe100c59b0 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe0effd640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 0x7efe100c8760 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efe00034040 tx=0x7efe000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0003a8f0 con 0x7efe100c59b0 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.881+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0003aa70 con 0x7efe100c59b0 2023-11-24T08:12:32.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.882+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0003ae20 con 0x7efe100c59b0 2023-11-24T08:12:32.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.882+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe100c6a80 con 0x7efe100c59b0 2023-11-24T08:12:32.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.882+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe100c7040 con 0x7efe100c59b0 2023-11-24T08:12:32.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.883+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1004e8b0 con 0x7efe100c59b0 2023-11-24T08:12:32.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.884+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7efe0004d440 con 0x7efe100c59b0 2023-11-24T08:12:32.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.885+0000 7efe077fe640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efdf006b2c0 0x7efdf006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:32.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.885+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efe000ac060 con 0x7efe100c59b0 2023-11-24T08:12:32.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.885+0000 7efe07fff640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efdf006b2c0 0x7efdf006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:32.888 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.885+0000 7efe07fff640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efdf006b2c0 0x7efdf006d780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efe10071210 tx=0x7efdf8007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:32.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:32.889+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efe000dee90 con 0x7efe100c59b0 2023-11-24T08:12:33.106 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.103+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp596", "target": ["mon-mgr", ""]}) v1 -- 0x7efe1015f3d0 con 0x7efdf006b2c0 2023-11-24T08:12:33.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.110+0000 7efe077fe640 1 -- 172.21.15.106:0/2333142663 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efdf80058b0 con 0x7efdf006b2c0 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efdf006b2c0 msgr2=0x7efdf006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efdf006b2c0 0x7efdf006d780 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7efe10071210 tx=0x7efdf8007930 comp rx=0 tx=0).stop 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 msgr2=0x7efe100c8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 0x7efe100c8760 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7efe00034040 tx=0x7efe000046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 shutdown_connections 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efdf006b2c0 0x7efdf006d780 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efe1015b440 0x7efe101f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efe100cbf20 0x7efe101f2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 --2- 172.21.15.106:0/2333142663 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efe100c59b0 0x7efe100c8760 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 >> 172.21.15.106:0/2333142663 conn(0x7efe100b0c50 msgr2=0x7efe100ac1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:33.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 shutdown_connections 2023-11-24T08:12:33.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.113+0000 7efe15516640 1 -- 172.21.15.106:0/2333142663 wait complete. 2023-11-24T08:12:33.134 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:12:33.134 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:12:33.266 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:12:33.393 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:12:33.393 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:33.394 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:33.453 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:33.453 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:33.506 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:33.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.720+0000 7f826591d640 1 Processor -- start 2023-11-24T08:12:33.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.721+0000 7f826591d640 1 -- start start 2023-11-24T08:12:33.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.722+0000 7f826591d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f826015b080 0x7f826015b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:33.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.722+0000 7f826591d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826015e3f0 con 0x7f82600b2250 2023-11-24T08:12:33.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.722+0000 7f826591d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826015ab80 con 0x7f826015df80 2023-11-24T08:12:33.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.722+0000 7f826591d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826015a990 con 0x7f826015b080 2023-11-24T08:12:33.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f82600b2250 0x7f82600b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:38180/0 (socket says 172.21.15.106:38180) 2023-11-24T08:12:33.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825effd640 1 -- 172.21.15.106:0/2274118043 learned_addr learned my addr 172.21.15.106:0/2274118043 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:33.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825f7fe640 1 --2- 172.21.15.106:0/2274118043 >> v2:172.21.15.106:3300/0 conn(0x7f826015b080 0x7f826015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:33.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825e7fc640 1 -- 172.21.15.106:0/2274118043 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326675050 0 0) 0x7f826015e3f0 con 0x7f82600b2250 2023-11-24T08:12:33.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825e7fc640 1 -- 172.21.15.106:0/2274118043 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f824c003520 con 0x7f82600b2250 2023-11-24T08:12:33.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825f7fe640 1 -- 172.21.15.106:0/2274118043 >> v1:172.21.15.156:6789/0 conn(0x7f826015df80 legacy=0x7f82600ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825f7fe640 1 -- 172.21.15.106:0/2274118043 >> v1:172.21.15.106:6789/0 conn(0x7f82600b2250 legacy=0x7f82600b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.723+0000 7f825f7fe640 1 -- 172.21.15.106:0/2274118043 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82600b2d40 con 0x7f826015b080 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.724+0000 7f825f7fe640 1 --2- 172.21.15.106:0/2274118043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015b080 0x7f826015b490 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8258009d90 tx=0x7f825802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac56a802293a97b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f825e7fc640 1 -- 172.21.15.106:0/2274118043 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825802cd00 con 0x7f826015b080 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f825e7fc640 1 -- 172.21.15.106:0/2274118043 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825802ce60 con 0x7f826015b080 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f825e7fc640 1 -- 172.21.15.106:0/2274118043 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8258034660 con 0x7f826015b080 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 -- 172.21.15.106:0/2274118043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015b080 msgr2=0x7f826015b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 --2- 172.21.15.106:0/2274118043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015b080 0x7f826015b490 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f8258009d90 tx=0x7f825802bf50 comp rx=0 tx=0).stop 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 -- 172.21.15.106:0/2274118043 shutdown_connections 2023-11-24T08:12:33.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 --2- 172.21.15.106:0/2274118043 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015b080 0x7f826015b490 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 -- 172.21.15.106:0/2274118043 >> 172.21.15.106:0/2274118043 conn(0x7f826006a920 msgr2=0x7f826006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:33.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 -- 172.21.15.106:0/2274118043 shutdown_connections 2023-11-24T08:12:33.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.725+0000 7f826591d640 1 -- 172.21.15.106:0/2274118043 wait complete. 2023-11-24T08:12:33.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.727+0000 7f826591d640 1 Processor -- start 2023-11-24T08:12:33.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.727+0000 7f826591d640 1 -- start start 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f826591d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82600b2250 0x7f8260153930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f826591d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826015b080 0x7f82601f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f826591d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 0x7f82600adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f826591d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260071400 con 0x7f826015df80 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f826591d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8260153e70 con 0x7f826015b080 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f826591d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8260070aa0 con 0x7f82600b2250 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f825f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 0x7f82600adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f825f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 0x7f82600adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46690/0 (socket says 172.21.15.106:46690) 2023-11-24T08:12:33.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.728+0000 7f825f7fe640 1 -- 172.21.15.106:0/695220903 learned_addr learned my addr 172.21.15.106:0/695220903 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:33.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825f7fe640 1 -- 172.21.15.106:0/695220903 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82600b2250 msgr2=0x7f8260153930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825f7fe640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82600b2250 0x7f8260153930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825f7fe640 1 -- 172.21.15.106:0/695220903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826015b080 msgr2=0x7f82601f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825f7fe640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826015b080 0x7f82601f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825f7fe640 1 -- 172.21.15.106:0/695220903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8258009a70 con 0x7f826015df80 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825f7fe640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 0x7f82600adb70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8258009d90 tx=0x7f8258004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825803d600 con 0x7f826015df80 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f825803d780 con 0x7f826015df80 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f825803db30 con 0x7f826015df80 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.729+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8260070d40 con 0x7f826015df80 2023-11-24T08:12:33.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.730+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82600ae550 con 0x7f826015df80 2023-11-24T08:12:33.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.731+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8260071540 con 0x7f826015df80 2023-11-24T08:12:33.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.732+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f825803b040 con 0x7f826015df80 2023-11-24T08:12:33.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.732+0000 7f825cff9640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f824006b2c0 0x7f824006d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:33.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.733+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f82580ac270 con 0x7f826015df80 2023-11-24T08:12:33.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.733+0000 7f825effd640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f824006b2c0 0x7f824006d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:33.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.733+0000 7f825effd640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f824006b2c0 0x7f824006d780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f826009d310 tx=0x7f8250007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:33.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.737+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8258076cf0 con 0x7f826015df80 2023-11-24T08:12:33.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.929+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8260067470 con 0x7f826015df80 2023-11-24T08:12:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.930+0000 7f825cff9640 1 -- 172.21.15.106:0/695220903 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16616 (secure 0 0 0) 0x7f8258009450 con 0x7f826015df80 2023-11-24T08:12:33.933 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:33.933 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":75,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:11.536568+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:33.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f824006b2c0 msgr2=0x7f824006d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f824006b2c0 0x7f824006d780 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f826009d310 tx=0x7f8250007930 comp rx=0 tx=0).stop 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 msgr2=0x7f82600adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 0x7f82600adb70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f8258009d90 tx=0x7f8258004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 shutdown_connections 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f826015df80 0x7f82600adb70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f826015b080 0x7f82601f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f824006b2c0 0x7f824006d780 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 --2- 172.21.15.106:0/695220903 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82600b2250 0x7f8260153930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 >> 172.21.15.106:0/695220903 conn(0x7f826006a920 msgr2=0x7f82600cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 shutdown_connections 2023-11-24T08:12:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:33.933+0000 7f826591d640 1 -- 172.21.15.106:0/695220903 wait complete. 2023-11-24T08:12:33.955 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:12:07.722409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:12:08.984297+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:33.955 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:34.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.162+0000 7f80b95ea640 1 Processor -- start 2023-11-24T08:12:34.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.162+0000 7f80b95ea640 1 -- start start 2023-11-24T08:12:34.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.163+0000 7f80b95ea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f80b40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b95ea640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b40cd510 con 0x7f80b40cd110 2023-11-24T08:12:34.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b95ea640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b4161a10 con 0x7f80b4146eb0 2023-11-24T08:12:34.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b95ea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4161cf0 con 0x7f80b406e160 2023-11-24T08:12:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f80b40cd110 0x7f80b4070e40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42792/0 (socket says 172.21.15.106:42792) 2023-11-24T08:12:34.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b37fe640 1 -- 172.21.15.106:0/2075903963 learned_addr learned my addr 172.21.15.106:0/2075903963 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:34.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1136703125 0 0) 0x7f80b40cd510 con 0x7f80b40cd110 2023-11-24T08:12:34.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.164+0000 7f80b27fc640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f80b40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f809c003520 con 0x7f80b40cd110 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939497720 0 0) 0x7f809c003520 con 0x7f80b40cd110 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 msgr2=0x7f80b40ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f80b40ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 >> v1:172.21.15.156:6789/0 conn(0x7f80b4146eb0 legacy=0x7f80b4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b4161fd0 con 0x7f80b40cd110 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f80ac0019f0 con 0x7f80b40cd110 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b27fc640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f80b40ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 >> v1:172.21.15.106:6789/0 conn(0x7f80b40cd110 legacy=0x7f80b4070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0039a0 0x7f809c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f809c0047c0 0x7f809c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f809c0051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b4161cf0 con 0x7f809c0047c0 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809c00bb10 con 0x7f809c0039a0 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809c00bc80 con 0x7f80b406e160 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.165+0000 7f80b27fc640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f809c0047c0 0x7f809c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b2ffd640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0039a0 0x7f809c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b37fe640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f809c0051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b27fc640 1 -- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 msgr2=0x7f809c0051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b27fc640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f809c0051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b27fc640 1 -- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0039a0 msgr2=0x7f809c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b27fc640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0039a0 0x7f809c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b27fc640 1 -- 172.21.15.106:0/2075903963 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f809c003520 con 0x7f809c0047c0 2023-11-24T08:12:34.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.166+0000 7f80b37fe640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b406e160 0x7f809c0051d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.167+0000 7f80b27fc640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f809c0047c0 0x7f809c004c40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f80a40098a0 tx=0x7f809c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963c56e5cdeb3622 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.167+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4004230 con 0x7f809c0047c0 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.167+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4004390 con 0x7f809c0047c0 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.167+0000 7f80b1ffb640 1 -- 172.21.15.106:0/2075903963 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4003830 con 0x7f809c0047c0 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b95ea640 1 -- 172.21.15.106:0/2075903963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f809c0047c0 msgr2=0x7f809c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b95ea640 1 --2- 172.21.15.106:0/2075903963 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f809c0047c0 0x7f809c004c40 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f80a40098a0 tx=0x7f809c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b37fe640 1 -- 172.21.15.106:0/2075903963 reap_dead start 2023-11-24T08:12:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b95ea640 1 -- 172.21.15.106:0/2075903963 shutdown_connections 2023-11-24T08:12:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b95ea640 1 -- 172.21.15.106:0/2075903963 >> 172.21.15.106:0/2075903963 conn(0x7f80b40b0c50 msgr2=0x7f80b40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b95ea640 1 -- 172.21.15.106:0/2075903963 shutdown_connections 2023-11-24T08:12:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.168+0000 7f80b95ea640 1 -- 172.21.15.106:0/2075903963 wait complete. 2023-11-24T08:12:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.170+0000 7f80b95ea640 1 Processor -- start 2023-11-24T08:12:34.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.170+0000 7f80b95ea640 1 -- start start 2023-11-24T08:12:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b95ea640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0047c0 0x7f80b415d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b95ea640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 0x7f80b4099e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b95ea640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b415ef50 0x7f80b40acf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b95ea640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b41603b0 con 0x7f80b4160840 2023-11-24T08:12:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b95ea640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b415f7a0 con 0x7f809c0047c0 2023-11-24T08:12:34.176 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b95ea640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b415fa80 con 0x7f80b415ef50 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 0x7f80b4099e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 0x7f80b4099e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36328/0 (socket says 172.21.15.106:36328) 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b27fc640 1 -- 172.21.15.106:0/494107698 learned_addr learned my addr 172.21.15.106:0/494107698 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.171+0000 7f80b2ffd640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0047c0 0x7f80b415d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b37fe640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b415ef50 0x7f80b40acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b27fc640 1 -- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b415ef50 msgr2=0x7f80b40acf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b27fc640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b415ef50 0x7f80b40acf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b27fc640 1 -- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0047c0 msgr2=0x7f80b415d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b27fc640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0047c0 0x7f80b415d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b27fc640 1 -- 172.21.15.106:0/494107698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b415fd60 con 0x7f80b4160840 2023-11-24T08:12:34.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b2ffd640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0047c0 0x7f80b415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b27fc640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 0x7f80b4099e10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f80a402bea0 tx=0x7f80a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4030610 con 0x7f80b4160840 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4030770 con 0x7f80b4160840 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4030b20 con 0x7f80b4160840 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.172+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b4160040 con 0x7f80b4160840 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.173+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b40ad4c0 con 0x7f80b4160840 2023-11-24T08:12:34.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.174+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b404e8b0 con 0x7f80b4160840 2023-11-24T08:12:34.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.175+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 11) v1 ==== 89639+0+0 (secure 0 0 0) 0x7f80a4030c80 con 0x7f80b4160840 2023-11-24T08:12:34.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.175+0000 7f809b7fe640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80a006b2f0 0x7f80a006d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.176+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80a40ab680 con 0x7f80b4160840 2023-11-24T08:12:34.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.176+0000 7f80b2ffd640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80a006b2f0 0x7f80a006d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.176+0000 7f80b2ffd640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80a006b2f0 0x7f80a006d7b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f80a8000c00 tx=0x7f80a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:34.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.180+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f80a4076070 con 0x7f80b4160840 2023-11-24T08:12:34.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.359+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f80a4076440 con 0x7f80b4160840 2023-11-24T08:12:34.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.373+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f80b406ded0 con 0x7f80b4160840 2023-11-24T08:12:34.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.374+0000 7f809b7fe640 1 -- 172.21.15.106:0/494107698 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f80a408da80 con 0x7f80b4160840 2023-11-24T08:12:34.377 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80a006b2f0 msgr2=0x7f80a006d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80a006b2f0 0x7f80a006d7b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f80a8000c00 tx=0x7f80a8009040 comp rx=0 tx=0).stop 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 msgr2=0x7f80b4099e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 0x7f80b4099e10 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f80a402bea0 tx=0x7f80a4004730 comp rx=0 tx=0).stop 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 shutdown_connections 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f80a006b2f0 0x7f80a006d7b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f80b415ef50 0x7f80b40acf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f80b4160840 0x7f80b4099e10 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 --2- 172.21.15.106:0/494107698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f809c0047c0 0x7f80b415d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 >> 172.21.15.106:0/494107698 conn(0x7f80b40b0c50 msgr2=0x7f80b4161150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:34.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 shutdown_connections 2023-11-24T08:12:34.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.377+0000 7f80b95ea640 1 -- 172.21.15.106:0/494107698 wait complete. 2023-11-24T08:12:34.398 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:12:34.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.606+0000 7fc2f1661640 1 Processor -- start 2023-11-24T08:12:34.609 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.606+0000 7fc2f1661640 1 -- start start 2023-11-24T08:12:34.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.607+0000 7fc2f1661640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc2ec15b440 0x7fc2ec0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2f1661640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 0x7fc2ec0c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2f1661640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2ec15eea0 con 0x7fc2ec0c57e0 2023-11-24T08:12:34.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2f1661640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cc3d0 con 0x7fc2ec15b440 2023-11-24T08:12:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2f1661640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec15b820 con 0x7fc2ec0cbf20 2023-11-24T08:12:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2eb7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc2ec0c57e0 0x7fc2ec0c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42802/0 (socket says 172.21.15.106:42802) 2023-11-24T08:12:34.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2eb7fe640 1 -- 172.21.15.106:0/2525949635 learned_addr learned my addr 172.21.15.106:0/2525949635 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:34.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2ea7fc640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 0x7fc2ec0c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/2525949635 >> v2:172.21.15.156:3300/0 conn(0x7fc2ec15b440 0x7fc2ec0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1974474750 0 0) 0x7fc2ec15eea0 con 0x7fc2ec0c57e0 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.608+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2d4003520 con 0x7fc2ec0c57e0 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4179673285 0 0) 0x7fc2d4003520 con 0x7fc2ec0c57e0 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 msgr2=0x7fc2ec0c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 0x7fc2ec0c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 >> v2:172.21.15.156:3300/0 conn(0x7fc2ec15b440 msgr2=0x7fc2ec0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 --2- 172.21.15.106:0/2525949635 >> v2:172.21.15.156:3300/0 conn(0x7fc2ec15b440 0x7fc2ec0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec06d600 con 0x7fc2ec0c57e0 2023-11-24T08:12:34.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc2e40019f0 con 0x7fc2ec0c57e0 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 >> v1:172.21.15.106:6789/0 conn(0x7fc2ec0c57e0 legacy=0x7fc2ec0c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2d4003970 0x7fc2d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d40047c0 0x7fc2d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 0x7fc2d4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cc3d0 con 0x7fc2d4003970 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec15b820 con 0x7fc2d40047c0 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d400b9b0 con 0x7fc2ec0cbf20 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.609+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2d4003970 0x7fc2d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.610+0000 7fc2ea7fc640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d40047c0 0x7fc2d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.610+0000 7fc2eaffd640 1 -- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 msgr2=0x7fc2d4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.610+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2ec0cbf20 0x7fc2d4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.610+0000 7fc2eaffd640 1 -- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d40047c0 msgr2=0x7fc2d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.610+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2d40047c0 0x7fc2d4004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.610+0000 7fc2eaffd640 1 -- 172.21.15.106:0/2525949635 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d4003520 con 0x7fc2d4003970 2023-11-24T08:12:34.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2d4003970 0x7fc2d4003d90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009ac0 tx=0x7fc2d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688eeefedd9f048f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:34.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc02c6d0 con 0x7fc2d4003970 2023-11-24T08:12:34.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc02c830 con 0x7fc2d4003970 2023-11-24T08:12:34.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2e9ffb640 1 -- 172.21.15.106:0/2525949635 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc02cbc0 con 0x7fc2d4003970 2023-11-24T08:12:34.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2f1661640 1 -- 172.21.15.106:0/2525949635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2d4003970 msgr2=0x7fc2d4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2f1661640 1 --2- 172.21.15.106:0/2525949635 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2d4003970 0x7fc2d4003d90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc009ac0 tx=0x7fc2d40057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.611+0000 7fc2eb7fe640 1 -- 172.21.15.106:0/2525949635 reap_dead start 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.612+0000 7fc2f1661640 1 -- 172.21.15.106:0/2525949635 shutdown_connections 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.612+0000 7fc2f1661640 1 -- 172.21.15.106:0/2525949635 >> 172.21.15.106:0/2525949635 conn(0x7fc2ec0b0c50 msgr2=0x7fc2ec0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.612+0000 7fc2f1661640 1 -- 172.21.15.106:0/2525949635 shutdown_connections 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.612+0000 7fc2f1661640 1 -- 172.21.15.106:0/2525949635 wait complete. 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.613+0000 7fc2f1661640 1 Processor -- start 2023-11-24T08:12:34.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.614+0000 7fc2f1661640 1 -- start start 2023-11-24T08:12:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.614+0000 7fc2f1661640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d4003970 0x7fc2ec15a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.614+0000 7fc2f1661640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2ec155de0 0x7fc2ec0c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.614+0000 7fc2f1661640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 0x7fc2ec09b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2f1661640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec15acc0 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2f1661640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0cdf70 con 0x7fc2ec155de0 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2f1661640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ec0ce250 con 0x7fc2d4003970 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 0x7fc2ec09b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 0x7fc2ec09b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36356/0 (socket says 172.21.15.106:36356) 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 -- 172.21.15.106:0/966425780 learned_addr learned my addr 172.21.15.106:0/966425780 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d4003970 0x7fc2ec15a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2ea7fc640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2ec155de0 0x7fc2ec0c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 -- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d4003970 msgr2=0x7fc2ec15a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d4003970 0x7fc2ec15a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 -- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2ec155de0 msgr2=0x7fc2ec0c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2ec155de0 0x7fc2ec0c6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.615+0000 7fc2eb7fe640 1 -- 172.21.15.106:0/966425780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ec0ce530 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.616+0000 7fc2eb7fe640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 0x7fc2ec09b4d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0cc8d0 tx=0x7fc2e400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.616+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400a610 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.616+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e400a770 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.616+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ec0ce810 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.616+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e400ab20 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.616+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ec0cdc20 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.617+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ec04e8b0 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.618+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fc2e400ac80 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.619+0000 7fc2d37fe640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2d806b3e0 0x7fc2d806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:34.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.619+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc2e4089cb0 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.619+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2d806b3e0 0x7fc2d806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:34.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.620+0000 7fc2eaffd640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2d806b3e0 0x7fc2d806d8a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc036000 tx=0x7fc2dc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:34.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.623+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2e4054540 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.815+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2ec06dc80 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.815+0000 7fc2d37fe640 1 -- 172.21.15.106:0/966425780 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v76) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc2e4057b60 con 0x7fc2ec0c6e40 2023-11-24T08:12:34.818 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:12:34.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2d806b3e0 msgr2=0x7fc2d806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2d806b3e0 0x7fc2d806d8a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc036000 tx=0x7fc2dc030000 comp rx=0 tx=0).stop 2023-11-24T08:12:34.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 msgr2=0x7fc2ec09b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 0x7fc2ec09b4d0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec0cc8d0 tx=0x7fc2e400ce80 comp rx=0 tx=0).stop 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 shutdown_connections 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc2ec0c6e40 0x7fc2ec09b4d0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2ec155de0 0x7fc2ec0c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc2d806b3e0 0x7fc2d806d8a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 --2- 172.21.15.106:0/966425780 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2d4003970 0x7fc2ec15a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:34.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 >> 172.21.15.106:0/966425780 conn(0x7fc2ec0b0c50 msgr2=0x7fc2ec071c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:34.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 shutdown_connections 2023-11-24T08:12:34.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:34.818+0000 7fc2f1661640 1 -- 172.21.15.106:0/966425780 wait complete. 2023-11-24T08:12:34.840 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:12:35.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.054+0000 7fba281e5640 1 Processor -- start 2023-11-24T08:12:35.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.054+0000 7fba281e5640 1 -- start start 2023-11-24T08:12:35.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba281e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba200c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba281e5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fba200c57e0 0x7fba2006c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba281e5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba2015eea0 con 0x7fba2015b440 2023-11-24T08:12:35.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba281e5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba2015b820 con 0x7fba200c57e0 2023-11-24T08:12:35.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba281e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba200abf30 con 0x7fba200cc0f0 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba25f81640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fba2015b440 0x7fba200cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42816/0 (socket says 172.21.15.106:42816) 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba25f81640 1 -- 172.21.15.106:0/1998453224 learned_addr learned my addr 172.21.15.106:0/1998453224 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba26782640 1 --2- 172.21.15.106:0/1998453224 >> v2:172.21.15.106:3300/0 conn(0x7fba200c57e0 0x7fba2006c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba24f7f640 1 -- 172.21.15.106:0/1998453224 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159215106 0 0) 0x7fba2015eea0 con 0x7fba2015b440 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.056+0000 7fba25780640 1 --2- 172.21.15.106:0/1998453224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba200c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.057+0000 7fba24f7f640 1 -- 172.21.15.106:0/1998453224 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba0c003520 con 0x7fba2015b440 2023-11-24T08:12:35.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.057+0000 7fba26782640 1 -- 172.21.15.106:0/1998453224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 msgr2=0x7fba200c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:35.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.057+0000 7fba26782640 1 --2- 172.21.15.106:0/1998453224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba200c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.057+0000 7fba26782640 1 -- 172.21.15.106:0/1998453224 >> v1:172.21.15.106:6789/0 conn(0x7fba2015b440 legacy=0x7fba200cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.061 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.057+0000 7fba26782640 1 -- 172.21.15.106:0/1998453224 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2015f0f0 con 0x7fba200c57e0 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.058+0000 7fba26782640 1 --2- 172.21.15.106:0/1998453224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba200c57e0 0x7fba2006c5f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fba18009d90 tx=0x7fba1802be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ccab62c8dd41c62 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.058+0000 7fba24f7f640 1 -- 172.21.15.106:0/1998453224 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1802cbc0 con 0x7fba200c57e0 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.058+0000 7fba24f7f640 1 -- 172.21.15.106:0/1998453224 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1802cd20 con 0x7fba200c57e0 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.058+0000 7fba24f7f640 1 -- 172.21.15.106:0/1998453224 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18034660 con 0x7fba200c57e0 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 -- 172.21.15.106:0/1998453224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba200c57e0 msgr2=0x7fba2006c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 --2- 172.21.15.106:0/1998453224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba200c57e0 0x7fba2006c5f0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fba18009d90 tx=0x7fba1802be10 comp rx=0 tx=0).stop 2023-11-24T08:12:35.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 -- 172.21.15.106:0/1998453224 shutdown_connections 2023-11-24T08:12:35.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 --2- 172.21.15.106:0/1998453224 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba200c57e0 0x7fba2006c5f0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 --2- 172.21.15.106:0/1998453224 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba200c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 -- 172.21.15.106:0/1998453224 >> 172.21.15.106:0/1998453224 conn(0x7fba200b0c50 msgr2=0x7fba200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:35.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 -- 172.21.15.106:0/1998453224 shutdown_connections 2023-11-24T08:12:35.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.059+0000 7fba281e5640 1 -- 172.21.15.106:0/1998453224 wait complete. 2023-11-24T08:12:35.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.061+0000 7fba281e5640 1 Processor -- start 2023-11-24T08:12:35.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.061+0000 7fba281e5640 1 -- start start 2023-11-24T08:12:35.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba281e5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba200c57e0 0x7fba200c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba281e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba20156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba281e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 0x7fba2009bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba281e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba200c84a0 con 0x7fba2015b440 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba281e5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20156840 con 0x7fba200c57e0 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba281e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba2015d2a0 con 0x7fba200cc0f0 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 0x7fba2009bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 0x7fba2009bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36374/0 (socket says 172.21.15.106:36374) 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 -- 172.21.15.106:0/3842238389 learned_addr learned my addr 172.21.15.106:0/3842238389 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba25780640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba20156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba25f81640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba200c57e0 0x7fba200c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 -- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 msgr2=0x7fba20156300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba20156300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 -- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba200c57e0 msgr2=0x7fba200c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba200c57e0 0x7fba200c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba26782640 1 -- 172.21.15.106:0/3842238389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba18009a70 con 0x7fba2015b440 2023-11-24T08:12:35.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba25f81640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba200c57e0 0x7fba200c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:35.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.062+0000 7fba25780640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba20156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:35.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.063+0000 7fba26782640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 0x7fba2009bde0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fba18009d90 tx=0x7fba18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:35.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.063+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1803e610 con 0x7fba2015b440 2023-11-24T08:12:35.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.063+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1802ce90 con 0x7fba2015b440 2023-11-24T08:12:35.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.063+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba200c7820 con 0x7fba2015b440 2023-11-24T08:12:35.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.063+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1803e9f0 con 0x7fba2015b440 2023-11-24T08:12:35.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.063+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba200c7de0 con 0x7fba2015b440 2023-11-24T08:12:35.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.065+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba200c85e0 con 0x7fba2015b440 2023-11-24T08:12:35.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.065+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fba1803b050 con 0x7fba2015b440 2023-11-24T08:12:35.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.066+0000 7fba167fc640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fba0406b3e0 0x7fba0406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.066+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fba180ac7f0 con 0x7fba2015b440 2023-11-24T08:12:35.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.066+0000 7fba25f81640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fba0406b3e0 0x7fba0406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.067+0000 7fba25f81640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fba0406b3e0 0x7fba0406d8a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0011e0 tx=0x7fba1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:35.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.070+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fba200c85e0 con 0x7fba2015b440 2023-11-24T08:12:35.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.264+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba200c85e0 con 0x7fba2015b440 2023-11-24T08:12:35.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.265+0000 7fba167fc640 1 -- 172.21.15.106:0/3842238389 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v76) v1 ==== 100+0+11 (secure 0 0 0) 0x7fba18039020 con 0x7fba2015b440 2023-11-24T08:12:35.268 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:12:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fba0406b3e0 msgr2=0x7fba0406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fba0406b3e0 0x7fba0406d8a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fba1c0011e0 tx=0x7fba1c009040 comp rx=0 tx=0).stop 2023-11-24T08:12:35.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 msgr2=0x7fba2009bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 0x7fba2009bde0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fba18009d90 tx=0x7fba18004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:35.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 shutdown_connections 2023-11-24T08:12:35.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fba2015b440 0x7fba2009bde0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fba0406b3e0 0x7fba0406d8a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fba200cc0f0 0x7fba20156300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 --2- 172.21.15.106:0/3842238389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fba200c57e0 0x7fba200c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 >> 172.21.15.106:0/3842238389 conn(0x7fba200b0c50 msgr2=0x7fba20070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 shutdown_connections 2023-11-24T08:12:35.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.268+0000 7fba281e5640 1 -- 172.21.15.106:0/3842238389 wait complete. 2023-11-24T08:12:35.290 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:12:35.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.513+0000 7f2fb8a4b640 1 Processor -- start 2023-11-24T08:12:35.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.514+0000 7f2fb8a4b640 1 -- start start 2023-11-24T08:12:35.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb8a4b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fb4159920 0x7f2fb40ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb8a4b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4072570 0x7f2fb40c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb8a4b640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb406d270 con 0x7f2fb4159d00 2023-11-24T08:12:35.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb8a4b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb406c660 con 0x7f2fb4159920 2023-11-24T08:12:35.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb8a4b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb406c940 con 0x7f2fb4072570 2023-11-24T08:12:35.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fb4159920 0x7f2fb40ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb37fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fb4159920 0x7f2fb40ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36380/0 (socket says 172.21.15.106:36380) 2023-11-24T08:12:35.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.515+0000 7f2fb37fe640 1 -- 172.21.15.106:0/4084248398 learned_addr learned my addr 172.21.15.106:0/4084248398 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:35.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.516+0000 7f2fb37fe640 1 -- 172.21.15.106:0/4084248398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4072570 msgr2=0x7f2fb40c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:35.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.516+0000 7f2fb3fff640 1 --2- 172.21.15.106:0/4084248398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4072570 0x7f2fb40c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.516+0000 7f2fb37fe640 1 --2- 172.21.15.106:0/4084248398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4072570 0x7f2fb40c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.516+0000 7f2fb37fe640 1 -- 172.21.15.106:0/4084248398 >> v1:172.21.15.156:6789/0 conn(0x7f2fb4159d00 legacy=0x7f2fb406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.516+0000 7f2fb37fe640 1 -- 172.21.15.106:0/4084248398 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb406cc20 con 0x7f2fb4159920 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.517+0000 7f2fb37fe640 1 --2- 172.21.15.106:0/4084248398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159920 0x7f2fb40ae310 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400bb10 tx=0x7f2fa402e030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3b50dfe18a6fa0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.517+0000 7f2fb2ffd640 1 -- 172.21.15.106:0/4084248398 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa402ede0 con 0x7f2fb4159920 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.517+0000 7f2fb2ffd640 1 -- 172.21.15.106:0/4084248398 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa4036440 con 0x7f2fb4159920 2023-11-24T08:12:35.520 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.517+0000 7f2fb2ffd640 1 -- 172.21.15.106:0/4084248398 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa40367d0 con 0x7f2fb4159920 2023-11-24T08:12:35.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/4084248398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159920 msgr2=0x7f2fb40ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/4084248398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159920 0x7f2fb40ae310 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400bb10 tx=0x7f2fa402e030 comp rx=0 tx=0).stop 2023-11-24T08:12:35.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/4084248398 shutdown_connections 2023-11-24T08:12:35.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/4084248398 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4072570 0x7f2fb40c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/4084248398 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159920 0x7f2fb40ae310 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/4084248398 >> 172.21.15.106:0/4084248398 conn(0x7f2fb40cbb40 msgr2=0x7f2fb40cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:35.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/4084248398 shutdown_connections 2023-11-24T08:12:35.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.518+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/4084248398 wait complete. 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.519+0000 7f2fb8a4b640 1 Processor -- start 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.520+0000 7f2fb8a4b640 1 -- start start 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.520+0000 7f2fb8a4b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4072570 0x7f2fb41567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.520+0000 7f2fb8a4b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4159920 0x7f2fb40b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb8a4b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 0x7f2fb40b3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb8a4b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40c2bb0 con 0x7f2fb4159d00 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb8a4b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb40b29f0 con 0x7f2fb4072570 2023-11-24T08:12:35.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb8a4b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb409a1c0 con 0x7f2fb4159920 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 0x7f2fb40b3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 0x7f2fb40b3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36386/0 (socket says 172.21.15.106:36386) 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 -- 172.21.15.106:0/1396928940 learned_addr learned my addr 172.21.15.106:0/1396928940 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb37fe640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4072570 0x7f2fb41567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fabfff640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4159920 0x7f2fb40b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 -- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4159920 msgr2=0x7f2fb40b1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4159920 0x7f2fb40b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 -- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4072570 msgr2=0x7f2fb41567f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4072570 0x7f2fb41567f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.521+0000 7f2fb3fff640 1 -- 172.21.15.106:0/1396928940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fa400b770 con 0x7f2fb4159d00 2023-11-24T08:12:35.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.522+0000 7f2fb3fff640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 0x7f2fb40b3e00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00cd70 tx=0x7f2fac00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.522+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac003b90 con 0x7f2fb4159d00 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.522+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fac003cf0 con 0x7f2fb4159d00 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.522+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fac004240 con 0x7f2fb4159d00 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.522+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb40b2190 con 0x7f2fb4159d00 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.522+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb40b2710 con 0x7f2fb4159d00 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.523+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb404e840 con 0x7f2fb4159d00 2023-11-24T08:12:35.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.524+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2fac0043a0 con 0x7f2fb4159d00 2023-11-24T08:12:35.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.525+0000 7f2fb0ff9640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f9406b410 0x7f2f9406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:35.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.525+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2fac012070 con 0x7f2fb4159d00 2023-11-24T08:12:35.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.525+0000 7f2fb37fe640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f9406b410 0x7f2f9406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:35.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.526+0000 7f2fb37fe640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f9406b410 0x7f2f9406d8d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4038000 tx=0x7f2fa4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:35.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.529+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fac055280 con 0x7f2fb4159d00 2023-11-24T08:12:35.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.740+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2fb40ca680 con 0x7f2fb4159d00 2023-11-24T08:12:35.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.741+0000 7f2fb0ff9640 1 -- 172.21.15.106:0/1396928940 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2fac0588a0 con 0x7f2fb4159d00 2023-11-24T08:12:35.743 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:12:35.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f9406b410 msgr2=0x7f2f9406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f9406b410 0x7f2f9406d8d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4038000 tx=0x7f2fa4008040 comp rx=0 tx=0).stop 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 msgr2=0x7f2fb40b3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 0x7f2fb40b3e00 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2fac00cd70 tx=0x7f2fac00a660 comp rx=0 tx=0).stop 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 shutdown_connections 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fb4159d00 0x7f2fb40b3e00 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2f9406b410 0x7f2f9406d8d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fb4159920 0x7f2fb40b1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 --2- 172.21.15.106:0/1396928940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fb4072570 0x7f2fb41567f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 >> 172.21.15.106:0/1396928940 conn(0x7f2fb40cbb40 msgr2=0x7f2fb4158500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:35.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 shutdown_connections 2023-11-24T08:12:35.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:35.744+0000 7f2fb8a4b640 1 -- 172.21.15.106:0/1396928940 wait complete. 2023-11-24T08:12:35.764 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:12:35.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:35.765 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T08:12:35.869 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:12:35.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:35.870 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:12:35.920 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:35.920 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:35.981 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:35.981 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:36.037 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:36.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.254+0000 7fe5cb8a2640 1 Processor -- start 2023-11-24T08:12:36.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.254+0000 7fe5cb8a2640 1 -- start start 2023-11-24T08:12:36.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5cb8a2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cf450 0x7fe5c40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5cb8a2640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe5c40cdfb0 0x7fe5c40ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5cb8a2640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40cece0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5cb8a2640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c4156010 con 0x7fe5c40cdfb0 2023-11-24T08:12:36.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5cb8a2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c406ce30 con 0x7fe5c40cf450 2023-11-24T08:12:36.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5c963e640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe5c415c7f0 0x7fe5c406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42828/0 (socket says 172.21.15.106:42828) 2023-11-24T08:12:36.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5c963e640 1 -- 172.21.15.106:0/2320200959 learned_addr learned my addr 172.21.15.106:0/2320200959 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:36.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5c9e3f640 1 --2- 172.21.15.106:0/2320200959 >> v2:172.21.15.106:3300/0 conn(0x7fe5c40cdfb0 0x7fe5c40ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5c8e3d640 1 --2- 172.21.15.106:0/2320200959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cf450 0x7fe5c40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.256+0000 7fe5bbfff640 1 -- 172.21.15.106:0/2320200959 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2784671425 0 0) 0x7fe5c40cece0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.257+0000 7fe5bbfff640 1 -- 172.21.15.106:0/2320200959 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b0003520 con 0x7fe5c415c7f0 2023-11-24T08:12:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.257+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/2320200959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cf450 msgr2=0x7fe5c40cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.257+0000 7fe5c9e3f640 1 --2- 172.21.15.106:0/2320200959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cf450 0x7fe5c40cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.257+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/2320200959 >> v1:172.21.15.106:6789/0 conn(0x7fe5c415c7f0 legacy=0x7fe5c406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.257+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/2320200959 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c406d040 con 0x7fe5c40cdfb0 2023-11-24T08:12:36.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.258+0000 7fe5c9e3f640 1 --2- 172.21.15.106:0/2320200959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c40cdfb0 0x7fe5c40ce440 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc02bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80a5c76684f766a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:36.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.258+0000 7fe5bbfff640 1 -- 172.21.15.106:0/2320200959 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc02cc50 con 0x7fe5c40cdfb0 2023-11-24T08:12:36.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.258+0000 7fe5bbfff640 1 -- 172.21.15.106:0/2320200959 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc02cdb0 con 0x7fe5c40cdfb0 2023-11-24T08:12:36.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.258+0000 7fe5bbfff640 1 -- 172.21.15.106:0/2320200959 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc034660 con 0x7fe5c40cdfb0 2023-11-24T08:12:36.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/2320200959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c40cdfb0 msgr2=0x7fe5c40ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/2320200959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c40cdfb0 0x7fe5c40ce440 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc02bea0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/2320200959 shutdown_connections 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/2320200959 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c40cdfb0 0x7fe5c40ce440 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/2320200959 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cf450 0x7fe5c40cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/2320200959 >> 172.21.15.106:0/2320200959 conn(0x7fe5c406a920 msgr2=0x7fe5c406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/2320200959 shutdown_connections 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.259+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/2320200959 wait complete. 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.261+0000 7fe5cb8a2640 1 Processor -- start 2023-11-24T08:12:36.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.261+0000 7fe5cb8a2640 1 -- start start 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5cb8a2640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cdfb0 0x7fe5c4158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5cb8a2640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c40cf450 0x7fe5c4153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5cb8a2640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 0x7fe5c41f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5cb8a2640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4158760 con 0x7fe5c415c7f0 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5cb8a2640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4158170 con 0x7fe5c40cf450 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5cb8a2640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c41f4f70 con 0x7fe5c40cdfb0 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 0x7fe5c41f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 0x7fe5c41f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36404/0 (socket says 172.21.15.106:36404) 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/4017429810 learned_addr learned my addr 172.21.15.106:0/4017429810 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c963e640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cdfb0 0x7fe5c4158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cdfb0 msgr2=0x7fe5c4158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cdfb0 0x7fe5c4158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c40cf450 msgr2=0x7fe5c4153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c40cf450 0x7fe5c4153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.262+0000 7fe5c9e3f640 1 -- 172.21.15.106:0/4017429810 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc009a70 con 0x7fe5c415c7f0 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.263+0000 7fe5c9e3f640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 0x7fe5c41f4540 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.263+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc040650 con 0x7fe5c415c7f0 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.263+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0407b0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.263+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc040b60 con 0x7fe5c415c7f0 2023-11-24T08:12:36.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.263+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c41f5250 con 0x7fe5c415c7f0 2023-11-24T08:12:36.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.263+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c41539c0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.264+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c41588a0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.265+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe5bc040cc0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.266+0000 7fe5b9ffb640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5a806b410 0x7fe5a806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.267+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fe5bc0acb90 con 0x7fe5c415c7f0 2023-11-24T08:12:36.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.267+0000 7fe5c963e640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5a806b410 0x7fe5a806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.267+0000 7fe5c963e640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5a806b410 0x7fe5a806d8d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00011e0 tx=0x7fe5c0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:36.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.271+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe5c41588a0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.467 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.464+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5c41588a0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.466+0000 7fe5b9ffb640 1 -- 172.21.15.106:0/4017429810 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16824 (secure 0 0 0) 0x7fe5c41588a0 con 0x7fe5c415c7f0 2023-11-24T08:12:36.469 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:36.469 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:35.407263+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:36.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.468+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5a806b410 msgr2=0x7fe5a806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.468+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5a806b410 0x7fe5a806d8d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00011e0 tx=0x7fe5c0009040 comp rx=0 tx=0).stop 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 msgr2=0x7fe5c41f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 0x7fe5c41f4540 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc009d90 tx=0x7fe5bc004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 shutdown_connections 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe5c415c7f0 0x7fe5c41f4540 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5c40cf450 0x7fe5c4153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe5a806b410 0x7fe5a806d8d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 --2- 172.21.15.106:0/4017429810 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe5c40cdfb0 0x7fe5c4158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 >> 172.21.15.106:0/4017429810 conn(0x7fe5c406a920 msgr2=0x7fe5c415e120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:36.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 shutdown_connections 2023-11-24T08:12:36.473 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.469+0000 7fe5cb8a2640 1 -- 172.21.15.106:0/4017429810 wait complete. 2023-11-24T08:12:36.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:12:07.722409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:12:08.984297+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:36.492 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:36.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.705+0000 7f639f5a9640 1 Processor -- start 2023-11-24T08:12:36.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.706+0000 7f639f5a9640 1 -- start start 2023-11-24T08:12:36.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.707+0000 7f639f5a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f639806de10 0x7f639815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.707+0000 7f639f5a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.707+0000 7f639f5a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639815ec90 con 0x7f6398072160 2023-11-24T08:12:36.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.707+0000 7f639f5a9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63980cd530 con 0x7f639806de10 2023-11-24T08:12:36.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.707+0000 7f639f5a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63980c4ed0 con 0x7f63980c49f0 2023-11-24T08:12:36.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639cb44640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f639806de10 0x7f639815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639d345640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6398072160 0x7f639806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42840/0 (socket says 172.21.15.106:42840) 2023-11-24T08:12:36.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639cb44640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f639806de10 0x7f639815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36412/0 (socket says 172.21.15.106:36412) 2023-11-24T08:12:36.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639d345640 1 -- 172.21.15.106:0/3107190874 learned_addr learned my addr 172.21.15.106:0/3107190874 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:36.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639cb44640 1 -- 172.21.15.106:0/3107190874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 msgr2=0x7f63980705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:36.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639db46640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639cb44640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980705c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639cb44640 1 -- 172.21.15.106:0/3107190874 >> v1:172.21.15.106:6789/0 conn(0x7f6398072160 legacy=0x7f639806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639cb44640 1 -- 172.21.15.106:0/3107190874 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980c51b0 con 0x7f639806de10 2023-11-24T08:12:36.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f638ffff640 1 -- 172.21.15.106:0/3107190874 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2118315541 0 0) 0x7f6394001110 con 0x7f6398072160 2023-11-24T08:12:36.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.708+0000 7f639db46640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:36.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.709+0000 7f639cb44640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f639815e750 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f63880097a0 tx=0x7f638802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dffbb80c0df50f5b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:36.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f638ffff640 1 -- 172.21.15.106:0/3107190874 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f638802d040 con 0x7f639806de10 2023-11-24T08:12:36.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f638ffff640 1 -- 172.21.15.106:0/3107190874 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6388031810 con 0x7f639806de10 2023-11-24T08:12:36.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f638ffff640 1 -- 172.21.15.106:0/3107190874 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6388031ba0 con 0x7f639806de10 2023-11-24T08:12:36.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f639f5a9640 1 -- 172.21.15.106:0/3107190874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 msgr2=0x7f639815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f639f5a9640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f639815e750 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f63880097a0 tx=0x7f638802b8b0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f639f5a9640 1 -- 172.21.15.106:0/3107190874 shutdown_connections 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f639f5a9640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f639f5a9640 1 --2- 172.21.15.106:0/3107190874 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f639815e750 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.710+0000 7f639f5a9640 1 -- 172.21.15.106:0/3107190874 >> 172.21.15.106:0/3107190874 conn(0x7f63980b0c50 msgr2=0x7f63980b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.711+0000 7f639f5a9640 1 -- 172.21.15.106:0/3107190874 shutdown_connections 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.711+0000 7f639f5a9640 1 -- 172.21.15.106:0/3107190874 wait complete. 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.712+0000 7f639f5a9640 1 Processor -- start 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.712+0000 7f639f5a9640 1 -- start start 2023-11-24T08:12:36.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.713+0000 7f639f5a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f63980cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.713+0000 7f639f5a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398072160 0x7f6398154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.713+0000 7f639f5a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.713+0000 7f639f5a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980cb7e0 con 0x7f639806de10 2023-11-24T08:12:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.713+0000 7f639f5a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398155350 con 0x7f6398072160 2023-11-24T08:12:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.713+0000 7f639f5a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63981555b0 con 0x7f63980c49f0 2023-11-24T08:12:36.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f63980cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f63980cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36420/0 (socket says 172.21.15.106:36420) 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 -- 172.21.15.106:0/340544414 learned_addr learned my addr 172.21.15.106:0/340544414 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 -- 172.21.15.106:0/340544414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 msgr2=0x7f63980accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 -- 172.21.15.106:0/340544414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398072160 msgr2=0x7f6398154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398072160 0x7f6398154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.714+0000 7f639d345640 1 -- 172.21.15.106:0/340544414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388009400 con 0x7f639806de10 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.715+0000 7f639d345640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f63980cb2a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f639400c780 tx=0x7f639400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:36.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.715+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940072d0 con 0x7f639806de10 2023-11-24T08:12:36.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.715+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6394007450 con 0x7f639806de10 2023-11-24T08:12:36.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.715+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940047e0 con 0x7f639806de10 2023-11-24T08:12:36.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.715+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398155890 con 0x7f639806de10 2023-11-24T08:12:36.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.715+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398155e50 con 0x7f639806de10 2023-11-24T08:12:36.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.716+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639804e8b0 con 0x7f639806de10 2023-11-24T08:12:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.717+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6394004960 con 0x7f639806de10 2023-11-24T08:12:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.718+0000 7f638dffb640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637c06b3e0 0x7f637c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.718+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f63940891e0 con 0x7f639806de10 2023-11-24T08:12:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.718+0000 7f639cb44640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637c06b3e0 0x7f637c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.719+0000 7f639cb44640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637c06b3e0 0x7f637c06d8a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6388036000 tx=0x7f6388030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:36.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.722+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6394053970 con 0x7f639806de10 2023-11-24T08:12:36.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.910+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f639815ec90 con 0x7f639806de10 2023-11-24T08:12:36.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.911+0000 7f638dffb640 1 -- 172.21.15.106:0/340544414 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f639815ec90 con 0x7f639806de10 2023-11-24T08:12:36.914 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:36.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637c06b3e0 msgr2=0x7f637c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637c06b3e0 0x7f637c06d8a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6388036000 tx=0x7f6388030000 comp rx=0 tx=0).stop 2023-11-24T08:12:36.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 msgr2=0x7f63980cb2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:36.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f63980cb2a0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f639400c780 tx=0x7f639400cc40 comp rx=0 tx=0).stop 2023-11-24T08:12:36.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 shutdown_connections 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f637c06b3e0 0x7f637c06d8a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63980c49f0 0x7f63980accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6398072160 0x7f6398154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 --2- 172.21.15.106:0/340544414 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f639806de10 0x7f63980cb2a0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.914+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 >> 172.21.15.106:0/340544414 conn(0x7f63980b0c50 msgr2=0x7f63980b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.915+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 shutdown_connections 2023-11-24T08:12:36.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:36.915+0000 7f639f5a9640 1 -- 172.21.15.106:0/340544414 wait complete. 2023-11-24T08:12:36.936 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:12:37.147 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.144+0000 7f583a23d640 1 Processor -- start 2023-11-24T08:12:37.147 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.145+0000 7f583a23d640 1 -- start start 2023-11-24T08:12:37.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.146+0000 7f583a23d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f58340aef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.146+0000 7f583a23d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340b4eb0 con 0x7f583415b6d0 2023-11-24T08:12:37.148 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.146+0000 7f583a23d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58340b5620 con 0x7f583415b2f0 2023-11-24T08:12:37.149 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.146+0000 7f583a23d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cbf10 con 0x7f583406bc20 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.146+0000 7f5833fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f583415b6d0 0x7f583406d0e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42852/0 (socket says 172.21.15.106:42852) 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.146+0000 7f5833fff640 1 -- 172.21.15.106:0/3477595842 learned_addr learned my addr 172.21.15.106:0/3477595842 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f5832ffd640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f58340aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420010572 0 0) 0x7f58340b4eb0 con 0x7f583415b6d0 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5820003520 con 0x7f583415b6d0 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1885668970 0 0) 0x7f5820003520 con 0x7f583415b6d0 2023-11-24T08:12:37.151 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 msgr2=0x7f58340aef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f58340aef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 >> v1:172.21.15.156:6789/0 conn(0x7f583415b2f0 legacy=0x7f583406b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58340cc1f0 con 0x7f583415b6d0 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f582c001670 con 0x7f583415b6d0 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.147+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 >> v1:172.21.15.106:6789/0 conn(0x7f583415b6d0 legacy=0x7f583406d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58327fc640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58200039a0 0x7f5820003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58327fc640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f5820004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58327fc640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f5820005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340cbf10 con 0x7f5820004350 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582000ba30 con 0x7f58200039a0 2023-11-24T08:12:37.152 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582000bbf0 con 0x7f583406bc20 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5832ffd640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f5820004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f58337fe640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58200039a0 0x7f5820003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5833fff640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f5820005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5832ffd640 1 -- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 msgr2=0x7f5820005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5832ffd640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f5820005120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5832ffd640 1 -- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58200039a0 msgr2=0x7f5820003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5832ffd640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58200039a0 0x7f5820003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5832ffd640 1 -- 172.21.15.106:0/3477595842 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5820003520 con 0x7f5820004350 2023-11-24T08:12:37.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.148+0000 7f5833fff640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f583406bc20 0x7f5820005120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f5832ffd640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f5820004820 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f581c009ac0 tx=0x7f58200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98d6f1f9abf74896 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c02c6b0 con 0x7f5820004350 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c02c810 con 0x7f5820004350 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f58327fc640 1 -- 172.21.15.106:0/3477595842 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c02cba0 con 0x7f5820004350 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f583a23d640 1 -- 172.21.15.106:0/3477595842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 msgr2=0x7f5820004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f583a23d640 1 --2- 172.21.15.106:0/3477595842 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f5820004820 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f581c009ac0 tx=0x7f58200057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f5833fff640 1 -- 172.21.15.106:0/3477595842 reap_dead start 2023-11-24T08:12:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f583a23d640 1 -- 172.21.15.106:0/3477595842 shutdown_connections 2023-11-24T08:12:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.150+0000 7f583a23d640 1 -- 172.21.15.106:0/3477595842 >> 172.21.15.106:0/3477595842 conn(0x7f5834070fa0 msgr2=0x7f58340713b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.151+0000 7f583a23d640 1 -- 172.21.15.106:0/3477595842 shutdown_connections 2023-11-24T08:12:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.151+0000 7f583a23d640 1 -- 172.21.15.106:0/3477595842 wait complete. 2023-11-24T08:12:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.152+0000 7f583a23d640 1 Processor -- start 2023-11-24T08:12:37.156 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.152+0000 7f583a23d640 1 -- start start 2023-11-24T08:12:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.153+0000 7f583a23d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f58341527d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.153+0000 7f583a23d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58340b0820 0x7f58340b0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.153+0000 7f583a23d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58341540c0 0x7f5834154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.153+0000 7f583a23d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58340b11e0 con 0x7f5820004350 2023-11-24T08:12:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.153+0000 7f583a23d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5834155a00 con 0x7f58341540c0 2023-11-24T08:12:37.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.153+0000 7f583a23d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5834154df0 con 0x7f58340b0820 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f58341527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f58341527d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36444/0 (socket says 172.21.15.106:36444) 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 -- 172.21.15.106:0/1221558947 learned_addr learned my addr 172.21.15.106:0/1221558947 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f5832ffd640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58340b0820 0x7f58340b0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f5833fff640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58341540c0 0x7f5834154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 -- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58340b0820 msgr2=0x7f58340b0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58340b0820 0x7f58340b0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 -- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58341540c0 msgr2=0x7f5834154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58341540c0 0x7f5834154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f58337fe640 1 -- 172.21.15.106:0/1221558947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58341550d0 con 0x7f5820004350 2023-11-24T08:12:37.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f5832ffd640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58340b0820 0x7f58340b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:37.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.154+0000 7f5833fff640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58341540c0 0x7f5834154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.155+0000 7f58337fe640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f58341527d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f582400e940 tx=0x7f582400ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.155+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5824003b80 con 0x7f5820004350 2023-11-24T08:12:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.155+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5824003ce0 con 0x7f5820004350 2023-11-24T08:12:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.155+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582400c9c0 con 0x7f5820004350 2023-11-24T08:12:37.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.155+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58341553b0 con 0x7f5820004350 2023-11-24T08:12:37.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.155+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5834154a80 con 0x7f5820004350 2023-11-24T08:12:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.156+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583404e8b0 con 0x7f5820004350 2023-11-24T08:12:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.157+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f582400cb20 con 0x7f5820004350 2023-11-24T08:12:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.158+0000 7f582bfff640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f581406b410 0x7f581406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.158+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f582408a270 con 0x7f5820004350 2023-11-24T08:12:37.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.158+0000 7f5832ffd640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f581406b410 0x7f581406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.159+0000 7f5832ffd640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f581406b410 0x7f581406d8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f581c036000 tx=0x7f581c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:37.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.162+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5824054b40 con 0x7f5820004350 2023-11-24T08:12:37.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.356+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5834155b40 con 0x7f5820004350 2023-11-24T08:12:37.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.357+0000 7f582bfff640 1 -- 172.21.15.106:0/1221558947 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5824058160 con 0x7f5820004350 2023-11-24T08:12:37.359 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.359+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f581406b410 msgr2=0x7f581406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.359+0000 7f583a23d640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f581406b410 0x7f581406d8d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f581c036000 tx=0x7f581c030000 comp rx=0 tx=0).stop 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.359+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 msgr2=0x7f58341527d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.359+0000 7f583a23d640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f58341527d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f582400e940 tx=0x7f582400ee00 comp rx=0 tx=0).stop 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 shutdown_connections 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58341540c0 0x7f5834154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f581406b410 0x7f581406d8d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58340b0820 0x7f58340b0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 --2- 172.21.15.106:0/1221558947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5820004350 0x7f58341527d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 >> 172.21.15.106:0/1221558947 conn(0x7f5834070fa0 msgr2=0x7f583406d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:37.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 shutdown_connections 2023-11-24T08:12:37.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.360+0000 7f583a23d640 1 -- 172.21.15.106:0/1221558947 wait complete. 2023-11-24T08:12:37.381 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:12:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.589+0000 7f596f631640 1 Processor -- start 2023-11-24T08:12:37.592 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.589+0000 7f596f631640 1 -- start start 2023-11-24T08:12:37.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596f631640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f596815b440 0x7f59680cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596f631640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f59680cbf20 0x7f59680c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596f631640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59680c57e0 0x7f59680c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596f631640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59680cc3d0 con 0x7f596815b440 2023-11-24T08:12:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596f631640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f596815b820 con 0x7f59680cbf20 2023-11-24T08:12:37.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596f631640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596815eea0 con 0x7f59680c57e0 2023-11-24T08:12:37.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596d3cd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f596815b440 0x7f59680cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596d3cd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f596815b440 0x7f59680cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36452/0 (socket says 172.21.15.106:36452) 2023-11-24T08:12:37.595 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.591+0000 7f596d3cd640 1 -- 172.21.15.106:0/2647911622 learned_addr learned my addr 172.21.15.106:0/2647911622 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596cbcc640 1 --2- 172.21.15.106:0/2647911622 >> v2:172.21.15.156:3300/0 conn(0x7f59680cbf20 0x7f59680c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596dbce640 1 --2- 172.21.15.106:0/2647911622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59680c57e0 0x7f59680c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596d3cd640 1 -- 172.21.15.106:0/2647911622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59680c57e0 msgr2=0x7f59680c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596d3cd640 1 --2- 172.21.15.106:0/2647911622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59680c57e0 0x7f59680c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596d3cd640 1 -- 172.21.15.106:0/2647911622 >> v2:172.21.15.156:3300/0 conn(0x7f59680cbf20 msgr2=0x7f59680c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596d3cd640 1 --2- 172.21.15.106:0/2647911622 >> v2:172.21.15.156:3300/0 conn(0x7f59680cbf20 0x7f59680c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.592+0000 7f596d3cd640 1 -- 172.21.15.106:0/2647911622 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59680abf30 con 0x7f596815b440 2023-11-24T08:12:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.593+0000 7f596d3cd640 1 --2- 172.21.15.106:0/2647911622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f596815b440 0x7f59680cb9e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f596806e890 tx=0x7f596402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ea7956a5e161fe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f595ffff640 1 -- 172.21.15.106:0/2647911622 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59640317d0 con 0x7f596815b440 2023-11-24T08:12:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f595ffff640 1 -- 172.21.15.106:0/2647911622 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964031930 con 0x7f596815b440 2023-11-24T08:12:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f595ffff640 1 -- 172.21.15.106:0/2647911622 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964031cc0 con 0x7f596815b440 2023-11-24T08:12:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 -- 172.21.15.106:0/2647911622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f596815b440 msgr2=0x7f59680cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 --2- 172.21.15.106:0/2647911622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f596815b440 0x7f59680cb9e0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f596806e890 tx=0x7f596402b870 comp rx=0 tx=0).stop 2023-11-24T08:12:37.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 -- 172.21.15.106:0/2647911622 shutdown_connections 2023-11-24T08:12:37.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 --2- 172.21.15.106:0/2647911622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f59680c57e0 0x7f59680c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 --2- 172.21.15.106:0/2647911622 >> v2:172.21.15.156:3300/0 conn(0x7f59680cbf20 0x7f59680c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 --2- 172.21.15.106:0/2647911622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f596815b440 0x7f59680cb9e0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 -- 172.21.15.106:0/2647911622 >> 172.21.15.106:0/2647911622 conn(0x7f59680b0c50 msgr2=0x7f59680b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:37.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 -- 172.21.15.106:0/2647911622 shutdown_connections 2023-11-24T08:12:37.601 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.594+0000 7f596f631640 1 -- 172.21.15.106:0/2647911622 wait complete. 2023-11-24T08:12:37.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.596+0000 7f596f631640 1 Processor -- start 2023-11-24T08:12:37.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.596+0000 7f596f631640 1 -- start start 2023-11-24T08:12:37.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.597+0000 7f596f631640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 0x7f5968159880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.597+0000 7f596f631640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59680cbf20 0x7f596815ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.597+0000 7f596f631640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f596815b440 0x7f59680b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.597+0000 7f596f631640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596815ba00 con 0x7f59680c57e0 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.597+0000 7f596f631640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596815bc70 con 0x7f59680cbf20 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.597+0000 7f596f631640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596815bf50 con 0x7f596815b440 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 0x7f5968159880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 0x7f5968159880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36464/0 (socket says 172.21.15.106:36464) 2023-11-24T08:12:37.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 -- 172.21.15.106:0/3713492752 learned_addr learned my addr 172.21.15.106:0/3713492752 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 -- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f596815b440 msgr2=0x7f59680b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596cbcc640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59680cbf20 0x7f596815ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f596815b440 0x7f59680b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 -- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59680cbf20 msgr2=0x7f596815ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59680cbf20 0x7f596815ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 -- 172.21.15.106:0/3713492752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59640094d0 con 0x7f59680c57e0 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.598+0000 7f596d3cd640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 0x7f5968159880 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f59640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:37.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.599+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964032870 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.599+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964032a10 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.599+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964032dc0 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.599+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596815c230 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.599+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5968159dc0 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.600+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596815c090 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.601+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5964030070 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.602+0000 7f595dffb640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f594c06b3e0 0x7f594c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.602+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f59640ac150 con 0x7f59680c57e0 2023-11-24T08:12:37.607 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.602+0000 7f596cbcc640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f594c06b3e0 0x7f594c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:37.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.602+0000 7f596cbcc640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f594c06b3e0 0x7f594c06d8a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f596815d580 tx=0x7f5958008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:37.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.606+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f596815c090 con 0x7f59680c57e0 2023-11-24T08:12:37.799 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.797+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f596815c750 con 0x7f59680c57e0 2023-11-24T08:12:37.800 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.798+0000 7f595dffb640 1 -- 172.21.15.106:0/3713492752 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7f596815c750 con 0x7f59680c57e0 2023-11-24T08:12:37.801 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.800+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f594c06b3e0 msgr2=0x7f594c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f594c06b3e0 0x7f594c06d8a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f596815d580 tx=0x7f5958008040 comp rx=0 tx=0).stop 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 msgr2=0x7f5968159880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 0x7f5968159880 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5964000c00 tx=0x7f59640046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 shutdown_connections 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f594c06b3e0 0x7f594c06d8a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f596815b440 0x7f59680b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59680cbf20 0x7f596815ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 --2- 172.21.15.106:0/3713492752 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f59680c57e0 0x7f5968159880 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:37.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 >> 172.21.15.106:0/3713492752 conn(0x7f59680b0c50 msgr2=0x7f59680cf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:37.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 shutdown_connections 2023-11-24T08:12:37.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:37.801+0000 7f596f631640 1 -- 172.21.15.106:0/3713492752 wait complete. 2023-11-24T08:12:37.824 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:12:38.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.022+0000 7fe4de91d640 1 Processor -- start 2023-11-24T08:12:38.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.022+0000 7fe4de91d640 1 -- start start 2023-11-24T08:12:38.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.023+0000 7fe4de91d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4d806de10 0x7fe4d815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.023+0000 7fe4de91d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 0x7fe4d80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.023+0000 7fe4de91d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d815ec90 con 0x7fe4d8072160 2023-11-24T08:12:38.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.023+0000 7fe4de91d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d80cd530 con 0x7fe4d806de10 2023-11-24T08:12:38.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.023+0000 7fe4de91d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d80c4ed0 con 0x7fe4d80c49f0 2023-11-24T08:12:38.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4d806de10 0x7fe4d815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe4d806de10 0x7fe4d815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36472/0 (socket says 172.21.15.106:36472) 2023-11-24T08:12:38.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 -- 172.21.15.106:0/59922241 learned_addr learned my addr 172.21.15.106:0/59922241 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:38.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 -- 172.21.15.106:0/59922241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 msgr2=0x7fe4d80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:38.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 --2- 172.21.15.106:0/59922241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 0x7fe4d80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 -- 172.21.15.106:0/59922241 >> v1:172.21.15.156:6789/0 conn(0x7fe4d8072160 legacy=0x7fe4d806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:38.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.024+0000 7fe4cffff640 1 -- 172.21.15.106:0/59922241 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d80c51b0 con 0x7fe4d806de10 2023-11-24T08:12:38.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.025+0000 7fe4cffff640 1 --2- 172.21.15.106:0/59922241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d806de10 0x7fe4d815e750 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009870 tx=0x7fe4c002db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af477d043fb659b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:38.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.025+0000 7fe4d77fe640 1 -- 172.21.15.106:0/59922241 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002f8e0 con 0x7fe4d806de10 2023-11-24T08:12:38.035 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4d77fe640 1 -- 172.21.15.106:0/59922241 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c002fa40 con 0x7fe4d806de10 2023-11-24T08:12:38.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4d77fe640 1 -- 172.21.15.106:0/59922241 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002fdd0 con 0x7fe4d806de10 2023-11-24T08:12:38.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 -- 172.21.15.106:0/59922241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d806de10 msgr2=0x7fe4d815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 --2- 172.21.15.106:0/59922241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d806de10 0x7fe4d815e750 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009870 tx=0x7fe4c002db20 comp rx=0 tx=0).stop 2023-11-24T08:12:38.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 -- 172.21.15.106:0/59922241 shutdown_connections 2023-11-24T08:12:38.036 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 --2- 172.21.15.106:0/59922241 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 0x7fe4d80705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 --2- 172.21.15.106:0/59922241 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d806de10 0x7fe4d815e750 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 -- 172.21.15.106:0/59922241 >> 172.21.15.106:0/59922241 conn(0x7fe4d80b0c50 msgr2=0x7fe4d80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:38.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 -- 172.21.15.106:0/59922241 shutdown_connections 2023-11-24T08:12:38.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.026+0000 7fe4de91d640 1 -- 172.21.15.106:0/59922241 wait complete. 2023-11-24T08:12:38.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.028+0000 7fe4de91d640 1 Processor -- start 2023-11-24T08:12:38.037 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.028+0000 7fe4de91d640 1 -- start start 2023-11-24T08:12:38.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4de91d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d806de10 0x7fe4d80cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4de91d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 0x7fe4d80999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4de91d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 0x7fe4d8157ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4de91d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d81577e0 con 0x7fe4d8072160 2023-11-24T08:12:38.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4de91d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d815cb20 con 0x7fe4d806de10 2023-11-24T08:12:38.038 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4de91d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d815ce00 con 0x7fe4d80c49f0 2023-11-24T08:12:38.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 0x7fe4d80999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4cffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 0x7fe4d80999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36484/0 (socket says 172.21.15.106:36484) 2023-11-24T08:12:38.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.029+0000 7fe4cffff640 1 -- 172.21.15.106:0/1516644839 learned_addr learned my addr 172.21.15.106:0/1516644839 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:38.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4cffff640 1 -- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 msgr2=0x7fe4d8157ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4cffff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 0x7fe4d8157ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4cffff640 1 -- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d806de10 msgr2=0x7fe4d80cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4d7fff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d806de10 0x7fe4d80cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4cffff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d806de10 0x7fe4d80cc540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4cffff640 1 -- 172.21.15.106:0/1516644839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c00094d0 con 0x7fe4d8072160 2023-11-24T08:12:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4d7fff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d806de10 0x7fe4d80cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4cffff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 0x7fe4d80999d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000c00 tx=0x7fe4c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:38.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c002f880 con 0x7fe4d8072160 2023-11-24T08:12:38.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0040420 con 0x7fe4d8072160 2023-11-24T08:12:38.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00407d0 con 0x7fe4d8072160 2023-11-24T08:12:38.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.030+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d815d0e0 con 0x7fe4d8072160 2023-11-24T08:12:38.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.031+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d8099f10 con 0x7fe4d8072160 2023-11-24T08:12:38.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.032+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d815cf40 con 0x7fe4d8072160 2023-11-24T08:12:38.041 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.033+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe4c0040930 con 0x7fe4d8072160 2023-11-24T08:12:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.033+0000 7fe4d57fa640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe4b806b410 0x7fe4b806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.034+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fe4c0033070 con 0x7fe4d8072160 2023-11-24T08:12:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.034+0000 7fe4d7fff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe4b806b410 0x7fe4b806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.035+0000 7fe4d7fff640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe4b806b410 0x7fe4b806d8d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8003eb0 tx=0x7fe4c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:38.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.038+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4d815cf40 con 0x7fe4d8072160 2023-11-24T08:12:38.248 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.245+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe4d815ec90 con 0x7fe4d8072160 2023-11-24T08:12:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.246+0000 7fe4d57fa640 1 -- 172.21.15.106:0/1516644839 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v77) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe4d815ec90 con 0x7fe4d8072160 2023-11-24T08:12:38.249 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:12:38.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe4b806b410 msgr2=0x7fe4b806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe4b806b410 0x7fe4b806d8d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8003eb0 tx=0x7fe4c8009040 comp rx=0 tx=0).stop 2023-11-24T08:12:38.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 msgr2=0x7fe4d80999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 0x7fe4d80999d0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0000c00 tx=0x7fe4c00046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 shutdown_connections 2023-11-24T08:12:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe4b806b410 0x7fe4b806d8d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe4d80c49f0 0x7fe4d8157ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4d8072160 0x7fe4d80999d0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 --2- 172.21.15.106:0/1516644839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4d806de10 0x7fe4d80cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 >> 172.21.15.106:0/1516644839 conn(0x7fe4d80b0c50 msgr2=0x7fe4d80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 shutdown_connections 2023-11-24T08:12:38.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.249+0000 7fe4de91d640 1 -- 172.21.15.106:0/1516644839 wait complete. 2023-11-24T08:12:38.269 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:12:38.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:38.269 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T08:12:38.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:12:38.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:12:38.389 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:12:38.440 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:38.440 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:38.501 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:12:38.502 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:38.502 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:38.563 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:12:38.564 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.781+0000 7f245531d640 1 Processor -- start 2023-11-24T08:12:38.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.781+0000 7f245531d640 1 -- start start 2023-11-24T08:12:38.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f245531d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 0x7f2450161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f245531d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24500ce0b0 con 0x7f245015dc80 2023-11-24T08:12:38.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f245531d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2450161960 con 0x7f2450157c70 2023-11-24T08:12:38.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f245531d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245015e8d0 con 0x7f24500cfb60 2023-11-24T08:12:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f244f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f245015dc80 0x7f24500b31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42864/0 (socket says 172.21.15.106:42864) 2023-11-24T08:12:38.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f244f7fe640 1 -- 172.21.15.106:0/2793051000 learned_addr learned my addr 172.21.15.106:0/2793051000 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.783+0000 7f244e7fc640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 0x7f2450161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248865679 0 0) 0x7f24500ce0b0 con 0x7f245015dc80 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f243c003520 con 0x7f245015dc80 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1251116117 0 0) 0x7f243c003520 con 0x7f245015dc80 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 msgr2=0x7f2450161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 0x7f2450161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 >> v1:172.21.15.156:6789/0 conn(0x7f2450157c70 legacy=0x7f24500cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245015ebb0 con 0x7f245015dc80 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f2448001670 con 0x7f245015dc80 2023-11-24T08:12:38.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 >> v1:172.21.15.106:6789/0 conn(0x7f245015dc80 legacy=0x7f24500b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0039a0 0x7f243c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c004350 0x7f243c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 0x7f243c005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245015e8d0 con 0x7f243c0039a0 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c00ba30 con 0x7f243c004350 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.784+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c00bbf0 con 0x7f24500cfb60 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244effd640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0039a0 0x7f243c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244e7fc640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c004350 0x7f243c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244effd640 1 -- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 msgr2=0x7f243c005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244effd640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f24500cfb60 0x7f243c005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244effd640 1 -- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c004350 msgr2=0x7f243c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:38.789 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244effd640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c004350 0x7f243c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244effd640 1 -- 172.21.15.106:0/2793051000 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c003520 con 0x7f243c0039a0 2023-11-24T08:12:38.790 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.785+0000 7f244e7fc640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c004350 0x7f243c004820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.786+0000 7f244effd640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0039a0 0x7f243c003d80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2440002010 tx=0x7f243c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d772fb181dd0447 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244002c770 con 0x7f243c0039a0 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244002c8d0 con 0x7f243c0039a0 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f244dffb640 1 -- 172.21.15.106:0/2793051000 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244002cc60 con 0x7f243c0039a0 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f245531d640 1 -- 172.21.15.106:0/2793051000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0039a0 msgr2=0x7f243c003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f245531d640 1 --2- 172.21.15.106:0/2793051000 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f243c0039a0 0x7f243c003d80 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f2440002010 tx=0x7f243c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f244f7fe640 1 -- 172.21.15.106:0/2793051000 reap_dead start 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f245531d640 1 -- 172.21.15.106:0/2793051000 shutdown_connections 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.787+0000 7f245531d640 1 -- 172.21.15.106:0/2793051000 >> 172.21.15.106:0/2793051000 conn(0x7f245006a920 msgr2=0x7f245006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.788+0000 7f245531d640 1 -- 172.21.15.106:0/2793051000 shutdown_connections 2023-11-24T08:12:38.791 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.788+0000 7f245531d640 1 -- 172.21.15.106:0/2793051000 wait complete. 2023-11-24T08:12:38.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.789+0000 7f245531d640 1 Processor -- start 2023-11-24T08:12:38.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.790+0000 7f245531d640 1 -- start start 2023-11-24T08:12:38.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.790+0000 7f245531d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c0039a0 0x7f24500ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.790+0000 7f245531d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 0x7f24500ad550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.790+0000 7f245531d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2450146e00 0x7f2450147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f245531d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245009d1a0 con 0x7f2450145dc0 2023-11-24T08:12:38.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f245531d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245009aa50 con 0x7f243c0039a0 2023-11-24T08:12:38.793 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f245531d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24501477c0 con 0x7f2450146e00 2023-11-24T08:12:38.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 0x7f24500ad550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 0x7f24500ad550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36500/0 (socket says 172.21.15.106:36500) 2023-11-24T08:12:38.794 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 -- 172.21.15.106:0/4063126101 learned_addr learned my addr 172.21.15.106:0/4063126101 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 -- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2450146e00 msgr2=0x7f2450147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244effd640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c0039a0 0x7f24500ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2450146e00 0x7f2450147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 -- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c0039a0 msgr2=0x7f24500ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c0039a0 0x7f24500ad010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.791+0000 7f244e7fc640 1 -- 172.21.15.106:0/4063126101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245014c7e0 con 0x7f2450145dc0 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.792+0000 7f244e7fc640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 0x7f24500ad550 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f24380108e0 tx=0x7f2438010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.792+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243800ec20 con 0x7f2450145dc0 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.792+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243800ed80 con 0x7f2450145dc0 2023-11-24T08:12:38.795 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.792+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243801c600 con 0x7f2450145dc0 2023-11-24T08:12:38.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.792+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f245009a720 con 0x7f2450145dc0 2023-11-24T08:12:38.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.792+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2450147cf0 con 0x7f2450145dc0 2023-11-24T08:12:38.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.793+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2450067470 con 0x7f2450145dc0 2023-11-24T08:12:38.796 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.794+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f243801c760 con 0x7f2450145dc0 2023-11-24T08:12:38.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.795+0000 7f24477fe640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f243006b410 0x7f243006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:38.797 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.795+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f243808a670 con 0x7f2450145dc0 2023-11-24T08:12:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.795+0000 7f244effd640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f243006b410 0x7f243006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:38.798 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.796+0000 7f244effd640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f243006b410 0x7f243006d8d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2440036000 tx=0x7f2440030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:38.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:38.800+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2438054f40 con 0x7f2450145dc0 2023-11-24T08:12:39.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.026+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f245006dd00 con 0x7f2450145dc0 2023-11-24T08:12:39.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.027+0000 7f24477fe640 1 -- 172.21.15.106:0/4063126101 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f2438058560 con 0x7f2450145dc0 2023-11-24T08:12:39.030 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:39.030 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:09.955346+0000","modified":"2023-11-24T08:12:13.029327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/4067546353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4067546353},{"type":"v1","addr":"172.21.15.106:6837","nonce":4067546353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:12:39.030 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2023-11-24T08:12:39.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.029+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f243006b410 msgr2=0x7f243006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.032 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.029+0000 7f245531d640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f243006b410 0x7f243006d8d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2440036000 tx=0x7f2440030000 comp rx=0 tx=0).stop 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.029+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 msgr2=0x7f24500ad550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.029+0000 7f245531d640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 0x7f24500ad550 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f24380108e0 tx=0x7f2438010da0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 shutdown_connections 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f243006b410 0x7f243006d8d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2450146e00 0x7f2450147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2450145dc0 0x7f24500ad550 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 --2- 172.21.15.106:0/4063126101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f243c0039a0 0x7f24500ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 >> 172.21.15.106:0/4063126101 conn(0x7f245006a920 msgr2=0x7f24500af350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 shutdown_connections 2023-11-24T08:12:39.033 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.030+0000 7f245531d640 1 -- 172.21.15.106:0/4063126101 wait complete. 2023-11-24T08:12:39.052 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:39.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.262+0000 7fed99e3d640 1 Processor -- start 2023-11-24T08:12:39.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.263+0000 7fed99e3d640 1 -- start start 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed99e3d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fed940cfb60 0x7fed94161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed99e3d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fed9415dc80 0x7fed9415e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed99e3d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed940ce0b0 con 0x7fed94157c70 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed99e3d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed94161960 con 0x7fed940cfb60 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed99e3d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9415e670 con 0x7fed9415dc80 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed937fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fed94157c70 0x7fed940cf450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42876/0 (socket says 172.21.15.106:42876) 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed937fe640 1 -- 172.21.15.106:0/2232764699 learned_addr learned my addr 172.21.15.106:0/2232764699 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:39.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.264+0000 7fed8bfff640 1 --2- 172.21.15.106:0/2232764699 >> v2:172.21.15.106:3300/0 conn(0x7fed940cfb60 0x7fed94161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed93fff640 1 --2- 172.21.15.106:0/2232764699 >> v2:172.21.15.156:3300/0 conn(0x7fed9415dc80 0x7fed9415e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed92ffd640 1 -- 172.21.15.106:0/2232764699 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1558193102 0 0) 0x7fed940ce0b0 con 0x7fed94157c70 2023-11-24T08:12:39.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed92ffd640 1 -- 172.21.15.106:0/2232764699 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed80003520 con 0x7fed94157c70 2023-11-24T08:12:39.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed8bfff640 1 -- 172.21.15.106:0/2232764699 >> v2:172.21.15.156:3300/0 conn(0x7fed9415dc80 msgr2=0x7fed9415e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:39.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed8bfff640 1 --2- 172.21.15.106:0/2232764699 >> v2:172.21.15.156:3300/0 conn(0x7fed9415dc80 0x7fed9415e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed8bfff640 1 -- 172.21.15.106:0/2232764699 >> v1:172.21.15.106:6789/0 conn(0x7fed94157c70 legacy=0x7fed940cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed8bfff640 1 -- 172.21.15.106:0/2232764699 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9415e8d0 con 0x7fed940cfb60 2023-11-24T08:12:39.269 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.265+0000 7fed93fff640 1 --2- 172.21.15.106:0/2232764699 >> v2:172.21.15.156:3300/0 conn(0x7fed9415dc80 0x7fed9415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.266+0000 7fed8bfff640 1 --2- 172.21.15.106:0/2232764699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed940cfb60 0x7fed94161df0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fed940d0130 tx=0x7fed7c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abeaab6ac7b161d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed92ffd640 1 -- 172.21.15.106:0/2232764699 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c02e070 con 0x7fed940cfb60 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed92ffd640 1 -- 172.21.15.106:0/2232764699 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c0315a0 con 0x7fed940cfb60 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed92ffd640 1 -- 172.21.15.106:0/2232764699 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c031930 con 0x7fed940cfb60 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 -- 172.21.15.106:0/2232764699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed940cfb60 msgr2=0x7fed94161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 --2- 172.21.15.106:0/2232764699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed940cfb60 0x7fed94161df0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fed940d0130 tx=0x7fed7c02b840 comp rx=0 tx=0).stop 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 -- 172.21.15.106:0/2232764699 shutdown_connections 2023-11-24T08:12:39.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 --2- 172.21.15.106:0/2232764699 >> v2:172.21.15.156:3300/0 conn(0x7fed9415dc80 0x7fed9415e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 --2- 172.21.15.106:0/2232764699 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed940cfb60 0x7fed94161df0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 -- 172.21.15.106:0/2232764699 >> 172.21.15.106:0/2232764699 conn(0x7fed9406a920 msgr2=0x7fed9406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:39.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 -- 172.21.15.106:0/2232764699 shutdown_connections 2023-11-24T08:12:39.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.267+0000 7fed99e3d640 1 -- 172.21.15.106:0/2232764699 wait complete. 2023-11-24T08:12:39.271 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.269+0000 7fed99e3d640 1 Processor -- start 2023-11-24T08:12:39.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.269+0000 7fed99e3d640 1 -- start start 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.270+0000 7fed99e3d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed940cfb60 0x7fed94155510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.270+0000 7fed99e3d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 0x7fed941f72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.270+0000 7fed99e3d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed9415dc80 0x7fed9409b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.270+0000 7fed99e3d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed940abfc0 con 0x7fed94157c70 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.270+0000 7fed99e3d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9409cd10 con 0x7fed9415dc80 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.270+0000 7fed99e3d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed94155a50 con 0x7fed940cfb60 2023-11-24T08:12:39.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 0x7fed941f72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 0x7fed941f72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36518/0 (socket says 172.21.15.106:36518) 2023-11-24T08:12:39.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 -- 172.21.15.106:0/322060646 learned_addr learned my addr 172.21.15.106:0/322060646 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed93fff640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed9415dc80 0x7fed9409b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 -- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed940cfb60 msgr2=0x7fed94155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed937fe640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed940cfb60 0x7fed94155510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed940cfb60 0x7fed94155510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 -- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed9415dc80 msgr2=0x7fed9409b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed9415dc80 0x7fed9409b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 -- 172.21.15.106:0/322060646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c009400 con 0x7fed94157c70 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.271+0000 7fed8bfff640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 0x7fed941f72f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000c00 tx=0x7fed7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.272+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032840 con 0x7fed94157c70 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.272+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c0329c0 con 0x7fed94157c70 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.272+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9409bef0 con 0x7fed94157c70 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.272+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032d70 con 0x7fed94157c70 2023-11-24T08:12:39.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.272+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed940ac210 con 0x7fed94157c70 2023-11-24T08:12:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.273+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed94067470 con 0x7fed94157c70 2023-11-24T08:12:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.274+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fed7c04d3f0 con 0x7fed94157c70 2023-11-24T08:12:39.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.275+0000 7fed90ff9640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fed7406b3e0 0x7fed7406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.275+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fed7c0abcf0 con 0x7fed94157c70 2023-11-24T08:12:39.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.275+0000 7fed937fe640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fed7406b3e0 0x7fed7406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.276+0000 7fed937fe640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fed7406b3e0 0x7fed7406d8a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed840011e0 tx=0x7fed84009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:39.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.279+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fed7c0763b0 con 0x7fed94157c70 2023-11-24T08:12:39.475 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.473+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fed940cd8f0 con 0x7fed94157c70 2023-11-24T08:12:39.477 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.475+0000 7fed90ff9640 1 -- 172.21.15.106:0/322060646 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16824 (secure 0 0 0) 0x7fed7c0799d0 con 0x7fed94157c70 2023-11-24T08:12:39.478 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:39.478 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:35.407263+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fed7406b3e0 msgr2=0x7fed7406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fed7406b3e0 0x7fed7406d8a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fed840011e0 tx=0x7fed84009040 comp rx=0 tx=0).stop 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 msgr2=0x7fed941f72f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 0x7fed941f72f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000c00 tx=0x7fed7c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 shutdown_connections 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed9415dc80 0x7fed9409b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fed94157c70 0x7fed941f72f0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fed7406b3e0 0x7fed7406d8a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 --2- 172.21.15.106:0/322060646 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fed940cfb60 0x7fed94155510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.477+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 >> 172.21.15.106:0/322060646 conn(0x7fed9406a920 msgr2=0x7fed9415fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.478+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 shutdown_connections 2023-11-24T08:12:39.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.478+0000 7fed99e3d640 1 -- 172.21.15.106:0/322060646 wait complete. 2023-11-24T08:12:39.500 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:12:07.722409+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:12:08.984297+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '77', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 77, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:39.500 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:39.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.716+0000 7f6b78f68640 1 Processor -- start 2023-11-24T08:12:39.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.716+0000 7f6b78f68640 1 -- start start 2023-11-24T08:12:39.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b78f68640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b74146eb0 0x7f6b74161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b78f68640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b7406e160 0x7f6b740ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b78f68640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b740cd530 con 0x7f6b740cd110 2023-11-24T08:12:39.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b78f68640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b74161a10 con 0x7f6b7406e160 2023-11-24T08:12:39.720 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b78f68640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74161cf0 con 0x7f6b74146eb0 2023-11-24T08:12:39.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b69d9b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b7406e160 0x7f6b740ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b7259c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b74146eb0 0x7f6b74161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b69d9b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6b7406e160 0x7f6b740ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:39600/0 (socket says 172.21.15.106:39600) 2023-11-24T08:12:39.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b69d9b640 1 -- 172.21.15.106:0/769357183 learned_addr learned my addr 172.21.15.106:0/769357183 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:39.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.718+0000 7f6b7259c640 1 --2- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b74146eb0 0x7f6b74161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:54836/0 (socket says 172.21.15.106:54836) 2023-11-24T08:12:39.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.719+0000 7f6b69d9b640 1 -- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b74146eb0 msgr2=0x7f6b74161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:39.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.719+0000 7f6b69d9b640 1 --2- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b74146eb0 0x7f6b74161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.719+0000 7f6b69d9b640 1 -- 172.21.15.106:0/769357183 >> v1:172.21.15.156:6789/0 conn(0x7f6b740cd110 legacy=0x7f6b7406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.719+0000 7f6b69d9b640 1 -- 172.21.15.106:0/769357183 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b74161fd0 con 0x7f6b7406e160 2023-11-24T08:12:39.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.720+0000 7f6b69d9b640 1 --2- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7406e160 0x7f6b740ccbd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009870 tx=0x7f6b5c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c661085aff565748 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:39.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.721+0000 7f6b71d9b640 1 -- 172.21.15.106:0/769357183 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c030840 con 0x7f6b7406e160 2023-11-24T08:12:39.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.721+0000 7f6b71d9b640 1 -- 172.21.15.106:0/769357183 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b5c0309a0 con 0x7f6b7406e160 2023-11-24T08:12:39.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.721+0000 7f6b71d9b640 1 -- 172.21.15.106:0/769357183 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5c030d30 con 0x7f6b7406e160 2023-11-24T08:12:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 -- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7406e160 msgr2=0x7f6b740ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 --2- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7406e160 0x7f6b740ccbd0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c009870 tx=0x7f6b5c02b980 comp rx=0 tx=0).stop 2023-11-24T08:12:39.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 -- 172.21.15.106:0/769357183 shutdown_connections 2023-11-24T08:12:39.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 --2- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b7406e160 0x7f6b740ccbd0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 --2- 172.21.15.106:0/769357183 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b74146eb0 0x7f6b74161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 -- 172.21.15.106:0/769357183 >> 172.21.15.106:0/769357183 conn(0x7f6b740b0c50 msgr2=0x7f6b740b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:39.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 -- 172.21.15.106:0/769357183 shutdown_connections 2023-11-24T08:12:39.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.722+0000 7f6b78f68640 1 -- 172.21.15.106:0/769357183 wait complete. 2023-11-24T08:12:39.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.724+0000 7f6b78f68640 1 Processor -- start 2023-11-24T08:12:39.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.724+0000 7f6b78f68640 1 -- start start 2023-11-24T08:12:39.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b78f68640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7406e160 0x7f6b74160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b78f68640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b740cd110 0x7f6b7415f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b78f68640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 0x7f6b7415fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b78f68640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b74160c90 con 0x7f6b74146eb0 2023-11-24T08:12:39.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b78f68640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741f3560 con 0x7f6b740cd110 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b78f68640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b741e7910 con 0x7f6b7406e160 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b72d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 0x7f6b7415fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b72d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 0x7f6b7415fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36528/0 (socket says 172.21.15.106:36528) 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.725+0000 7f6b72d9d640 1 -- 172.21.15.106:0/3505912004 learned_addr learned my addr 172.21.15.106:0/3505912004 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b69d9b640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b740cd110 0x7f6b7415f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b7259c640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7406e160 0x7f6b74160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b72d9d640 1 -- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7406e160 msgr2=0x7f6b74160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b72d9d640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7406e160 0x7f6b74160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b72d9d640 1 -- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b740cd110 msgr2=0x7f6b7415f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b72d9d640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b740cd110 0x7f6b7415f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b72d9d640 1 -- 172.21.15.106:0/3505912004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b5c0094d0 con 0x7f6b74146eb0 2023-11-24T08:12:39.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b72d9d640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 0x7f6b7415fbc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00c7d0 tx=0x7f6b6c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:39.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c003b70 con 0x7f6b74146eb0 2023-11-24T08:12:39.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c003cf0 con 0x7f6b74146eb0 2023-11-24T08:12:39.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00aa90 con 0x7f6b74146eb0 2023-11-24T08:12:39.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.726+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b74160100 con 0x7f6b74146eb0 2023-11-24T08:12:39.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.727+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b741e7300 con 0x7f6b74146eb0 2023-11-24T08:12:39.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.728+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b741e7a80 con 0x7f6b74146eb0 2023-11-24T08:12:39.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.729+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f6b6c00abf0 con 0x7f6b74146eb0 2023-11-24T08:12:39.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.729+0000 7f6b6b7fe640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b5406b410 0x7f6b5406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:39.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.730+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f6b6c089ee0 con 0x7f6b74146eb0 2023-11-24T08:12:39.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.730+0000 7f6b7259c640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b5406b410 0x7f6b5406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:39.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.730+0000 7f6b7259c640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b5406b410 0x7f6b5406d8d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009a30 tx=0x7f6b64008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:39.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.734+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6b6c08d030 con 0x7f6b74146eb0 2023-11-24T08:12:39.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.926+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b7404e8b0 con 0x7f6b74146eb0 2023-11-24T08:12:39.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.927+0000 7f6b6b7fe640 1 -- 172.21.15.106:0/3505912004 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b6c003e60 con 0x7f6b74146eb0 2023-11-24T08:12:39.929 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.929+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b5406b410 msgr2=0x7f6b5406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b5406b410 0x7f6b5406d8d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f6b64009a30 tx=0x7f6b64008040 comp rx=0 tx=0).stop 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 msgr2=0x7f6b7415fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 0x7f6b7415fbc0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00c7d0 tx=0x7f6b6c00cc90 comp rx=0 tx=0).stop 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 shutdown_connections 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6b74146eb0 0x7f6b7415fbc0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b740cd110 0x7f6b7415f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6b5406b410 0x7f6b5406d8d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 --2- 172.21.15.106:0/3505912004 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6b7406e160 0x7f6b74160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 >> 172.21.15.106:0/3505912004 conn(0x7f6b740b0c50 msgr2=0x7f6b740af180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 shutdown_connections 2023-11-24T08:12:39.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:39.930+0000 7f6b78f68640 1 -- 172.21.15.106:0/3505912004 wait complete. 2023-11-24T08:12:39.951 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:12:40.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.165+0000 7fbcaf59e640 1 Processor -- start 2023-11-24T08:12:40.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.165+0000 7fbcaf59e640 1 -- start start 2023-11-24T08:12:40.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbcaf59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcb0157c30 0x7fbcb00b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbcaf59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fbcb0072320 0x7fbcb006a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbcaf59e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcb015dce0 con 0x7fbcb0157850 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbcaf59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb015dad0 con 0x7fbcb0157c30 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbcaf59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcb00b3b70 con 0x7fbcb0072320 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbca5d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcb0157c30 0x7fbcb00b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbca5d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fbcb0157c30 0x7fbcb00b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36534/0 (socket says 172.21.15.106:36534) 2023-11-24T08:12:40.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.167+0000 7fbca5d9b640 1 -- 172.21.15.106:0/3020591263 learned_addr learned my addr 172.21.15.106:0/3020591263 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:40.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.168+0000 7fbcaed9d640 1 --2- 172.21.15.106:0/3020591263 >> v2:172.21.15.156:3300/0 conn(0x7fbcb0072320 0x7fbcb006a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.168+0000 7fbca5d9b640 1 -- 172.21.15.106:0/3020591263 >> v2:172.21.15.156:3300/0 conn(0x7fbcb0072320 msgr2=0x7fbcb006a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.168+0000 7fbca5d9b640 1 --2- 172.21.15.106:0/3020591263 >> v2:172.21.15.156:3300/0 conn(0x7fbcb0072320 0x7fbcb006a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.168+0000 7fbca5d9b640 1 -- 172.21.15.106:0/3020591263 >> v1:172.21.15.156:6789/0 conn(0x7fbcb0157850 legacy=0x7fbcb015e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.168+0000 7fbca5d9b640 1 -- 172.21.15.106:0/3020591263 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb00b35b0 con 0x7fbcb0157c30 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.169+0000 7fbca5d9b640 1 --2- 172.21.15.106:0/3020591263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00b2ed0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00af950 tx=0x7fbc9802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3674b5ad76e7095 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.169+0000 7fbcadd9b640 1 -- 172.21.15.106:0/3020591263 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98031870 con 0x7fbcb0157c30 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.169+0000 7fbcadd9b640 1 -- 172.21.15.106:0/3020591263 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc980319d0 con 0x7fbcb0157c30 2023-11-24T08:12:40.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.169+0000 7fbcadd9b640 1 -- 172.21.15.106:0/3020591263 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98031d60 con 0x7fbcb0157c30 2023-11-24T08:12:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 -- 172.21.15.106:0/3020591263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 msgr2=0x7fbcb00b2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/3020591263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00b2ed0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00af950 tx=0x7fbc9802b910 comp rx=0 tx=0).stop 2023-11-24T08:12:40.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 -- 172.21.15.106:0/3020591263 shutdown_connections 2023-11-24T08:12:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/3020591263 >> v2:172.21.15.156:3300/0 conn(0x7fbcb0072320 0x7fbcb006a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/3020591263 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00b2ed0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 -- 172.21.15.106:0/3020591263 >> 172.21.15.106:0/3020591263 conn(0x7fbcb00b1bd0 msgr2=0x7fbcb00b1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 -- 172.21.15.106:0/3020591263 shutdown_connections 2023-11-24T08:12:40.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.170+0000 7fbcaf59e640 1 -- 172.21.15.106:0/3020591263 wait complete. 2023-11-24T08:12:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.172+0000 7fbcaf59e640 1 Processor -- start 2023-11-24T08:12:40.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.173+0000 7fbcaf59e640 1 -- start start 2023-11-24T08:12:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaf59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcb0072320 0x7fbcb014ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaf59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcb0157850 0x7fbcb0160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaf59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00ccb60 con 0x7fbcb0157c30 2023-11-24T08:12:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaf59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00ccd50 con 0x7fbcb0072320 2023-11-24T08:12:40.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaf59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb00cd030 con 0x7fbcb0157850 2023-11-24T08:12:40.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaed9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00cc620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36544/0 (socket says 172.21.15.106:36544) 2023-11-24T08:12:40.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcaed9d640 1 -- 172.21.15.106:0/2024141054 learned_addr learned my addr 172.21.15.106:0/2024141054 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:40.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.174+0000 7fbcae59c640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcb0072320 0x7fbcb014ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbca5d9b640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcb0157850 0x7fbcb0160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaed9d640 1 -- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcb0157850 msgr2=0x7fbcb0160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaed9d640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcb0157850 0x7fbcb0160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaed9d640 1 -- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcb0072320 msgr2=0x7fbcb014ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaed9d640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcb0072320 0x7fbcb014ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaed9d640 1 -- 172.21.15.106:0/2024141054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc980094d0 con 0x7fbcb0157c30 2023-11-24T08:12:40.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbca5d9b640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcb0157850 0x7fbcb0160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:40.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaed9d640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00cc620 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbca8007e40 tx=0x7fbca800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:40.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca8014620 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca800ee90 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb00cd310 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca80149e0 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.175+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb01f31d0 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.177+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb006add0 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.177+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbca8014b40 con 0x7fbcb0157c30 2023-11-24T08:12:40.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.178+0000 7fbca77fe640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbc9406b3e0 0x7fbc9406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.178+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fbca808ab50 con 0x7fbcb0157c30 2023-11-24T08:12:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.179+0000 7fbcae59c640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbc9406b3e0 0x7fbc9406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.182 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.179+0000 7fbcae59c640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbc9406b3e0 0x7fbc9406d8a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbca0004ea0 tx=0x7fbca0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:40.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.183+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbca80552e0 con 0x7fbcb0157c30 2023-11-24T08:12:40.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.373+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbcb004e840 con 0x7fbcb0157c30 2023-11-24T08:12:40.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.374+0000 7fbca77fe640 1 -- 172.21.15.106:0/2024141054 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v77) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbca8058900 con 0x7fbcb0157c30 2023-11-24T08:12:40.376 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbc9406b3e0 msgr2=0x7fbc9406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbc9406b3e0 0x7fbc9406d8a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fbca0004ea0 tx=0x7fbca0009040 comp rx=0 tx=0).stop 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 msgr2=0x7fbcb00cc620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00cc620 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fbca8007e40 tx=0x7fbca800e4b0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 shutdown_connections 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbcb0157c30 0x7fbcb00cc620 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbc9406b3e0 0x7fbc9406d8a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbcb0157850 0x7fbcb0160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 --2- 172.21.15.106:0/2024141054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcb0072320 0x7fbcb014ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 >> 172.21.15.106:0/2024141054 conn(0x7fbcb00b1bd0 msgr2=0x7fbcb00afb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:40.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 shutdown_connections 2023-11-24T08:12:40.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.377+0000 7fbcaf59e640 1 -- 172.21.15.106:0/2024141054 wait complete. 2023-11-24T08:12:40.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:12:40.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.612+0000 7fcd27abb640 1 Processor -- start 2023-11-24T08:12:40.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.612+0000 7fcd27abb640 1 -- start start 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd27abb640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd2006de10 0x7fcd2015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd27abb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcd200c49f0 0x7fcd200705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd27abb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd2015ec90 con 0x7fcd20072160 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd27abb640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd200cd530 con 0x7fcd2006de10 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd27abb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd200c4ed0 con 0x7fcd200c49f0 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd25056640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd2006de10 0x7fcd2015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd25056640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcd2006de10 0x7fcd2015e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36554/0 (socket says 172.21.15.106:36554) 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd25857640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcd20072160 0x7fcd2006d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42886/0 (socket says 172.21.15.106:42886) 2023-11-24T08:12:40.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.614+0000 7fcd25056640 1 -- 172.21.15.106:0/2654295429 learned_addr learned my addr 172.21.15.106:0/2654295429 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:40.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.615+0000 7fcd25056640 1 -- 172.21.15.106:0/2654295429 >> v2:172.21.15.156:3300/0 conn(0x7fcd200c49f0 msgr2=0x7fcd200705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:40.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.615+0000 7fcd26058640 1 --2- 172.21.15.106:0/2654295429 >> v2:172.21.15.156:3300/0 conn(0x7fcd200c49f0 0x7fcd200705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.615+0000 7fcd25056640 1 --2- 172.21.15.106:0/2654295429 >> v2:172.21.15.156:3300/0 conn(0x7fcd200c49f0 0x7fcd200705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.615+0000 7fcd25056640 1 -- 172.21.15.106:0/2654295429 >> v1:172.21.15.106:6789/0 conn(0x7fcd20072160 legacy=0x7fcd2006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.615+0000 7fcd24855640 1 -- 172.21.15.106:0/2654295429 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1435161794 0 0) 0x7fcd1c001110 con 0x7fcd20072160 2023-11-24T08:12:40.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.615+0000 7fcd25056640 1 -- 172.21.15.106:0/2654295429 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd200c51b0 con 0x7fcd2006de10 2023-11-24T08:12:40.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.616+0000 7fcd25056640 1 --2- 172.21.15.106:0/2654295429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd2006de10 0x7fcd2015e750 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fcd20072a30 tx=0x7fcd1002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc2d22f39df216cf server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:40.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.616+0000 7fcd24855640 1 -- 172.21.15.106:0/2654295429 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10030860 con 0x7fcd2006de10 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.616+0000 7fcd24855640 1 -- 172.21.15.106:0/2654295429 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd100309c0 con 0x7fcd2006de10 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.616+0000 7fcd24855640 1 -- 172.21.15.106:0/2654295429 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10030d50 con 0x7fcd2006de10 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 -- 172.21.15.106:0/2654295429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd2006de10 msgr2=0x7fcd2015e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 --2- 172.21.15.106:0/2654295429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd2006de10 0x7fcd2015e750 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fcd20072a30 tx=0x7fcd1002b9a0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 -- 172.21.15.106:0/2654295429 shutdown_connections 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 --2- 172.21.15.106:0/2654295429 >> v2:172.21.15.156:3300/0 conn(0x7fcd200c49f0 0x7fcd200705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 --2- 172.21.15.106:0/2654295429 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd2006de10 0x7fcd2015e750 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 -- 172.21.15.106:0/2654295429 >> 172.21.15.106:0/2654295429 conn(0x7fcd200b0c50 msgr2=0x7fcd200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 -- 172.21.15.106:0/2654295429 shutdown_connections 2023-11-24T08:12:40.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.617+0000 7fcd27abb640 1 -- 172.21.15.106:0/2654295429 wait complete. 2023-11-24T08:12:40.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.619+0000 7fcd27abb640 1 Processor -- start 2023-11-24T08:12:40.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.619+0000 7fcd27abb640 1 -- start start 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd27abb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd2006de10 0x7fcd200cb6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd27abb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 0x7fcd2009c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd27abb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd200c49f0 0x7fcd20142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd27abb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200cbc30 con 0x7fcd20072160 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd27abb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd200cb260 con 0x7fcd200c49f0 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd27abb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2015a810 con 0x7fcd2006de10 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd25056640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 0x7fcd2009c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd25056640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 0x7fcd2009c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36558/0 (socket says 172.21.15.106:36558) 2023-11-24T08:12:40.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.620+0000 7fcd25056640 1 -- 172.21.15.106:0/540896222 learned_addr learned my addr 172.21.15.106:0/540896222 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:40.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25857640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd2006de10 0x7fcd200cb6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25056640 1 -- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd2006de10 msgr2=0x7fcd200cb6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25056640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd2006de10 0x7fcd200cb6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25056640 1 -- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd200c49f0 msgr2=0x7fcd20142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25056640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd200c49f0 0x7fcd20142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25056640 1 -- 172.21.15.106:0/540896222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd100094d0 con 0x7fcd20072160 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25857640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd2006de10 0x7fcd200cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd25056640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 0x7fcd2009c2f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd10004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.621+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10030830 con 0x7fcd20072160 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.622+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10043400 con 0x7fcd20072160 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.622+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd100437b0 con 0x7fcd20072160 2023-11-24T08:12:40.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.622+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2015aaf0 con 0x7fcd20072160 2023-11-24T08:12:40.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.622+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd200cabe0 con 0x7fcd20072160 2023-11-24T08:12:40.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.623+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd200cb3f0 con 0x7fcd20072160 2023-11-24T08:12:40.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.624+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fcd10043910 con 0x7fcd20072160 2023-11-24T08:12:40.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.624+0000 7fcd167fc640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd0406b3e0 0x7fcd0406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:40.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.624+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fcd1003e070 con 0x7fcd20072160 2023-11-24T08:12:40.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.625+0000 7fcd25857640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd0406b3e0 0x7fcd0406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:40.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.625+0000 7fcd25857640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd0406b3e0 0x7fcd0406d8a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000f60 tx=0x7fcd1c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:40.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.630+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcd10077d40 con 0x7fcd20072160 2023-11-24T08:12:40.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.823+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcd2015ace0 con 0x7fcd20072160 2023-11-24T08:12:40.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.824+0000 7fcd167fc640 1 -- 172.21.15.106:0/540896222 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v77) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcd1007b360 con 0x7fcd20072160 2023-11-24T08:12:40.826 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:12:40.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.826+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd0406b3e0 msgr2=0x7fcd0406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.826+0000 7fcd27abb640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd0406b3e0 0x7fcd0406d8a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcd1c000f60 tx=0x7fcd1c009040 comp rx=0 tx=0).stop 2023-11-24T08:12:40.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.826+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 msgr2=0x7fcd2009c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:40.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.826+0000 7fcd27abb640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 0x7fcd2009c2f0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7fcd10000c00 tx=0x7fcd10004730 comp rx=0 tx=0).stop 2023-11-24T08:12:40.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.826+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 shutdown_connections 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcd200c49f0 0x7fcd20142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcd20072160 0x7fcd2009c2f0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcd0406b3e0 0x7fcd0406d8a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 --2- 172.21.15.106:0/540896222 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcd2006de10 0x7fcd200cb6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 >> 172.21.15.106:0/540896222 conn(0x7fcd200b0c50 msgr2=0x7fcd2006c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 shutdown_connections 2023-11-24T08:12:40.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:40.827+0000 7fcd27abb640 1 -- 172.21.15.106:0/540896222 wait complete. 2023-11-24T08:12:40.845 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:41.054 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.053+0000 7f2bcc7de640 1 Processor -- start 2023-11-24T08:12:41.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.053+0000 7f2bcc7de640 1 -- start start 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.054+0000 7f2bcc7de640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2bc40cf450 0x7f2bc40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.054+0000 7f2bcc7de640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cdfb0 0x7f2bc40ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.054+0000 7f2bcc7de640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bc40cece0 con 0x7f2bc415c7f0 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.054+0000 7f2bcc7de640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bc40cfe40 con 0x7f2bc40cf450 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.054+0000 7f2bcc7de640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4156010 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2bc40cf450 0x7f2bc40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2bc40cf450 0x7f2bc40cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36572/0 (socket says 172.21.15.106:36572) 2023-11-24T08:12:41.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 -- 172.21.15.106:0/761579843 learned_addr learned my addr 172.21.15.106:0/761579843 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:41.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bcad7b640 1 --2- 172.21.15.106:0/761579843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cdfb0 0x7f2bc40ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 -- 172.21.15.106:0/761579843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cdfb0 msgr2=0x7f2bc40ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 --2- 172.21.15.106:0/761579843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cdfb0 0x7f2bc40ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 -- 172.21.15.106:0/761579843 >> v1:172.21.15.106:6789/0 conn(0x7f2bc415c7f0 legacy=0x7f2bc406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9d79640 1 -- 172.21.15.106:0/761579843 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc406d040 con 0x7f2bc40cf450 2023-11-24T08:12:41.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.055+0000 7f2bc9578640 1 -- 172.21.15.106:0/761579843 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113470699 0 0) 0x7f2bc40cece0 con 0x7f2bc415c7f0 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.056+0000 7f2bc9d79640 1 --2- 172.21.15.106:0/761579843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cf450 0x7f2bc40cf900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009870 tx=0x7f2bb802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62ba2453ca66e826 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bc9578640 1 -- 172.21.15.106:0/761579843 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8030840 con 0x7f2bc40cf450 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bc9578640 1 -- 172.21.15.106:0/761579843 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bb80309a0 con 0x7f2bc40cf450 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bc9578640 1 -- 172.21.15.106:0/761579843 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8030d30 con 0x7f2bc40cf450 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 -- 172.21.15.106:0/761579843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cf450 msgr2=0x7f2bc40cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/761579843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cf450 0x7f2bc40cf900 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8009870 tx=0x7f2bb802b980 comp rx=0 tx=0).stop 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 -- 172.21.15.106:0/761579843 shutdown_connections 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/761579843 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cdfb0 0x7f2bc40ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.059 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/761579843 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cf450 0x7f2bc40cf900 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 -- 172.21.15.106:0/761579843 >> 172.21.15.106:0/761579843 conn(0x7f2bc406a920 msgr2=0x7f2bc406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:41.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 -- 172.21.15.106:0/761579843 shutdown_connections 2023-11-24T08:12:41.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.057+0000 7f2bcc7de640 1 -- 172.21.15.106:0/761579843 wait complete. 2023-11-24T08:12:41.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.059+0000 7f2bcc7de640 1 Processor -- start 2023-11-24T08:12:41.060 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.059+0000 7f2bcc7de640 1 -- start start 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bcc7de640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 0x7f2bc4153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bcc7de640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cf450 0x7f2bc415e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bcc7de640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bc415c7f0 0x7f2bc4099ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bcc7de640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4156150 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bcc7de640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc4153890 con 0x7f2bc415c7f0 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bcc7de640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc415d150 con 0x7f2bc40cf450 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.060+0000 7f2bca57a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 0x7f2bc4153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 0x7f2bc4153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36576/0 (socket says 172.21.15.106:36576) 2023-11-24T08:12:41.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 -- 172.21.15.106:0/2343344037 learned_addr learned my addr 172.21.15.106:0/2343344037 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:41.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 -- 172.21.15.106:0/2343344037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cf450 msgr2=0x7f2bc415e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cf450 0x7f2bc415e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 -- 172.21.15.106:0/2343344037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bc415c7f0 msgr2=0x7f2bc4099ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bc415c7f0 0x7f2bc4099ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 -- 172.21.15.106:0/2343344037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80094d0 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bca57a640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 0x7f2bc4153350 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c410 tx=0x7f2bc000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.061+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc00035f0 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.062+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc0003770 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.062+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc415d430 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.062+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0003b20 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.062+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc415d9f0 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.063+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc41562c0 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.065 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.064+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f2bc0003c80 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.065+0000 7f2bb6ffd640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ba806b3e0 0x7f2ba806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.065+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f2bc00896d0 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.065+0000 7f2bc9d79640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ba806b3e0 0x7f2ba806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.066+0000 7f2bc9d79640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ba806b3e0 0x7f2ba806d8a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8036000 tx=0x7f2bb803b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.069+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2bc0053ef0 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.295+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bc4067470 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.297 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.296+0000 7f2bb6ffd640 1 -- 172.21.15.106:0/2343344037 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f2bc0057510 con 0x7f2bc40cdfb0 2023-11-24T08:12:41.298 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:41.298 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:09.955346+0000","modified":"2023-11-24T08:12:13.029327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/4067546353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4067546353},{"type":"v1","addr":"172.21.15.106:6837","nonce":4067546353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:12:41.298 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ba806b3e0 msgr2=0x7f2ba806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ba806b3e0 0x7f2ba806d8a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8036000 tx=0x7f2bb803b040 comp rx=0 tx=0).stop 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 msgr2=0x7f2bc4153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 0x7f2bc4153350 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c410 tx=0x7f2bc000c8d0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 shutdown_connections 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bc415c7f0 0x7f2bc4099ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2ba806b3e0 0x7f2ba806d8a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2bc40cf450 0x7f2bc415e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 --2- 172.21.15.106:0/2343344037 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2bc40cdfb0 0x7f2bc4153350 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.300 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 >> 172.21.15.106:0/2343344037 conn(0x7f2bc406a920 msgr2=0x7f2bc406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:41.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 shutdown_connections 2023-11-24T08:12:41.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.298+0000 7f2bcc7de640 1 -- 172.21.15.106:0/2343344037 wait complete. 2023-11-24T08:12:41.318 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:41.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.525+0000 7f32d753c640 1 Processor -- start 2023-11-24T08:12:41.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.525+0000 7f32d753c640 1 -- start start 2023-11-24T08:12:41.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.526+0000 7f32d753c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 0x7f32d015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.526+0000 7f32d753c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00cec00 con 0x7f32d006d6c0 2023-11-24T08:12:41.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.526+0000 7f32d753c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32d00b1890 con 0x7f32d006db20 2023-11-24T08:12:41.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.526+0000 7f32d753c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d013f9e0 con 0x7f32d013d120 2023-11-24T08:12:41.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.526+0000 7f32d5d39640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f32d006d6c0 0x7f32d00b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42894/0 (socket says 172.21.15.106:42894) 2023-11-24T08:12:41.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5d39640 1 -- 172.21.15.106:0/1093633794 learned_addr learned my addr 172.21.15.106:0/1093633794 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:41.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d653a640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 0x7f32d015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245689340 0 0) 0x7f32d00cec00 con 0x7f32d006d6c0 2023-11-24T08:12:41.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32bc005730 con 0x7f32d006d6c0 2023-11-24T08:12:41.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3384407885 0 0) 0x7f32bc005730 con 0x7f32d006d6c0 2023-11-24T08:12:41.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 msgr2=0x7f32d015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 0x7f32d015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 >> v1:172.21.15.156:6789/0 conn(0x7f32d006db20 legacy=0x7f32d00b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d013fcc0 con 0x7f32d006d6c0 2023-11-24T08:12:41.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f32c0001670 con 0x7f32d006d6c0 2023-11-24T08:12:41.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 >> v1:172.21.15.106:6789/0 conn(0x7f32d006d6c0 legacy=0x7f32d00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32bc005bb0 0x7f32bc005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.527+0000 7f32d5538640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5538640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 0x7f32bc00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d013f9e0 con 0x7f32bc0069d0 2023-11-24T08:12:41.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc0076f0 con 0x7f32bc005bb0 2023-11-24T08:12:41.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32bc00e0d0 con 0x7f32d013d120 2023-11-24T08:12:41.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5d39640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d653a640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32bc005bb0 0x7f32bc005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5d39640 1 -- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 msgr2=0x7f32bc00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5d39640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d013d120 0x7f32bc00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5d39640 1 -- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32bc005bb0 msgr2=0x7f32bc005fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5d39640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32bc005bb0 0x7f32bc005fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.528+0000 7f32d5d39640 1 -- 172.21.15.106:0/1093633794 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32bc005730 con 0x7f32bc0069d0 2023-11-24T08:12:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.529+0000 7f32d5d39640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c240 tx=0x7f32bc007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e067204fa37f215 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c0005ad0 con 0x7f32bc0069d0 2023-11-24T08:12:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c0005c30 con 0x7f32bc0069d0 2023-11-24T08:12:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d5538640 1 -- 172.21.15.106:0/1093633794 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c002caf0 con 0x7f32bc0069d0 2023-11-24T08:12:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d753c640 1 -- 172.21.15.106:0/1093633794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 msgr2=0x7f32bc006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d753c640 1 --2- 172.21.15.106:0/1093633794 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32bc006e50 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f32c000c240 tx=0x7f32bc007a00 comp rx=0 tx=0).stop 2023-11-24T08:12:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d6d3b640 1 -- 172.21.15.106:0/1093633794 reap_dead start 2023-11-24T08:12:41.540 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.530+0000 7f32d753c640 1 -- 172.21.15.106:0/1093633794 shutdown_connections 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.531+0000 7f32d753c640 1 -- 172.21.15.106:0/1093633794 >> 172.21.15.106:0/1093633794 conn(0x7f32d00cb830 msgr2=0x7f32d00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.531+0000 7f32d753c640 1 -- 172.21.15.106:0/1093633794 shutdown_connections 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.531+0000 7f32d753c640 1 -- 172.21.15.106:0/1093633794 wait complete. 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.532+0000 7f32d753c640 1 Processor -- start 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 -- start start 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32d01f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d015c840 0x7f32d01f6520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32d00b3d20 0x7f32d00b01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01f73e0 con 0x7f32bc0069d0 2023-11-24T08:12:41.541 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01f6e20 con 0x7f32d00b3d20 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.533+0000 7f32d753c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32d01f7100 con 0x7f32d015c840 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32d01f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32d01f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36594/0 (socket says 172.21.15.106:36594) 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 -- 172.21.15.106:0/3032790527 learned_addr learned my addr 172.21.15.106:0/3032790527 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 -- 172.21.15.106:0/3032790527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d015c840 msgr2=0x7f32d01f6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d015c840 0x7f32d01f6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 -- 172.21.15.106:0/3032790527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32d00b3d20 msgr2=0x7f32d00b01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.542 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32d00b3d20 0x7f32d00b01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 -- 172.21.15.106:0/3032790527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d01f30d0 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.534+0000 7f32d653a640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32d01f38d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00f620 tx=0x7f32cc00fae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.535+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc014870 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.535+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32cc0149d0 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.535+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32cc014d60 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.535+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32d01f33b0 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.535+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32d01f6a60 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.536+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32d00cb4d0 con 0x7f32bc0069d0 2023-11-24T08:12:41.543 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.537+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f32cc015410 con 0x7f32bc0069d0 2023-11-24T08:12:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.538+0000 7f32c6ffd640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32b406b410 0x7f32b406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.538+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f32cc08ab30 con 0x7f32bc0069d0 2023-11-24T08:12:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.538+0000 7f32d5d39640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32b406b410 0x7f32b406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.544 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.539+0000 7f32d5d39640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32b406b410 0x7f32b406d8d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f32c0034000 tx=0x7f32c0007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.545 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.542+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32cc0552c0 con 0x7f32bc0069d0 2023-11-24T08:12:41.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.732+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32d01f2f30 con 0x7f32bc0069d0 2023-11-24T08:12:41.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.735+0000 7f32c6ffd640 1 -- 172.21.15.106:0/3032790527 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16824 (secure 0 0 0) 0x7f32d01f2f30 con 0x7f32bc0069d0 2023-11-24T08:12:41.736 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:41.737 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:35.407263+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:41.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.737+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32b406b410 msgr2=0x7f32b406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.737+0000 7f32d753c640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32b406b410 0x7f32b406d8d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f32c0034000 tx=0x7f32c0007000 comp rx=0 tx=0).stop 2023-11-24T08:12:41.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.737+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 msgr2=0x7f32d01f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.737+0000 7f32d753c640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32d01f38d0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f32cc00f620 tx=0x7f32cc00fae0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 shutdown_connections 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32d00b3d20 0x7f32d00b01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f32b406b410 0x7f32b406d8d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32d015c840 0x7f32d01f6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 --2- 172.21.15.106:0/3032790527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32bc0069d0 0x7f32d01f38d0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 >> 172.21.15.106:0/3032790527 conn(0x7f32d00cb830 msgr2=0x7f32d0071bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 shutdown_connections 2023-11-24T08:12:41.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.738+0000 7f32d753c640 1 -- 172.21.15.106:0/3032790527 wait complete. 2023-11-24T08:12:41.759 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:12:41.760 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:41.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.963+0000 7f3781c48640 1 Processor -- start 2023-11-24T08:12:41.965 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.963+0000 7f3781c48640 1 -- start start 2023-11-24T08:12:41.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.964+0000 7f3781c48640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f377c0b2250 0x7f377c15df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.969 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f3781c48640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f377c0b2630 0x7f377c0ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f3781c48640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c15b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f3781c48640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377c15a990 con 0x7f377c0b2630 2023-11-24T08:12:41.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f3781c48640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f377c15ab80 con 0x7f377c0b2250 2023-11-24T08:12:41.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f3781c48640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c15e680 con 0x7f377c0b2aa0 2023-11-24T08:12:41.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f377c0b2630 0x7f377c0ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f377c0b2630 0x7f377c0ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36600/0 (socket says 172.21.15.106:36600) 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 -- 172.21.15.106:0/1752221473 learned_addr learned my addr 172.21.15.106:0/1752221473 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377b7fe640 1 --2- 172.21.15.106:0/1752221473 >> v2:172.21.15.156:3300/0 conn(0x7f377c0b2250 0x7f377c15df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377bfff640 1 --2- 172.21.15.106:0/1752221473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c15b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 -- 172.21.15.106:0/1752221473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 msgr2=0x7f377c15b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 --2- 172.21.15.106:0/1752221473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c15b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 -- 172.21.15.106:0/1752221473 >> v2:172.21.15.156:3300/0 conn(0x7f377c0b2250 msgr2=0x7f377c15df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 --2- 172.21.15.106:0/1752221473 >> v2:172.21.15.156:3300/0 conn(0x7f377c0b2250 0x7f377c15df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377affd640 1 -- 172.21.15.106:0/1752221473 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f377c15e960 con 0x7f377c0b2630 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.965+0000 7f377b7fe640 1 --2- 172.21.15.106:0/1752221473 >> v2:172.21.15.156:3300/0 conn(0x7f377c0b2250 0x7f377c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.966+0000 7f377affd640 1 --2- 172.21.15.106:0/1752221473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2630 0x7f377c0ccf30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f377c072a70 tx=0x7f376402e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6be42935710c00e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f377a7fc640 1 -- 172.21.15.106:0/1752221473 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376402ea80 con 0x7f377c0b2630 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f377a7fc640 1 -- 172.21.15.106:0/1752221473 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376402ebe0 con 0x7f377c0b2630 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f377a7fc640 1 -- 172.21.15.106:0/1752221473 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3764009600 con 0x7f377c0b2630 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 -- 172.21.15.106:0/1752221473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2630 msgr2=0x7f377c0ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 --2- 172.21.15.106:0/1752221473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2630 0x7f377c0ccf30 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f377c072a70 tx=0x7f376402e0e0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 -- 172.21.15.106:0/1752221473 shutdown_connections 2023-11-24T08:12:41.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 --2- 172.21.15.106:0/1752221473 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c15b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 --2- 172.21.15.106:0/1752221473 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2630 0x7f377c0ccf30 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 --2- 172.21.15.106:0/1752221473 >> v2:172.21.15.156:3300/0 conn(0x7f377c0b2250 0x7f377c15df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 -- 172.21.15.106:0/1752221473 >> 172.21.15.106:0/1752221473 conn(0x7f377c06a920 msgr2=0x7f377c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:41.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 -- 172.21.15.106:0/1752221473 shutdown_connections 2023-11-24T08:12:41.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.967+0000 7f3781c48640 1 -- 172.21.15.106:0/1752221473 wait complete. 2023-11-24T08:12:41.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.969+0000 7f3781c48640 1 Processor -- start 2023-11-24T08:12:41.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.969+0000 7f3781c48640 1 -- start start 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f3781c48640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 0x7f377c0c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f3781c48640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c0b2630 0x7f377c0abfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f3781c48640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c0ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f3781c48640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c1570c0 con 0x7f377c0b2250 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f3781c48640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f377c156020 con 0x7f377c0b2630 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f3781c48640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f377c156300 con 0x7f377c0b2aa0 2023-11-24T08:12:41.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f377b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 0x7f377c0c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f377b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 0x7f377c0c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36602/0 (socket says 172.21.15.106:36602) 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.970+0000 7f377b7fe640 1 -- 172.21.15.106:0/2854308322 learned_addr learned my addr 172.21.15.106:0/2854308322 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f377b7fe640 1 -- 172.21.15.106:0/2854308322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 msgr2=0x7f377c0ac500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f377b7fe640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c0ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f377b7fe640 1 -- 172.21.15.106:0/2854308322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c0b2630 msgr2=0x7f377c0abfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f377b7fe640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c0b2630 0x7f377c0abfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f377b7fe640 1 -- 172.21.15.106:0/2854308322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376400bd40 con 0x7f377c0b2250 2023-11-24T08:12:41.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f377b7fe640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 0x7f377c0c9b30 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c7c0 tx=0x7f376c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c007a30 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f376c007b90 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f377c1565e0 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.971+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f376c00ce70 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.972+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f377c156970 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.973+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f377c156de0 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.974+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f376c0113f0 con 0x7f377c0b2250 2023-11-24T08:12:41.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.974+0000 7f3773fff640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f375c06b3e0 0x7f375c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.974+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f376c08a410 con 0x7f377c0b2250 2023-11-24T08:12:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.975+0000 7f377affd640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f375c06b3e0 0x7f375c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:41.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.975+0000 7f377affd640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f375c06b3e0 0x7f375c06d8a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3764037000 tx=0x7f3764030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:41.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:41.979+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f377c156de0 con 0x7f377c0b2250 2023-11-24T08:12:42.190 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.189+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f377c156770 con 0x7f377c0b2250 2023-11-24T08:12:42.191 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.190+0000 7f3773fff640 1 -- 172.21.15.106:0/2854308322 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f376c054a40 con 0x7f377c0b2250 2023-11-24T08:12:42.192 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:42.192 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:09.955346+0000","modified":"2023-11-24T08:12:13.029327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/4067546353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4067546353},{"type":"v1","addr":"172.21.15.106:6837","nonce":4067546353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:12:42.192 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2023-11-24T08:12:42.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.192+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f375c06b3e0 msgr2=0x7f375c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.192+0000 7f3781c48640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f375c06b3e0 0x7f375c06d8a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f3764037000 tx=0x7f3764030000 comp rx=0 tx=0).stop 2023-11-24T08:12:42.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.192+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 msgr2=0x7f377c0c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.192+0000 7f3781c48640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 0x7f377c0c9b30 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f376c00c7c0 tx=0x7f376c00cc80 comp rx=0 tx=0).stop 2023-11-24T08:12:42.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 shutdown_connections 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f375c06b3e0 0x7f375c06d8a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f377c0b2aa0 0x7f377c0ac500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f377c0b2630 0x7f377c0abfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 --2- 172.21.15.106:0/2854308322 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f377c0b2250 0x7f377c0c9b30 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 >> 172.21.15.106:0/2854308322 conn(0x7f377c06a920 msgr2=0x7f377c158e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 shutdown_connections 2023-11-24T08:12:42.195 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.193+0000 7f3781c48640 1 -- 172.21.15.106:0/2854308322 wait complete. 2023-11-24T08:12:42.214 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:42.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.418+0000 7f1e0358a640 1 Processor -- start 2023-11-24T08:12:42.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.418+0000 7f1e0358a640 1 -- start start 2023-11-24T08:12:42.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.419+0000 7f1e0358a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1dfc0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.419+0000 7f1e0358a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc0cece0 con 0x7f1dfc0cf450 2023-11-24T08:12:42.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.419+0000 7f1e0358a640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dfc06d040 con 0x7f1dfc15c7f0 2023-11-24T08:12:42.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.419+0000 7f1e0358a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cfe40 con 0x7f1dfc0ce6c0 2023-11-24T08:12:42.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1e00b25640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1dfc0cf450 0x7f1dfc0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42906/0 (socket says 172.21.15.106:42906) 2023-11-24T08:12:42.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1e00b25640 1 -- 172.21.15.106:0/3920696059 learned_addr learned my addr 172.21.15.106:0/3920696059 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1e01b27640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1dfc0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2817479427 0 0) 0x7f1dfc0cece0 con 0x7f1dfc0cf450 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1de8003520 con 0x7f1dfc0cf450 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4050147964 0 0) 0x7f1de8003520 con 0x7f1dfc0cf450 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 msgr2=0x7f1dfc0cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.420+0000 7f1df3fff640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1dfc0cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 >> v1:172.21.15.156:6789/0 conn(0x7f1dfc15c7f0 legacy=0x7f1dfc06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc06d320 con 0x7f1dfc0cf450 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f1dec001670 con 0x7f1dfc0cf450 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 >> v1:172.21.15.106:6789/0 conn(0x7f1dfc0cf450 legacy=0x7f1dfc0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de80039a0 0x7f1de8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1de80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de8004d00 0x7f1de8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0cfe40 con 0x7f1de8004d00 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de800ba30 con 0x7f1de80039a0 2023-11-24T08:12:42.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de800bbf0 con 0x7f1dfc0ce6c0 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e01b27640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de8004d00 0x7f1de8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e01326640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de80039a0 0x7f1de8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e00b25640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1de80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e01b27640 1 -- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 msgr2=0x7f1de80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e01b27640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1de80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e01b27640 1 -- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de80039a0 msgr2=0x7f1de8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.421+0000 7f1e01b27640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de80039a0 0x7f1de8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.422+0000 7f1e01b27640 1 -- 172.21.15.106:0/3920696059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de8003520 con 0x7f1de8004d00 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.422+0000 7f1e01326640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de80039a0 0x7f1de8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.422+0000 7f1e00b25640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc0ce6c0 0x7f1de80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.423+0000 7f1e01b27640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de8004d00 0x7f1de8005180 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009a90 tx=0x7f1de80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e89b93b1d990468a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.423+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df402c680 con 0x7f1de8004d00 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.423+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df402c7e0 con 0x7f1de8004d00 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.423+0000 7f1df3fff640 1 -- 172.21.15.106:0/3920696059 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df402cb70 con 0x7f1de8004d00 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e0358a640 1 -- 172.21.15.106:0/3920696059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de8004d00 msgr2=0x7f1de8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e0358a640 1 --2- 172.21.15.106:0/3920696059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1de8004d00 0x7f1de8005180 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009a90 tx=0x7f1de80057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e01b27640 1 -- 172.21.15.106:0/3920696059 reap_dead start 2023-11-24T08:12:42.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e0358a640 1 -- 172.21.15.106:0/3920696059 shutdown_connections 2023-11-24T08:12:42.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e0358a640 1 -- 172.21.15.106:0/3920696059 >> 172.21.15.106:0/3920696059 conn(0x7f1dfc06a920 msgr2=0x7f1dfc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:42.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e0358a640 1 -- 172.21.15.106:0/3920696059 shutdown_connections 2023-11-24T08:12:42.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.424+0000 7f1e0358a640 1 -- 172.21.15.106:0/3920696059 wait complete. 2023-11-24T08:12:42.435 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.426+0000 7f1e0358a640 1 Processor -- start 2023-11-24T08:12:42.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.426+0000 7f1e0358a640 1 -- start start 2023-11-24T08:12:42.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e0358a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de8004d00 0x7f1dfc1f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e0358a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc1584f0 0x7f1dfc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e0358a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 0x7f1dfc15b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.437 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e0358a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc15b620 con 0x7f1dfc15ac60 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e0358a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc158d00 con 0x7f1de8004d00 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e0358a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc0aedf0 con 0x7f1dfc1584f0 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 0x7f1dfc15b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 0x7f1dfc15b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36620/0 (socket says 172.21.15.106:36620) 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 -- 172.21.15.106:0/1765721479 learned_addr learned my addr 172.21.15.106:0/1765721479 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01326640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de8004d00 0x7f1dfc1f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.438 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e00b25640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc1584f0 0x7f1dfc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 -- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc1584f0 msgr2=0x7f1dfc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc1584f0 0x7f1dfc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 -- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de8004d00 msgr2=0x7f1dfc1f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de8004d00 0x7f1dfc1f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01b27640 1 -- 172.21.15.106:0/1765721479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc1f4770 con 0x7f1dfc15ac60 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e00b25640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc1584f0 0x7f1dfc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.427+0000 7f1e01326640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de8004d00 0x7f1dfc1f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:42.439 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.428+0000 7f1e01b27640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 0x7f1dfc15b0e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009a90 tx=0x7f1df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.428+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df402cd60 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.428+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1df4003610 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.428+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1df40039c0 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.428+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dfc158a20 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.428+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dfc0af370 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.429+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dfc0aec50 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.430+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1df4003b20 con 0x7f1dfc15ac60 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.431+0000 7f1df1ffb640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1de006b3e0 0x7f1de006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.440 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.431+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7f1df40ab320 con 0x7f1dfc15ac60 2023-11-24T08:12:42.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.431+0000 7f1e01326640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1de006b3e0 0x7f1de006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.432+0000 7f1e01326640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1de006b3e0 0x7f1de006d8a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1df8001dc0 tx=0x7f1df8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:42.441 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.435+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1dfc0aec50 con 0x7f1dfc15ac60 2023-11-24T08:12:42.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.668+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1dfc0aec50 con 0x7f1dfc15ac60 2023-11-24T08:12:42.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.669+0000 7f1df1ffb640 1 -- 172.21.15.106:0/1765721479 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4927 (secure 0 0 0) 0x7f1dfc0aec50 con 0x7f1dfc15ac60 2023-11-24T08:12:42.671 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:42.671 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":44,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":43,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:09.955346+0000","modified":"2023-11-24T08:12:13.029327+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6516},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6516":{"gid":6516,"name":"c","rank":0,"incarnation":39,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/4067546353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":4067546353},{"type":"v1","addr":"172.21.15.106:6837","nonce":4067546353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-11-24T08:12:42.672 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 44 2023-11-24T08:12:42.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.672+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1de006b3e0 msgr2=0x7f1de006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.672+0000 7f1e0358a640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1de006b3e0 0x7f1de006d8a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f1df8001dc0 tx=0x7f1df8009040 comp rx=0 tx=0).stop 2023-11-24T08:12:42.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.672+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 msgr2=0x7f1dfc15b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.672+0000 7f1e0358a640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 0x7f1dfc15b0e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f1df4009a90 tx=0x7f1df4004730 comp rx=0 tx=0).stop 2023-11-24T08:12:42.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.672+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 shutdown_connections 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1dfc15ac60 0x7f1dfc15b0e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1de006b3e0 0x7f1de006d8a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1dfc1584f0 0x7f1dfc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 --2- 172.21.15.106:0/1765721479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1de8004d00 0x7f1dfc1f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 >> 172.21.15.106:0/1765721479 conn(0x7f1dfc06a920 msgr2=0x7f1dfc15a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 shutdown_connections 2023-11-24T08:12:42.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.673+0000 7f1e0358a640 1 -- 172.21.15.106:0/1765721479 wait complete. 2023-11-24T08:12:42.693 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:42.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.905+0000 7fa8ce1d5640 1 Processor -- start 2023-11-24T08:12:42.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.905+0000 7fa8ce1d5640 1 -- start start 2023-11-24T08:12:42.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.906+0000 7fa8ce1d5640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fa8c815c7f0 0x7fa8c806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.906+0000 7fa8ce1d5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8c80ce6c0 0x7fa8c80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.906+0000 7fa8ce1d5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c80cece0 con 0x7fa8c80cf450 2023-11-24T08:12:42.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.906+0000 7fa8ce1d5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c80cfe40 con 0x7fa8c80ce6c0 2023-11-24T08:12:42.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.906+0000 7fa8ce1d5640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c80ceaf0 con 0x7fa8c815c7f0 2023-11-24T08:12:42.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8c7fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8c80ce6c0 0x7fa8c80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8bffff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa8c80cf450 0x7fa8c80cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42908/0 (socket says 172.21.15.106:42908) 2023-11-24T08:12:42.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8bffff640 1 -- 172.21.15.106:0/1460912146 learned_addr learned my addr 172.21.15.106:0/1460912146 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:42.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8c77fe640 1 --2- 172.21.15.106:0/1460912146 >> v2:172.21.15.156:3300/0 conn(0x7fa8c815c7f0 0x7fa8c806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8c7fff640 1 -- 172.21.15.106:0/1460912146 >> v2:172.21.15.156:3300/0 conn(0x7fa8c815c7f0 msgr2=0x7fa8c806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:42.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8c7fff640 1 --2- 172.21.15.106:0/1460912146 >> v2:172.21.15.156:3300/0 conn(0x7fa8c815c7f0 0x7fa8c806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.907+0000 7fa8c7fff640 1 -- 172.21.15.106:0/1460912146 >> v1:172.21.15.106:6789/0 conn(0x7fa8c80cf450 legacy=0x7fa8c80cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.908+0000 7fa8c7fff640 1 -- 172.21.15.106:0/1460912146 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c806d040 con 0x7fa8c80ce6c0 2023-11-24T08:12:42.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.909+0000 7fa8c7fff640 1 --2- 172.21.15.106:0/1460912146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c80cf900 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80d0130 tx=0x7fa8c002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17560d6747bc3002 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:42.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.909+0000 7fa8c6ffd640 1 -- 172.21.15.106:0/1460912146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c002cc70 con 0x7fa8c80ce6c0 2023-11-24T08:12:42.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.909+0000 7fa8c6ffd640 1 -- 172.21.15.106:0/1460912146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c002cdd0 con 0x7fa8c80ce6c0 2023-11-24T08:12:42.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.909+0000 7fa8c6ffd640 1 -- 172.21.15.106:0/1460912146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0034660 con 0x7fa8c80ce6c0 2023-11-24T08:12:42.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.909+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/1460912146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80ce6c0 msgr2=0x7fa8c80cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.909+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/1460912146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c80cf900 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fa8c80d0130 tx=0x7fa8c002bec0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.910+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/1460912146 shutdown_connections 2023-11-24T08:12:42.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.910+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/1460912146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c80cf900 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.910+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/1460912146 >> v2:172.21.15.156:3300/0 conn(0x7fa8c815c7f0 0x7fa8c806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.910+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/1460912146 >> 172.21.15.106:0/1460912146 conn(0x7fa8c806a920 msgr2=0x7fa8c806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:42.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.910+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/1460912146 shutdown_connections 2023-11-24T08:12:42.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.910+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/1460912146 wait complete. 2023-11-24T08:12:42.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.911+0000 7fa8ce1d5640 1 Processor -- start 2023-11-24T08:12:42.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 -- start start 2023-11-24T08:12:42.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c81f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 0x7fa8c81567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8c815c7f0 0x7fa8c80ad140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c81f6210 con 0x7fa8c80cf450 2023-11-24T08:12:42.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8157160 con 0x7fa8c80ce6c0 2023-11-24T08:12:42.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.912+0000 7fa8ce1d5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c8157440 con 0x7fa8c815c7f0 2023-11-24T08:12:42.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 0x7fa8c81567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 0x7fa8c81567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36642/0 (socket says 172.21.15.106:36642) 2023-11-24T08:12:42.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 -- 172.21.15.106:0/130500764 learned_addr learned my addr 172.21.15.106:0/130500764 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:42.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8c77fe640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c81f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 -- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8c815c7f0 msgr2=0x7fa8c80ad140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8c815c7f0 0x7fa8c80ad140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 -- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c80ce6c0 msgr2=0x7fa8c81f5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:42.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c81f5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:42.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 -- 172.21.15.106:0/130500764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0009a70 con 0x7fa8c80cf450 2023-11-24T08:12:42.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8c77fe640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c81f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:42.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.913+0000 7fa8bffff640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 0x7fa8c81567f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c4a0 tx=0x7fa8b000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:42.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.914+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0005ce0 con 0x7fa8c80cf450 2023-11-24T08:12:42.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.914+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b0005e40 con 0x7fa8c80cf450 2023-11-24T08:12:42.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.914+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b0007270 con 0x7fa8c80cf450 2023-11-24T08:12:42.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.914+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c8157720 con 0x7fa8c80cf450 2023-11-24T08:12:42.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.914+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c8157ce0 con 0x7fa8c80cf450 2023-11-24T08:12:42.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.915+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c8157f00 con 0x7fa8c80cf450 2023-11-24T08:12:42.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.916+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fa8b00073d0 con 0x7fa8c80cf450 2023-11-24T08:12:42.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.917+0000 7fa8c4ff9640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8a806b3e0 0x7fa8a806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:42.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.917+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5823+0+0 (secure 0 0 0) 0x7fa8b0089520 con 0x7fa8c80cf450 2023-11-24T08:12:42.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.917+0000 7fa8c77fe640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8a806b3e0 0x7fa8a806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:42.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.918+0000 7fa8c77fe640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8a806b3e0 0x7fa8a806d8a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006fd0 tx=0x7fa8b8008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:42.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:42.921+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8b0053da0 con 0x7fa8c80cf450 2023-11-24T08:12:43.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.113+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c8067470 con 0x7fa8c80cf450 2023-11-24T08:12:43.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.116+0000 7fa8c4ff9640 1 -- 172.21.15.106:0/130500764 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+16824 (secure 0 0 0) 0x7fa8b00573c0 con 0x7fa8c80cf450 2023-11-24T08:12:43.118 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:43.118 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":77,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:35.407263+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:07.722409+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:08.984297+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"77","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":77,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.22","mappings":[{"from":6,"to":0}]},{"pgid":"11.38","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[{"pool":11,"snaps":[{"begin":2,"length":2}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.118+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8a806b3e0 msgr2=0x7fa8a806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.118+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8a806b3e0 0x7fa8a806d8a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8006fd0 tx=0x7fa8b8008040 comp rx=0 tx=0).stop 2023-11-24T08:12:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.118+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 msgr2=0x7fa8c81567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.118+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 0x7fa8c81567f0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa8b000c4a0 tx=0x7fa8b000c960 comp rx=0 tx=0).stop 2023-11-24T08:12:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 shutdown_connections 2023-11-24T08:12:43.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8a806b3e0 0x7fa8a806d8a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8c815c7f0 0x7fa8c80ad140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8c80cf450 0x7fa8c81567f0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 --2- 172.21.15.106:0/130500764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8c80ce6c0 0x7fa8c81f5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 >> 172.21.15.106:0/130500764 conn(0x7fa8c806a920 msgr2=0x7fa8c806c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:43.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 shutdown_connections 2023-11-24T08:12:43.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.119+0000 7fa8ce1d5640 1 -- 172.21.15.106:0/130500764 wait complete. 2023-11-24T08:12:43.139 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:12:43.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.355+0000 7ff2adc83640 1 Processor -- start 2023-11-24T08:12:43.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.355+0000 7ff2adc83640 1 -- start start 2023-11-24T08:12:43.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.356+0000 7ff2adc83640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff2a815d720 0x7ff2a80d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.356+0000 7ff2adc83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a80b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.356+0000 7ff2adc83640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a815db00 con 0x7ff2a80d3870 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.356+0000 7ff2adc83640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80d3d20 con 0x7ff2a815d720 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.356+0000 7ff2adc83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80d0ce0 con 0x7ff2a806ca90 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff2a815d720 0x7ff2a80d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a6ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff2a80d3870 0x7ff2a80b0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42910/0 (socket says 172.21.15.106:42910) 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a6ffd640 1 -- 172.21.15.106:0/2746139101 learned_addr learned my addr 172.21.15.106:0/2746139101 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a77fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff2a815d720 0x7ff2a80d3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36652/0 (socket says 172.21.15.106:36652) 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a77fe640 1 -- 172.21.15.106:0/2746139101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 msgr2=0x7ff2a80b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a77fe640 1 --2- 172.21.15.106:0/2746139101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a80b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a77fe640 1 -- 172.21.15.106:0/2746139101 >> v1:172.21.15.106:6789/0 conn(0x7ff2a80d3870 legacy=0x7ff2a80b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a77fe640 1 -- 172.21.15.106:0/2746139101 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a806d400 con 0x7ff2a815d720 2023-11-24T08:12:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.357+0000 7ff2a67fc640 1 -- 172.21.15.106:0/2746139101 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623262303 0 0) 0x7ff2a815db00 con 0x7ff2a80d3870 2023-11-24T08:12:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.358+0000 7ff2a77fe640 1 --2- 172.21.15.106:0/2746139101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80d3330 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097a0 tx=0x7ff29802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22da6304d3cc93d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2a67fc640 1 -- 172.21.15.106:0/2746139101 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29802e070 con 0x7ff2a815d720 2023-11-24T08:12:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2a67fc640 1 -- 172.21.15.106:0/2746139101 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2980315a0 con 0x7ff2a815d720 2023-11-24T08:12:43.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2a67fc640 1 -- 172.21.15.106:0/2746139101 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff298031930 con 0x7ff2a815d720 2023-11-24T08:12:43.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2adc83640 1 -- 172.21.15.106:0/2746139101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 msgr2=0x7ff2a80d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2746139101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80d3330 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7ff2980097a0 tx=0x7ff29802b840 comp rx=0 tx=0).stop 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2adc83640 1 -- 172.21.15.106:0/2746139101 shutdown_connections 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2746139101 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a80b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.359+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2746139101 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80d3330 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.360+0000 7ff2adc83640 1 -- 172.21.15.106:0/2746139101 >> 172.21.15.106:0/2746139101 conn(0x7ff2a808af60 msgr2=0x7ff2a809c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.360+0000 7ff2adc83640 1 -- 172.21.15.106:0/2746139101 shutdown_connections 2023-11-24T08:12:43.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.360+0000 7ff2adc83640 1 -- 172.21.15.106:0/2746139101 wait complete. 2023-11-24T08:12:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.361+0000 7ff2adc83640 1 Processor -- start 2023-11-24T08:12:43.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.361+0000 7ff2adc83640 1 -- start start 2023-11-24T08:12:43.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.362+0000 7ff2adc83640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.362+0000 7ff2adc83640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80d3870 0x7ff2a80cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.362+0000 7ff2adc83640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.362+0000 7ff2adc83640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cab70 con 0x7ff2a815d720 2023-11-24T08:12:43.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.362+0000 7ff2adc83640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cce50 con 0x7ff2a80d3870 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.362+0000 7ff2adc83640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cc240 con 0x7ff2a806ca90 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36668/0 (socket says 172.21.15.106:36668) 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 -- 172.21.15.106:0/2488163288 learned_addr learned my addr 172.21.15.106:0/2488163288 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a77fe640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a6ffd640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80d3870 0x7ff2a80cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 -- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 msgr2=0x7ff2a814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 -- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80d3870 msgr2=0x7ff2a80cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80d3870 0x7ff2a80cd2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 -- 172.21.15.106:0/2488163288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff298009400 con 0x7ff2a815d720 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a6ffd640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80d3870 0x7ff2a80cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.363+0000 7ff2a7fff640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80cb6a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000b970 tx=0x7ff2a000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.364+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000cb40 con 0x7ff2a815d720 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.364+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000ccc0 con 0x7ff2a815d720 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.364+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0014680 con 0x7ff2a815d720 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.364+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80cc520 con 0x7ff2a815d720 2023-11-24T08:12:43.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.364+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80ccae0 con 0x7ff2a815d720 2023-11-24T08:12:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.365+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80cd000 con 0x7ff2a815d720 2023-11-24T08:12:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.366+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ff2a001e440 con 0x7ff2a815d720 2023-11-24T08:12:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.367+0000 7ff28ffff640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff29406b3e0 0x7ff29406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.367+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5795+0+0 (secure 0 0 0) 0x7ff2a008a0c0 con 0x7ff2a815d720 2023-11-24T08:12:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.367+0000 7ff2a77fe640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff29406b3e0 0x7ff29406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.368+0000 7ff2a77fe640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff29406b3e0 0x7ff29406d8a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff298036000 tx=0x7ff29803d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:43.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.372+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ff2a00bce90 con 0x7ff2a815d720 2023-11-24T08:12:43.608 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.606+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff2a804e8b0 con 0x7ff2a815d720 2023-11-24T08:12:43.609 INFO:tasks.ceph.mon.a.smithi106.stderr:2023-11-24T08:12:43.608+0000 7fb1127dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:12:43.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.611+0000 7ff28ffff640 1 -- 172.21.15.106:0/2488163288 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v45) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff2a0019070 con 0x7ff2a815d720 2023-11-24T08:12:43.613 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:12:43.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff29406b3e0 msgr2=0x7ff29406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff29406b3e0 0x7ff29406d8a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7ff298036000 tx=0x7ff29803d040 comp rx=0 tx=0).stop 2023-11-24T08:12:43.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 msgr2=0x7ff2a80cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.616 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80cb6a0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7ff2a000b970 tx=0x7ff2a000be30 comp rx=0 tx=0).stop 2023-11-24T08:12:43.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 shutdown_connections 2023-11-24T08:12:43.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff2a815d720 0x7ff2a80cb6a0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2a80d3870 0x7ff2a80cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff29406b3e0 0x7ff29406d8a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 --2- 172.21.15.106:0/2488163288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff2a806ca90 0x7ff2a814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 >> 172.21.15.106:0/2488163288 conn(0x7ff2a808af60 msgr2=0x7ff2a80b5780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:43.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.615+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 shutdown_connections 2023-11-24T08:12:43.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.616+0000 7ff2adc83640 1 -- 172.21.15.106:0/2488163288 wait complete. 2023-11-24T08:12:43.636 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:12:43.705 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:12:43.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.836+0000 7fcac7fc9640 1 Processor -- start 2023-11-24T08:12:43.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.836+0000 7fcac7fc9640 1 -- start start 2023-11-24T08:12:43.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.837+0000 7fcac7fc9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcac0146eb0 0x7fcac0161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.837+0000 7fcac7fc9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcac00cd2e0 0x7fcac0070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.837+0000 7fcac7fc9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcac0161a10 con 0x7fcac006e160 2023-11-24T08:12:43.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.837+0000 7fcac7fc9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac0161cf0 con 0x7fcac00cd2e0 2023-11-24T08:12:43.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.837+0000 7fcac7fc9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcac0161fd0 con 0x7fcac0146eb0 2023-11-24T08:12:43.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac5564640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcac006e160 0x7fcac00ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:42934/0 (socket says 172.21.15.106:42934) 2023-11-24T08:12:43.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac5564640 1 -- 172.21.15.106:0/800317014 learned_addr learned my addr 172.21.15.106:0/800317014 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:43.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac6566640 1 --2- 172.21.15.106:0/800317014 >> v2:172.21.15.106:3300/0 conn(0x7fcac00cd2e0 0x7fcac0070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac5d65640 1 --2- 172.21.15.106:0/800317014 >> v2:172.21.15.156:3300/0 conn(0x7fcac0146eb0 0x7fcac0161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac4d63640 1 -- 172.21.15.106:0/800317014 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533033484 0 0) 0x7fcac0161a10 con 0x7fcac006e160 2023-11-24T08:12:43.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac4d63640 1 -- 172.21.15.106:0/800317014 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaac003520 con 0x7fcac006e160 2023-11-24T08:12:43.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac6566640 1 -- 172.21.15.106:0/800317014 >> v2:172.21.15.156:3300/0 conn(0x7fcac0146eb0 msgr2=0x7fcac0161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:43.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac6566640 1 --2- 172.21.15.106:0/800317014 >> v2:172.21.15.156:3300/0 conn(0x7fcac0146eb0 0x7fcac0161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac6566640 1 -- 172.21.15.106:0/800317014 >> v1:172.21.15.106:6789/0 conn(0x7fcac006e160 legacy=0x7fcac00ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.838+0000 7fcac6566640 1 -- 172.21.15.106:0/800317014 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0070660 con 0x7fcac00cd2e0 2023-11-24T08:12:43.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.840+0000 7fcac6566640 1 --2- 172.21.15.106:0/800317014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac0070e40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcac006ea50 tx=0x7fcab802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8302183ff1399fd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.840+0000 7fcac4d63640 1 -- 172.21.15.106:0/800317014 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab802cc80 con 0x7fcac00cd2e0 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.840+0000 7fcac4d63640 1 -- 172.21.15.106:0/800317014 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab802cde0 con 0x7fcac00cd2e0 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.840+0000 7fcac4d63640 1 -- 172.21.15.106:0/800317014 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab8034660 con 0x7fcac00cd2e0 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 -- 172.21.15.106:0/800317014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 msgr2=0x7fcac0070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/800317014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac0070e40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fcac006ea50 tx=0x7fcab802bed0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 -- 172.21.15.106:0/800317014 shutdown_connections 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/800317014 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac0070e40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/800317014 >> v2:172.21.15.156:3300/0 conn(0x7fcac0146eb0 0x7fcac0161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 -- 172.21.15.106:0/800317014 >> 172.21.15.106:0/800317014 conn(0x7fcac00b0c50 msgr2=0x7fcac00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:43.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 -- 172.21.15.106:0/800317014 shutdown_connections 2023-11-24T08:12:43.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.841+0000 7fcac7fc9640 1 -- 172.21.15.106:0/800317014 wait complete. 2023-11-24T08:12:43.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.843+0000 7fcac7fc9640 1 Processor -- start 2023-11-24T08:12:43.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.843+0000 7fcac7fc9640 1 -- start start 2023-11-24T08:12:43.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac7fc9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcac006e160 0x7fcac00999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac7fc9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac7fc9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcac0146eb0 0x7fcac009b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac7fc9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac0099f10 con 0x7fcac00cd2e0 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac7fc9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcac009a1b0 con 0x7fcac006e160 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac7fc9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcac009a490 con 0x7fcac0146eb0 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac5564640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac5564640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac00b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36704/0 (socket says 172.21.15.106:36704) 2023-11-24T08:12:43.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac5564640 1 -- 172.21.15.106:0/2456710631 learned_addr learned my addr 172.21.15.106:0/2456710631 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:43.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac5d65640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcac006e160 0x7fcac00999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac6566640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcac0146eb0 0x7fcac009b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac5564640 1 -- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcac0146eb0 msgr2=0x7fcac009b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.844+0000 7fcac5564640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcac0146eb0 0x7fcac009b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac5564640 1 -- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcac006e160 msgr2=0x7fcac00999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac5564640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcac006e160 0x7fcac00999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac5564640 1 -- 172.21.15.106:0/2456710631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab8009a70 con 0x7fcac00cd2e0 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac5d65640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcac006e160 0x7fcac00999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac6566640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcac0146eb0 0x7fcac009b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac5564640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac00b4440 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcab000c500 tx=0x7fcab000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:43.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0005ce0 con 0x7fcac00cd2e0 2023-11-24T08:12:43.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0005e40 con 0x7fcac00cd2e0 2023-11-24T08:12:43.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0007330 con 0x7fcac00cd2e0 2023-11-24T08:12:43.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac009a770 con 0x7fcac00cd2e0 2023-11-24T08:12:43.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.845+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac009ad30 con 0x7fcac00cd2e0 2023-11-24T08:12:43.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.846+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcac00b05a0 con 0x7fcac00cd2e0 2023-11-24T08:12:43.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.847+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fcab0007550 con 0x7fcac00cd2e0 2023-11-24T08:12:43.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.848+0000 7fcab67fc640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcaa406b3e0 0x7fcaa406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:43.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.848+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fcab0089720 con 0x7fcac00cd2e0 2023-11-24T08:12:43.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.848+0000 7fcac5d65640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcaa406b3e0 0x7fcaa406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:43.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.849+0000 7fcac5d65640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcaa406b3e0 0x7fcaa406d8a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcabc006fd0 tx=0x7fcabc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:43.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:43.853+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcab0054020 con 0x7fcac00cd2e0 2023-11-24T08:12:44.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.088+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcac004e8b0 con 0x7fcac00cd2e0 2023-11-24T08:12:44.617 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.615+0000 7fcab67fc640 1 -- 172.21.15.106:0/2456710631 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v46) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcab0057640 con 0x7fcac00cd2e0 2023-11-24T08:12:44.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcaa406b3e0 msgr2=0x7fcaa406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:44.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcaa406b3e0 0x7fcaa406d8a0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fcabc006fd0 tx=0x7fcabc008040 comp rx=0 tx=0).stop 2023-11-24T08:12:44.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 msgr2=0x7fcac00b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:44.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac00b4440 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcab000c500 tx=0x7fcab000c9c0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 shutdown_connections 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcaa406b3e0 0x7fcaa406d8a0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.619+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcac0146eb0 0x7fcac009b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.620+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcac00cd2e0 0x7fcac00b4440 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.620+0000 7fcac7fc9640 1 --2- 172.21.15.106:0/2456710631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcac006e160 0x7fcac00999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.620+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 >> 172.21.15.106:0/2456710631 conn(0x7fcac00b0c50 msgr2=0x7fcac00cbca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.620+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 shutdown_connections 2023-11-24T08:12:44.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.620+0000 7fcac7fc9640 1 -- 172.21.15.106:0/2456710631 wait complete. 2023-11-24T08:12:44.642 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:12:44.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.852+0000 7f4bda590640 1 Processor -- start 2023-11-24T08:12:44.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.852+0000 7f4bda590640 1 -- start start 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bda590640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4bd40cd0b0 0x7f4bd406e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bda590640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 0x7f4bd415f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bda590640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bd40cd490 con 0x7f4bd40b4d10 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bda590640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bd415c790 con 0x7f4bd40cd0b0 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bda590640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4155e70 con 0x7f4bd40b51c0 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bd37fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4bd40b4d10 0x7f4bd415ef60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35362/0 (socket says 172.21.15.106:35362) 2023-11-24T08:12:44.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.854+0000 7f4bd37fe640 1 -- 172.21.15.106:0/1653230418 learned_addr learned my addr 172.21.15.106:0/1653230418 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd3fff640 1 --2- 172.21.15.106:0/1653230418 >> v2:172.21.15.156:3300/0 conn(0x7f4bd40cd0b0 0x7f4bd406e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697916380 0 0) 0x7f4bc8001490 con 0x7f4bd40b4d10 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd8b2d640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 0x7f4bd415f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4bbc003520 con 0x7f4bd40b4d10 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2904679322 0 0) 0x7f4bbc003520 con 0x7f4bd40b4d10 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 msgr2=0x7f4bd415f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:44.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 0x7f4bd415f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 >> v2:172.21.15.156:3300/0 conn(0x7f4bd40cd0b0 msgr2=0x7f4bd406e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 --2- 172.21.15.106:0/1653230418 >> v2:172.21.15.156:3300/0 conn(0x7f4bd40cd0b0 0x7f4bd406e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd415b920 con 0x7f4bd40b4d10 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f4bc8001b80 con 0x7f4bd40b4d10 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 >> v1:172.21.15.106:6789/0 conn(0x7f4bd40b4d10 legacy=0x7f4bd415ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc003970 0x7f4bbc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bbc0047c0 0x7f4bbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 0x7f4bbc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd415c790 con 0x7f4bbc0047c0 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4155e70 con 0x7f4bbc003970 2023-11-24T08:12:44.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.855+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc00bae0 con 0x7f4bd40b51c0 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bbc0047c0 0x7f4bbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd3fff640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc003970 0x7f4bbc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd37fe640 1 -- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 msgr2=0x7f4bbc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd40b51c0 0x7f4bbc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd37fe640 1 -- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc003970 msgr2=0x7f4bbc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc003970 0x7f4bbc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd37fe640 1 -- 172.21.15.106:0/1653230418 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc003520 con 0x7f4bbc0047c0 2023-11-24T08:12:44.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.856+0000 7f4bd3fff640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc003970 0x7f4bbc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.857+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bbc0047c0 0x7f4bbc004c40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800a810 tx=0x7f4bbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0df4a5fa39dc4d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.857+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8034630 con 0x7f4bbc0047c0 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.857+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc802f070 con 0x7f4bbc0047c0 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.857+0000 7f4bd2ffd640 1 -- 172.21.15.106:0/1653230418 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc80349f0 con 0x7f4bbc0047c0 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bda590640 1 -- 172.21.15.106:0/1653230418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bbc0047c0 msgr2=0x7f4bbc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bda590640 1 --2- 172.21.15.106:0/1653230418 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bbc0047c0 0x7f4bbc004c40 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800a810 tx=0x7f4bbc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bd8b2d640 1 -- 172.21.15.106:0/1653230418 reap_dead start 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bda590640 1 -- 172.21.15.106:0/1653230418 shutdown_connections 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bda590640 1 -- 172.21.15.106:0/1653230418 >> 172.21.15.106:0/1653230418 conn(0x7f4bd4071d40 msgr2=0x7f4bd406a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:44.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bda590640 1 -- 172.21.15.106:0/1653230418 shutdown_connections 2023-11-24T08:12:44.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.858+0000 7f4bda590640 1 -- 172.21.15.106:0/1653230418 wait complete. 2023-11-24T08:12:44.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.860+0000 7f4bda590640 1 Processor -- start 2023-11-24T08:12:44.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.860+0000 7f4bda590640 1 -- start start 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bda590640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc0047c0 0x7f4bd40c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bda590640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 0x7f4bd40c4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bda590640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd4147180 0x7f4bd415d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bda590640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c5c80 con 0x7f4bd4150350 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bda590640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd4158450 con 0x7f4bbc0047c0 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bda590640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bd40c5000 con 0x7f4bd4147180 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bd37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 0x7f4bd40c4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bd37fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 0x7f4bd40c4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58852/0 (socket says 172.21.15.106:58852) 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bd37fe640 1 -- 172.21.15.106:0/690286356 learned_addr learned my addr 172.21.15.106:0/690286356 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.861+0000 7f4bd3fff640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc0047c0 0x7f4bd40c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd37fe640 1 -- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd4147180 msgr2=0x7f4bd415d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd4147180 0x7f4bd415d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd37fe640 1 -- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc0047c0 msgr2=0x7f4bd40c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc0047c0 0x7f4bd40c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd37fe640 1 -- 172.21.15.106:0/690286356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd40c52e0 con 0x7f4bd4150350 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd37fe640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 0x7f4bd40c4970 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800a930 tx=0x7f4bd415c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8004d60 con 0x7f4bd4150350 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bc802ce90 con 0x7f4bd4150350 2023-11-24T08:12:44.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bd40c55c0 con 0x7f4bd4150350 2023-11-24T08:12:44.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bc8005250 con 0x7f4bd4150350 2023-11-24T08:12:44.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.862+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bd4147600 con 0x7f4bd4150350 2023-11-24T08:12:44.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.863+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bd4067470 con 0x7f4bd4150350 2023-11-24T08:12:44.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.864+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4bc8031030 con 0x7f4bd4150350 2023-11-24T08:12:44.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.865+0000 7f4bd0ff9640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4bc006b3e0 0x7f4bc006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:44.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.865+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f4bc80ac0d0 con 0x7f4bd4150350 2023-11-24T08:12:44.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.865+0000 7f4bd3fff640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4bc006b3e0 0x7f4bc006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:44.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.866+0000 7f4bd3fff640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4bc006b3e0 0x7f4bc006d8a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:44.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:44.870+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4bc80767c0 con 0x7f4bd4150350 2023-11-24T08:12:45.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.050+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4bd4067470 con 0x7f4bd4150350 2023-11-24T08:12:45.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.619+0000 7f4bd0ff9640 1 -- 172.21.15.106:0/690286356 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v80) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4bc80053b0 con 0x7f4bd4150350 2023-11-24T08:12:45.621 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:12:45.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.623+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4bc006b3e0 msgr2=0x7f4bc006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:45.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.623+0000 7f4bda590640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4bc006b3e0 0x7f4bc006d8a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4008040 comp rx=0 tx=0).stop 2023-11-24T08:12:45.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.623+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 msgr2=0x7f4bd40c4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:45.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.623+0000 7f4bda590640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 0x7f4bd40c4970 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f4bc800a930 tx=0x7f4bd415c8d0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.624+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 shutdown_connections 2023-11-24T08:12:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.624+0000 7f4bda590640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4bc006b3e0 0x7f4bc006d8a0 secure :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4bc4006fd0 tx=0x7f4bc4008040 comp rx=0 tx=0).stop 2023-11-24T08:12:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.624+0000 7f4bda590640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4bd4147180 0x7f4bd415d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.624+0000 7f4bda590640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4bd4150350 0x7f4bd40c4970 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.624+0000 7f4bda590640 1 --2- 172.21.15.106:0/690286356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4bbc0047c0 0x7f4bd40c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.624+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 >> 172.21.15.106:0/690286356 conn(0x7f4bd4071d40 msgr2=0x7f4bd415e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:45.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.625+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 shutdown_connections 2023-11-24T08:12:45.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.625+0000 7f4bda590640 1 -- 172.21.15.106:0/690286356 wait complete. 2023-11-24T08:12:45.648 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:12:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.852+0000 7f72a7490640 1 Processor -- start 2023-11-24T08:12:45.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.852+0000 7f72a7490640 1 -- start start 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a7490640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f72a015c7f0 0x7f72a006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a7490640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a00cf450 0x7f72a00cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a7490640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f72a00cdfb0 0x7f72a00ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a7490640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a0156010 con 0x7f72a00cdfb0 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a7490640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a006ce30 con 0x7f72a015c7f0 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a7490640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a00cfe10 con 0x7f72a00cf450 2023-11-24T08:12:45.856 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a5a2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f72a00cdfb0 0x7f72a00ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a5a2d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f72a00cdfb0 0x7f72a00ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58860/0 (socket says 172.21.15.106:58860) 2023-11-24T08:12:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.854+0000 7f72a5a2d640 1 -- 172.21.15.106:0/1037403008 learned_addr learned my addr 172.21.15.106:0/1037403008 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.855+0000 7f72a5a2d640 1 -- 172.21.15.106:0/1037403008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a00cf450 msgr2=0x7f72a00cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:45.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.855+0000 7f72a5a2d640 1 --2- 172.21.15.106:0/1037403008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a00cf450 0x7f72a00cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.855+0000 7f72a5a2d640 1 -- 172.21.15.106:0/1037403008 >> v2:172.21.15.156:3300/0 conn(0x7f72a015c7f0 msgr2=0x7f72a006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:45.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.855+0000 7f72a5a2d640 1 --2- 172.21.15.106:0/1037403008 >> v2:172.21.15.156:3300/0 conn(0x7f72a015c7f0 0x7f72a006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.855+0000 7f72a5a2d640 1 -- 172.21.15.106:0/1037403008 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a00cece0 con 0x7f72a00cdfb0 2023-11-24T08:12:45.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.856+0000 7f72a5a2d640 1 --2- 172.21.15.106:0/1037403008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a00ce440 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f72a00d0130 tx=0x7f729802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2befb9bddf589b4a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:45.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.856+0000 7f7297fff640 1 -- 172.21.15.106:0/1037403008 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729802ca80 con 0x7f72a00cdfb0 2023-11-24T08:12:45.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.856+0000 7f7297fff640 1 -- 172.21.15.106:0/1037403008 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729802cbe0 con 0x7f72a00cdfb0 2023-11-24T08:12:45.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.856+0000 7f7297fff640 1 -- 172.21.15.106:0/1037403008 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980343f0 con 0x7f72a00cdfb0 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 -- 172.21.15.106:0/1037403008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 msgr2=0x7f72a00ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 --2- 172.21.15.106:0/1037403008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a00ce440 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f72a00d0130 tx=0x7f729802bee0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 -- 172.21.15.106:0/1037403008 shutdown_connections 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 --2- 172.21.15.106:0/1037403008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a00ce440 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 --2- 172.21.15.106:0/1037403008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a00cf450 0x7f72a00cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 --2- 172.21.15.106:0/1037403008 >> v2:172.21.15.156:3300/0 conn(0x7f72a015c7f0 0x7f72a006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 -- 172.21.15.106:0/1037403008 >> 172.21.15.106:0/1037403008 conn(0x7f72a006a920 msgr2=0x7f72a006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 -- 172.21.15.106:0/1037403008 shutdown_connections 2023-11-24T08:12:45.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.857+0000 7f72a7490640 1 -- 172.21.15.106:0/1037403008 wait complete. 2023-11-24T08:12:45.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.858+0000 7f72a7490640 1 Processor -- start 2023-11-24T08:12:45.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 -- start start 2023-11-24T08:12:45.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a014b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a00cf450 0x7f72a014a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a015c7f0 0x7f72a0153850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a014b760 con 0x7f72a00cdfb0 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a014ba40 con 0x7f72a00cf450 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.859+0000 7f72a7490640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a014bd20 con 0x7f72a015c7f0 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a014b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a014b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58866/0 (socket says 172.21.15.106:58866) 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 -- 172.21.15.106:0/2410480071 learned_addr learned my addr 172.21.15.106:0/2410480071 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 -- 172.21.15.106:0/2410480071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a015c7f0 msgr2=0x7f72a0153850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:45.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a015c7f0 0x7f72a0153850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 -- 172.21.15.106:0/2410480071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a00cf450 msgr2=0x7f72a014a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a00cf450 0x7f72a014a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 -- 172.21.15.106:0/2410480071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7298009b30 con 0x7f72a00cdfb0 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.860+0000 7f72a522c640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a014b180 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b4a0 tx=0x7f729c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.861+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c010610 con 0x7f72a00cdfb0 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.861+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c010770 con 0x7f72a00cdfb0 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.861+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a014c000 con 0x7f72a00cdfb0 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.861+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c010b20 con 0x7f72a00cdfb0 2023-11-24T08:12:45.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.861+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a014efa0 con 0x7f72a00cdfb0 2023-11-24T08:12:45.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.862+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a014f220 con 0x7f72a00cdfb0 2023-11-24T08:12:45.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.863+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f729c010c80 con 0x7f72a00cdfb0 2023-11-24T08:12:45.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.864+0000 7f7295ffb640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f728406b3e0 0x7f728406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:45.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.864+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f729c089ae0 con 0x7f72a00cdfb0 2023-11-24T08:12:45.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.864+0000 7f72a4a2b640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f728406b3e0 0x7f728406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:45.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.865+0000 7f72a4a2b640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f728406b3e0 0x7f728406d8a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f72a0153380 tx=0x7f7290006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:45.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:45.868+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f729c0544d0 con 0x7f72a00cdfb0 2023-11-24T08:12:46.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.044+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72a0067470 con 0x7f72a00cdfb0 2023-11-24T08:12:46.326 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:12:46.335 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:12:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.635+0000 7f7295ffb640 1 -- 172.21.15.106:0/2410480071 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v81) v1 ==== 169+0+0 (secure 0 0 0) 0x7f729c01e0d0 con 0x7f72a00cdfb0 2023-11-24T08:12:46.637 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2023-11-24T08:12:46.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.640+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f728406b3e0 msgr2=0x7f728406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:46.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.640+0000 7f72a7490640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f728406b3e0 0x7f728406d8a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f72a0153380 tx=0x7f7290006c90 comp rx=0 tx=0).stop 2023-11-24T08:12:46.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.640+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 msgr2=0x7f72a014b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:46.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.640+0000 7f72a7490640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a014b180 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f729c00b4a0 tx=0x7f729c00b960 comp rx=0 tx=0).stop 2023-11-24T08:12:46.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 shutdown_connections 2023-11-24T08:12:46.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f728406b3e0 0x7f728406d8a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f72a015c7f0 0x7f72a0153850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72a00cf450 0x7f72a014a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 --2- 172.21.15.106:0/2410480071 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f72a00cdfb0 0x7f72a014b180 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 >> 172.21.15.106:0/2410480071 conn(0x7f72a006a920 msgr2=0x7f72a0070770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:46.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 shutdown_connections 2023-11-24T08:12:46.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.641+0000 7f72a7490640 1 -- 172.21.15.106:0/2410480071 wait complete. 2023-11-24T08:12:46.662 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:12:46.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.868+0000 7f8c0ec7c640 1 Processor -- start 2023-11-24T08:12:46.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.868+0000 7f8c0ec7c640 1 -- start start 2023-11-24T08:12:46.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.869+0000 7f8c0ec7c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8c081605f0 0x7f8c08160a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:46.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.869+0000 7f8c0ec7c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c080ce410 con 0x7f8c080d2000 2023-11-24T08:12:46.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.869+0000 7f8c0ec7c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c08160f80 con 0x7f8c080d1c20 2023-11-24T08:12:46.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0ec7c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c0806c270 con 0x7f8c081605f0 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0d219640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8c081605f0 0x7f8c08160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0d219640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8c081605f0 0x7f8c08160a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58880/0 (socket says 172.21.15.106:58880) 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0d219640 1 -- 172.21.15.106:0/1452248864 learned_addr learned my addr 172.21.15.106:0/1452248864 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c07fff640 1 --1- 172.21.15.106:0/1452248864 >> v1:172.21.15.106:6789/0 conn(0x7f8c080d2000 0x7f8c080cfcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35366/0 (socket says 172.21.15.106:35366) 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0d219640 1 -- 172.21.15.106:0/1452248864 >> v1:172.21.15.156:6789/0 conn(0x7f8c080d1c20 legacy=0x7f8c0806bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0d219640 1 -- 172.21.15.106:0/1452248864 >> v1:172.21.15.106:6789/0 conn(0x7f8c080d2000 legacy=0x7f8c080cfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:46.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.870+0000 7f8c0d219640 1 -- 172.21.15.106:0/1452248864 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c080d06f0 con 0x7f8c081605f0 2023-11-24T08:12:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.871+0000 7f8c0d219640 1 --2- 172.21.15.106:0/1452248864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c081605f0 0x7f8c08160a40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009870 tx=0x7f8c0002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b8b5250e165ee61 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.872+0000 7f8c077fe640 1 -- 172.21.15.106:0/1452248864 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00030850 con 0x7f8c081605f0 2023-11-24T08:12:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.872+0000 7f8c077fe640 1 -- 172.21.15.106:0/1452248864 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c000309b0 con 0x7f8c081605f0 2023-11-24T08:12:46.873 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.872+0000 7f8c077fe640 1 -- 172.21.15.106:0/1452248864 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00030d40 con 0x7f8c081605f0 2023-11-24T08:12:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.872+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/1452248864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c081605f0 msgr2=0x7f8c08160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.872+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/1452248864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c081605f0 0x7f8c08160a40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009870 tx=0x7f8c0002b990 comp rx=0 tx=0).stop 2023-11-24T08:12:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.873+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/1452248864 shutdown_connections 2023-11-24T08:12:46.874 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.873+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/1452248864 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c081605f0 0x7f8c08160a40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.873+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/1452248864 >> 172.21.15.106:0/1452248864 conn(0x7f8c0808cea0 msgr2=0x7f8c0809c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:46.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.873+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/1452248864 shutdown_connections 2023-11-24T08:12:46.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.873+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/1452248864 wait complete. 2023-11-24T08:12:46.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.875+0000 7f8c0ec7c640 1 Processor -- start 2023-11-24T08:12:46.876 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.875+0000 7f8c0ec7c640 1 -- start start 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.876+0000 7f8c0ec7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 0x7f8c0814fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.876+0000 7f8c0ec7c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c080d2000 0x7f8c08152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.876+0000 7f8c0ec7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c081605f0 0x7f8c080b81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.876+0000 7f8c0ec7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0815f740 con 0x7f8c080d1c20 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.876+0000 7f8c0ec7c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c08153310 con 0x7f8c080d2000 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.876+0000 7f8c0ec7c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0809e410 con 0x7f8c081605f0 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 0x7f8c0814fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 0x7f8c0814fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58890/0 (socket says 172.21.15.106:58890) 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 -- 172.21.15.106:0/3971770432 learned_addr learned my addr 172.21.15.106:0/3971770432 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c07fff640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c080d2000 0x7f8c08152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:46.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 -- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c081605f0 msgr2=0x7f8c080b81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c081605f0 0x7f8c080b81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 -- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c080d2000 msgr2=0x7f8c08152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c080d2000 0x7f8c08152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.877+0000 7f8c0ca18640 1 -- 172.21.15.106:0/3971770432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000094d0 con 0x7f8c080d1c20 2023-11-24T08:12:46.880 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.878+0000 7f8c0ca18640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 0x7f8c0814fc50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800ba50 tx=0x7f8bf800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.878+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf800cbe0 con 0x7f8c080d1c20 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.878+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf800cd80 con 0x7f8c080d1c20 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.878+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0809e6f0 con 0x7f8c080d1c20 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.878+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8014680 con 0x7f8c080d1c20 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.878+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c080b0020 con 0x7f8c080d1c20 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.879+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c08150330 con 0x7f8c080d1c20 2023-11-24T08:12:46.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.880+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8bf80147e0 con 0x7f8c080d1c20 2023-11-24T08:12:46.882 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.881+0000 7f8c057fa640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8be806b3e0 0x7f8be806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.881+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8bf8089c80 con 0x7f8c080d1c20 2023-11-24T08:12:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.881+0000 7f8c07fff640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8be806b3e0 0x7f8be806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:46.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.882+0000 7f8c07fff640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8be806b3e0 0x7f8be806d8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0007fd0 tx=0x7f8bf0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:46.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:46.886+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c08150330 con 0x7f8c080d1c20 2023-11-24T08:12:47.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.080+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8c08150330 con 0x7f8c080d1c20 2023-11-24T08:12:47.089 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.087+0000 7f8c057fa640 1 -- 172.21.15.106:0/3971770432 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8c08150330 con 0x7f8c080d1c20 2023-11-24T08:12:47.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8be806b3e0 msgr2=0x7f8be806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8be806b3e0 0x7f8be806d8a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0007fd0 tx=0x7f8bf0009040 comp rx=0 tx=0).stop 2023-11-24T08:12:47.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 msgr2=0x7f8c0814fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 0x7f8c0814fc50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f8bf800ba50 tx=0x7f8bf800bf10 comp rx=0 tx=0).stop 2023-11-24T08:12:47.093 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 shutdown_connections 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8be806b3e0 0x7f8be806d8a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c081605f0 0x7f8c080b81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c080d2000 0x7f8c08152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 --2- 172.21.15.106:0/3971770432 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c080d1c20 0x7f8c0814fc50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.091+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 >> 172.21.15.106:0/3971770432 conn(0x7f8c0808cea0 msgr2=0x7f8c080c6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.092+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 shutdown_connections 2023-11-24T08:12:47.094 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.092+0000 7f8c0ec7c640 1 -- 172.21.15.106:0/3971770432 wait complete. 2023-11-24T08:12:47.113 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info' 2023-11-24T08:12:47.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.320+0000 7fd659dfb640 1 Processor -- start 2023-11-24T08:12:47.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.320+0000 7fd659dfb640 1 -- start start 2023-11-24T08:12:47.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.322+0000 7fd659dfb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6540d31a0 0x7fd6540d3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.323 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.322+0000 7fd659dfb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.322+0000 7fd659dfb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6540d1070 con 0x7fd6541609e0 2023-11-24T08:12:47.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.322+0000 7fd659dfb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6540d3b50 con 0x7fd6540d31a0 2023-11-24T08:12:47.324 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.322+0000 7fd659dfb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540cedc0 con 0x7fd6540ce010 2023-11-24T08:12:47.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd653fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd652ffd640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6540d31a0 0x7fd6540d3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd652ffd640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd6540d31a0 0x7fd6540d3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:37484/0 (socket says 172.21.15.106:37484) 2023-11-24T08:12:47.325 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd652ffd640 1 -- 172.21.15.106:0/1072014737 learned_addr learned my addr 172.21.15.106:0/1072014737 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:47.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd653fff640 1 -- 172.21.15.106:0/1072014737 >> v2:172.21.15.156:3300/0 conn(0x7fd6540d31a0 msgr2=0x7fd6540d3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:47.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd653fff640 1 --2- 172.21.15.106:0/1072014737 >> v2:172.21.15.156:3300/0 conn(0x7fd6540d31a0 0x7fd6540d3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd653fff640 1 -- 172.21.15.106:0/1072014737 >> v1:172.21.15.156:6789/0 conn(0x7fd6541609e0 legacy=0x7fd65406bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:47.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.323+0000 7fd653fff640 1 -- 172.21.15.106:0/1072014737 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6540cf0a0 con 0x7fd6540ce010 2023-11-24T08:12:47.326 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.325+0000 7fd653fff640 1 --2- 172.21.15.106:0/1072014737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540ce480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6540d43c0 tx=0x7fd64c02bde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1725f2d81c3d13d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:47.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.325+0000 7fd6527fc640 1 -- 172.21.15.106:0/1072014737 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c02cb90 con 0x7fd6540ce010 2023-11-24T08:12:47.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.325+0000 7fd6527fc640 1 -- 172.21.15.106:0/1072014737 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c02ccf0 con 0x7fd6540ce010 2023-11-24T08:12:47.327 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.325+0000 7fd6527fc640 1 -- 172.21.15.106:0/1072014737 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c034660 con 0x7fd6540ce010 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 -- 172.21.15.106:0/1072014737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 msgr2=0x7fd6540ce480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 --2- 172.21.15.106:0/1072014737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540ce480 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd6540d43c0 tx=0x7fd64c02bde0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 -- 172.21.15.106:0/1072014737 shutdown_connections 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 --2- 172.21.15.106:0/1072014737 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540ce480 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 --2- 172.21.15.106:0/1072014737 >> v2:172.21.15.156:3300/0 conn(0x7fd6540d31a0 0x7fd6540d3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 -- 172.21.15.106:0/1072014737 >> 172.21.15.106:0/1072014737 conn(0x7fd65408af60 msgr2=0x7fd65409c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 -- 172.21.15.106:0/1072014737 shutdown_connections 2023-11-24T08:12:47.328 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.326+0000 7fd659dfb640 1 -- 172.21.15.106:0/1072014737 wait complete. 2023-11-24T08:12:47.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.328+0000 7fd659dfb640 1 Processor -- start 2023-11-24T08:12:47.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.328+0000 7fd659dfb640 1 -- start start 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.329+0000 7fd659dfb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.329+0000 7fd659dfb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6540d31a0 0x7fd654152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd659dfb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 0x7fd6540b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd659dfb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540b4610 con 0x7fd6541609e0 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd659dfb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540b4330 con 0x7fd6540d31a0 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd659dfb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6540b6b80 con 0x7fd6540ce010 2023-11-24T08:12:47.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 0x7fd6540b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 0x7fd6540b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58906/0 (socket says 172.21.15.106:58906) 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 -- 172.21.15.106:0/3887142651 learned_addr learned my addr 172.21.15.106:0/3887142651 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd6537fe640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd652ffd640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6540d31a0 0x7fd654152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 -- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 msgr2=0x7fd6540b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 -- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6540d31a0 msgr2=0x7fd654152760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6540d31a0 0x7fd654152760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.330+0000 7fd653fff640 1 -- 172.21.15.106:0/3887142651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c009a70 con 0x7fd6541609e0 2023-11-24T08:12:47.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.331+0000 7fd653fff640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 0x7fd6540b4d20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd64c037040 tx=0x7fd64c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.331+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c040650 con 0x7fd6541609e0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.331+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64c0407b0 con 0x7fd6541609e0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.331+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64c040b60 con 0x7fd6541609e0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.331+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6540b6e60 con 0x7fd6541609e0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.331+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd654152ca0 con 0x7fd6541609e0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.332+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6540b69e0 con 0x7fd6541609e0 2023-11-24T08:12:47.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.333+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd64c040cc0 con 0x7fd6541609e0 2023-11-24T08:12:47.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.335+0000 7fd64bfff640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd63406b3e0 0x7fd63406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.335+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd64c0ac970 con 0x7fd6541609e0 2023-11-24T08:12:47.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.335+0000 7fd6537fe640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd63406b3e0 0x7fd63406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.336+0000 7fd6537fe640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd63406b3e0 0x7fd63406d8a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd644006fd0 tx=0x7fd644008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:47.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.339+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd6540b69e0 con 0x7fd6541609e0 2023-11-24T08:12:47.610 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.609+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]} v 0) v1 -- 0x7fd6540b7380 con 0x7fd6541609e0 2023-11-24T08:12:47.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.637+0000 7fd64bfff640 1 -- 172.21.15.106:0/3887142651 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_info"]}]=0 v239) v1 ==== 166+0+0 (secure 0 0 0) 0x7fd6540b7380 con 0x7fd6541609e0 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd63406b3e0 msgr2=0x7fd63406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd63406b3e0 0x7fd63406d8a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fd644006fd0 tx=0x7fd644008040 comp rx=0 tx=0).stop 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 msgr2=0x7fd6540b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 0x7fd6540b4d20 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd64c037040 tx=0x7fd64c004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 shutdown_connections 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd6541609e0 0x7fd6540b4d20 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6540d31a0 0x7fd654152760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd63406b3e0 0x7fd63406d8a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 --2- 172.21.15.106:0/3887142651 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd6540ce010 0x7fd6540b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 >> 172.21.15.106:0/3887142651 conn(0x7fd65408af60 msgr2=0x7fd6540d1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:47.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 shutdown_connections 2023-11-24T08:12:47.644 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.641+0000 7fd659dfb640 1 -- 172.21.15.106:0/3887142651 wait complete. 2023-11-24T08:12:47.662 INFO:tasks.cephfs_test_runner:tests the scenario where 'fs subvolume snapshot info' command ... ok 2023-11-24T08:12:47.662 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:12:47.663 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:47.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.857+0000 7f9770ca8640 1 Processor -- start 2023-11-24T08:12:47.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.857+0000 7f9770ca8640 1 -- start start 2023-11-24T08:12:47.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.858+0000 7f9770ca8640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f976c144b30 0x7f976c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f9770ca8640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 0x7f976c0b0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f9770ca8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f976c0b1de0 0x7f976c0b2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f9770ca8640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c0b27b0 con 0x7f976c0b1de0 2023-11-24T08:12:47.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f9770ca8640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c15bf80 con 0x7f976c144b30 2023-11-24T08:12:47.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f9770ca8640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0b1560 con 0x7f976c15c410 2023-11-24T08:12:47.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f976bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f976c0b1de0 0x7f976c0b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f976bfff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f976c0b1de0 0x7f976c0b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58922/0 (socket says 172.21.15.106:58922) 2023-11-24T08:12:47.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f976bfff640 1 -- 172.21.15.106:0/2955062879 learned_addr learned my addr 172.21.15.106:0/2955062879 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:47.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f976bfff640 1 -- 172.21.15.106:0/2955062879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 msgr2=0x7f976c0b0a70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T08:12:47.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.859+0000 7f976bfff640 1 --2- 172.21.15.106:0/2955062879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 0x7f976c0b0a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.860+0000 7f976bfff640 1 -- 172.21.15.106:0/2955062879 >> v2:172.21.15.156:3300/0 conn(0x7f976c144b30 msgr2=0x7f976c15f920 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-24T08:12:47.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.860+0000 7f976bfff640 1 --2- 172.21.15.106:0/2955062879 >> v2:172.21.15.156:3300/0 conn(0x7f976c144b30 0x7f976c15f920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.860+0000 7f976bfff640 1 -- 172.21.15.106:0/2955062879 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0b0fb0 con 0x7f976c0b1de0 2023-11-24T08:12:47.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.861+0000 7f976bfff640 1 --2- 172.21.15.106:0/2955062879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c0b1de0 0x7f976c0b2270 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f976c06f690 tx=0x7f976402bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad07f6c317ba47 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:47.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.861+0000 7f976affd640 1 -- 172.21.15.106:0/2955062879 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976402cac0 con 0x7f976c0b1de0 2023-11-24T08:12:47.863 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.861+0000 7f976affd640 1 -- 172.21.15.106:0/2955062879 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976402cc20 con 0x7f976c0b1de0 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.861+0000 7f976affd640 1 -- 172.21.15.106:0/2955062879 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97640343f0 con 0x7f976c0b1de0 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 -- 172.21.15.106:0/2955062879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c0b1de0 msgr2=0x7f976c0b2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2955062879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c0b1de0 0x7f976c0b2270 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f976c06f690 tx=0x7f976402bf20 comp rx=0 tx=0).stop 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 -- 172.21.15.106:0/2955062879 shutdown_connections 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2955062879 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c0b1de0 0x7f976c0b2270 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2955062879 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 0x7f976c0b0a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2955062879 >> v2:172.21.15.156:3300/0 conn(0x7f976c144b30 0x7f976c15f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 -- 172.21.15.106:0/2955062879 >> 172.21.15.106:0/2955062879 conn(0x7f976c0cb830 msgr2=0x7f976c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 -- 172.21.15.106:0/2955062879 shutdown_connections 2023-11-24T08:12:47.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.862+0000 7f9770ca8640 1 -- 172.21.15.106:0/2955062879 wait complete. 2023-11-24T08:12:47.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.864+0000 7f9770ca8640 1 Processor -- start 2023-11-24T08:12:47.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.864+0000 7f9770ca8640 1 -- start start 2023-11-24T08:12:47.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9770ca8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f976c0b1de0 0x7f976c0b3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9770ca8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 0x7f976c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9770ca8640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 0x7f976c1f37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9770ca8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c1310e0 con 0x7f976c144b30 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9770ca8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c1304a0 con 0x7f976c0b1de0 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9770ca8640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c130740 con 0x7f976c15c410 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 0x7f976c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 0x7f976c0b4170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58930/0 (socket says 172.21.15.106:58930) 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 -- 172.21.15.106:0/2753432332 learned_addr learned my addr 172.21.15.106:0/2753432332 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 -- 172.21.15.106:0/2753432332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 msgr2=0x7f976c1f37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 0x7f976c1f37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 -- 172.21.15.106:0/2753432332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f976c0b1de0 msgr2=0x7f976c0b3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f976c0b1de0 0x7f976c0b3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:47.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.865+0000 7f9763fff640 1 -- 172.21.15.106:0/2753432332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764009b30 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.866+0000 7f9763fff640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 0x7f976c0b4170 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f975400b3e0 tx=0x7f975400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.866+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011830 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.866+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97540119b0 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.866+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c130a20 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.866+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754011d60 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.866+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c1f6730 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.867+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c04e840 con 0x7f976c144b30 2023-11-24T08:12:47.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.868+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f975401a400 con 0x7f976c144b30 2023-11-24T08:12:47.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.869+0000 7f9768ff9640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f974c06b3e0 0x7f974c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:47.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.869+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9754089dd0 con 0x7f976c144b30 2023-11-24T08:12:47.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.869+0000 7f976b7fe640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f974c06b3e0 0x7f974c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:47.871 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.870+0000 7f976b7fe640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f974c06b3e0 0x7f974c06d8a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f976c0721f0 tx=0x7f975c006c40 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:47.875 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:47.874+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f97540549a0 con 0x7f976c144b30 2023-11-24T08:12:48.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.066+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f976c13ea20 con 0x7f976c144b30 2023-11-24T08:12:48.069 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.067+0000 7f9768ff9640 1 -- 172.21.15.106:0/2753432332 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+13037 (secure 0 0 0) 0x7f9754015020 con 0x7f976c144b30 2023-11-24T08:12:48.069 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:48.069 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":81,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:46.631838+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/4067546353":"2023-11-25T08:12:43.608458+0000","172.21.15.106:6835/4067546353":"2023-11-25T08:12:43.608458+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f974c06b3e0 msgr2=0x7f974c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f974c06b3e0 0x7f974c06d8a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f976c0721f0 tx=0x7f975c006c40 comp rx=0 tx=0).stop 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 msgr2=0x7f976c0b4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 0x7f976c0b4170 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f975400b3e0 tx=0x7f975400b8a0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 shutdown_connections 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f974c06b3e0 0x7f974c06d8a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f976c15c410 0x7f976c1f37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f976c144b30 0x7f976c0b4170 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 --2- 172.21.15.106:0/2753432332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f976c0b1de0 0x7f976c0b3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 >> 172.21.15.106:0/2753432332 conn(0x7f976c0cb830 msgr2=0x7f976c15e390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:48.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.070+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 shutdown_connections 2023-11-24T08:12:48.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.071+0000 7f9770ca8640 1 -- 172.21.15.106:0/2753432332 wait complete. 2023-11-24T08:12:48.089 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:48.089 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:48.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.301+0000 7fbd786e0640 1 Processor -- start 2023-11-24T08:12:48.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.301+0000 7fbd786e0640 1 -- start start 2023-11-24T08:12:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.303+0000 7fbd786e0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd700c59b0 0x7fbd70166230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.303+0000 7fbd786e0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7015eea0 con 0x7fbd7015b440 2023-11-24T08:12:48.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.303+0000 7fbd786e0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd7006d600 con 0x7fbd700cc0f0 2023-11-24T08:12:48.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.303+0000 7fbd786e0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700c5e00 con 0x7fbd700c59b0 2023-11-24T08:12:48.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.303+0000 7fbd7647c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fbd7015b440 0x7fbd700cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35374/0 (socket says 172.21.15.106:35374) 2023-11-24T08:12:48.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.303+0000 7fbd7647c640 1 -- 172.21.15.106:0/993934383 learned_addr learned my addr 172.21.15.106:0/993934383 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd76c7d640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd700c59b0 0x7fbd70166230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890951334 0 0) 0x7fbd7015eea0 con 0x7fbd7015b440 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5c003520 con 0x7fbd7015b440 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3757946751 0 0) 0x7fbd5c003520 con 0x7fbd7015b440 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd700c59b0 msgr2=0x7fbd70166230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd700c59b0 0x7fbd70166230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 >> v1:172.21.15.156:6789/0 conn(0x7fbd700cc0f0 legacy=0x7fbd700c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700abf30 con 0x7fbd7015b440 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fbd6c001670 con 0x7fbd7015b440 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 >> v1:172.21.15.106:6789/0 conn(0x7fbd7015b440 legacy=0x7fbd700cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd5c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd5c0047c0 0x7fbd5c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c0051d0 0x7fbd5c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00c0a0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00c210 con 0x7fbd5c0051d0 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c00c3d0 con 0x7fbd5c0047c0 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.304+0000 7fbd76c7d640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd700c59b0 0x7fbd70166230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd7647c640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd5c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd7647c640 1 -- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd5c0047c0 msgr2=0x7fbd5c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd75c7b640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd5c0047c0 0x7fbd5c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd76c7d640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c0051d0 0x7fbd5c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd7647c640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd5c0047c0 0x7fbd5c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd7647c640 1 -- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c0051d0 msgr2=0x7fbd5c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd7647c640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c0051d0 0x7fbd5c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd7647c640 1 -- 172.21.15.106:0/993934383 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c003520 con 0x7fbd5c0039a0 2023-11-24T08:12:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd75c7b640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd5c0047c0 0x7fbd5c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:48.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.305+0000 7fbd76c7d640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd5c0051d0 0x7fbd5c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.306+0000 7fbd7647c640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd5c003dc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c240 tx=0x7fbd5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92bd8f31eb145c84 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c005ad0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c005c30 con 0x7fbd5c0039a0 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd7547a640 1 -- 172.21.15.106:0/993934383 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c02c8b0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd786e0640 1 -- 172.21.15.106:0/993934383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 msgr2=0x7fbd5c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd786e0640 1 --2- 172.21.15.106:0/993934383 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd5c003dc0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00c240 tx=0x7fbd5c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd76c7d640 1 -- 172.21.15.106:0/993934383 reap_dead start 2023-11-24T08:12:48.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd786e0640 1 -- 172.21.15.106:0/993934383 shutdown_connections 2023-11-24T08:12:48.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd786e0640 1 -- 172.21.15.106:0/993934383 >> 172.21.15.106:0/993934383 conn(0x7fbd700b0c50 msgr2=0x7fbd700b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:48.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd786e0640 1 -- 172.21.15.106:0/993934383 shutdown_connections 2023-11-24T08:12:48.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.307+0000 7fbd786e0640 1 -- 172.21.15.106:0/993934383 wait complete. 2023-11-24T08:12:48.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.309+0000 7fbd786e0640 1 Processor -- start 2023-11-24T08:12:48.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.309+0000 7fbd786e0640 1 -- start start 2023-11-24T08:12:48.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd786e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd70156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd786e0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700c6f30 0x7fbd700c7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd786e0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd701560e0 0x7fbd700ad5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd786e0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700c8fa0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd786e0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700c84c0 con 0x7fbd700c6f30 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd786e0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd700c6170 con 0x7fbd701560e0 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd7647c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd70156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd7647c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd70156870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58954/0 (socket says 172.21.15.106:58954) 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.310+0000 7fbd7647c640 1 -- 172.21.15.106:0/2997410463 learned_addr learned my addr 172.21.15.106:0/2997410463 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd7647c640 1 -- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd701560e0 msgr2=0x7fbd700ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd7647c640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd701560e0 0x7fbd700ad5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd75c7b640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700c6f30 0x7fbd700c7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd7647c640 1 -- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700c6f30 msgr2=0x7fbd700c7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd7647c640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700c6f30 0x7fbd700c7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd7647c640 1 -- 172.21.15.106:0/2997410463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700c6450 con 0x7fbd5c0039a0 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd75c7b640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700c6f30 0x7fbd700c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:48.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.311+0000 7fbd7647c640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd70156870 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c02c450 tx=0x7fbd6c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.312+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c039030 con 0x7fbd5c0039a0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.312+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c02ccc0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.312+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c03c040 con 0x7fbd5c0039a0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.312+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd700c6730 con 0x7fbd5c0039a0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.312+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd70156590 con 0x7fbd5c0039a0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.313+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7004e8b0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.314+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fbd6c032070 con 0x7fbd5c0039a0 2023-11-24T08:12:48.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.315+0000 7fbd66ffd640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd5406b3e0 0x7fbd5406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.315+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbd6c0ab1e0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.315+0000 7fbd75c7b640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd5406b3e0 0x7fbd5406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.316+0000 7fbd75c7b640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd5406b3e0 0x7fbd5406d8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd701f67f0 tx=0x7fbd60009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:48.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.319+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fbd6c075ce0 con 0x7fbd5c0039a0 2023-11-24T08:12:48.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.509+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbd70067680 con 0x7fbd5c0039a0 2023-11-24T08:12:48.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.510+0000 7fbd66ffd640 1 -- 172.21.15.106:0/2997410463 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd6c079300 con 0x7fbd5c0039a0 2023-11-24T08:12:48.512 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:48.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd5406b3e0 msgr2=0x7fbd5406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd5406b3e0 0x7fbd5406d8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbd701f67f0 tx=0x7fbd60009040 comp rx=0 tx=0).stop 2023-11-24T08:12:48.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 msgr2=0x7fbd70156870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd70156870 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c02c450 tx=0x7fbd6c001a00 comp rx=0 tx=0).stop 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 shutdown_connections 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fbd5406b3e0 0x7fbd5406d8a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fbd701560e0 0x7fbd700ad5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbd700c6f30 0x7fbd700c7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 --2- 172.21.15.106:0/2997410463 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fbd5c0039a0 0x7fbd70156870 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 >> 172.21.15.106:0/2997410463 conn(0x7fbd700b0c50 msgr2=0x7fbd700b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:48.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 shutdown_connections 2023-11-24T08:12:48.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.513+0000 7fbd786e0640 1 -- 172.21.15.106:0/2997410463 wait complete. 2023-11-24T08:12:48.534 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T08:12:48.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.740+0000 7f01cef43640 1 Processor -- start 2023-11-24T08:12:48.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.741+0000 7f01cef43640 1 -- start start 2023-11-24T08:12:48.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cef43640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 0x7f01c806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cef43640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f01c80cf450 0x7f01c80cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cef43640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c80cece0 con 0x7f01c80cdfb0 2023-11-24T08:12:48.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cef43640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c8156010 con 0x7f01c80cf450 2023-11-24T08:12:48.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cef43640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c806ce30 con 0x7f01c815c7f0 2023-11-24T08:12:48.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cd4e0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f01c80cdfb0 0x7f01c80ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35378/0 (socket says 172.21.15.106:35378) 2023-11-24T08:12:48.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.742+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2260273131 learned_addr learned my addr 172.21.15.106:0/2260273131 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:48.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c7fff640 1 --2- 172.21.15.106:0/2260273131 >> v2:172.21.15.156:3300/0 conn(0x7f01c80cf450 0x7f01c80cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984751458 0 0) 0x7f01c80cece0 con 0x7f01c80cdfb0 2023-11-24T08:12:48.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01cccdf640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 0x7f01c806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b8003520 con 0x7f01c80cdfb0 2023-11-24T08:12:48.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749811529 0 0) 0x7f01b8003520 con 0x7f01c80cdfb0 2023-11-24T08:12:48.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 msgr2=0x7f01c806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 0x7f01c806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 >> v2:172.21.15.156:3300/0 conn(0x7f01c80cf450 msgr2=0x7f01c80cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 --2- 172.21.15.106:0/2260273131 >> v2:172.21.15.156:3300/0 conn(0x7f01c80cf450 0x7f01c80cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c806d040 con 0x7f01c80cdfb0 2023-11-24T08:12:48.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f01c00019f0 con 0x7f01c80cdfb0 2023-11-24T08:12:48.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.743+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 >> v1:172.21.15.106:6789/0 conn(0x7f01c80cdfb0 legacy=0x7f01c80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c77fe640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 0x7f01b8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c77fe640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01b8003ef0 0x7f01b80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c77fe640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b8004d00 0x7f01b8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8156010 con 0x7f01b8004d00 2023-11-24T08:12:48.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c806ce30 con 0x7f01b8003ef0 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b800b980 con 0x7f01c815c7f0 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b8004d00 0x7f01b8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cccdf640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 0x7f01b8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01c7fff640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01b8003ef0 0x7f01b80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 msgr2=0x7f01b8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01c815c7f0 0x7f01b8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01b8003ef0 msgr2=0x7f01b80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01b8003ef0 0x7f01b80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.744+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2260273131 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b8003520 con 0x7f01b8004d00 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.745+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b8004d00 0x7f01b8005180 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f01c000a8d0 tx=0x7f01b80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96348984cd913510 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0034630 con 0x7f01b8004d00 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c002f070 con 0x7f01b8004d00 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01c77fe640 1 -- 172.21.15.106:0/2260273131 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c00349d0 con 0x7f01b8004d00 2023-11-24T08:12:48.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cef43640 1 -- 172.21.15.106:0/2260273131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b8004d00 msgr2=0x7f01b8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cef43640 1 --2- 172.21.15.106:0/2260273131 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01b8004d00 0x7f01b8005180 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f01c000a8d0 tx=0x7f01b80057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2260273131 reap_dead start 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cef43640 1 -- 172.21.15.106:0/2260273131 shutdown_connections 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cef43640 1 -- 172.21.15.106:0/2260273131 >> 172.21.15.106:0/2260273131 conn(0x7f01c806a920 msgr2=0x7f01c806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cef43640 1 -- 172.21.15.106:0/2260273131 shutdown_connections 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.746+0000 7f01cef43640 1 -- 172.21.15.106:0/2260273131 wait complete. 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.748+0000 7f01cef43640 1 Processor -- start 2023-11-24T08:12:48.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.748+0000 7f01cef43640 1 -- start start 2023-11-24T08:12:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.749+0000 7f01cef43640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01b8004d00 0x7f01c80ac010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.749+0000 7f01cef43640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c80ad740 0x7f01c809c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.749+0000 7f01cef43640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 0x7f01c8099730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.749+0000 7f01cef43640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c809cf10 con 0x7f01c814a2a0 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.749+0000 7f01cef43640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80ac6a0 con 0x7f01c80ad740 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.749+0000 7f01cef43640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80ac980 con 0x7f01b8004d00 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 0x7f01c8099730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 0x7f01c8099730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58980/0 (socket says 172.21.15.106:58980) 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2284818905 learned_addr learned my addr 172.21.15.106:0/2284818905 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01b8004d00 msgr2=0x7f01c80ac010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01b8004d00 0x7f01c80ac010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c80ad740 msgr2=0x7f01c809c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01c7fff640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c80ad740 0x7f01c809c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c80ad740 0x7f01c809c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:48.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 -- 172.21.15.106:0/2284818905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80acc60 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.750+0000 7f01cd4e0640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 0x7f01c8099730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f01c000a9f0 tx=0x7f01c0001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.751+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0043070 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.751+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c0002000 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.751+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c80acf40 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.751+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c0034880 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.751+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c814a730 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.752+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c80ad460 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.753+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f01c0031030 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.754+0000 7f01c57fa640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01ac06b3e0 0x7f01ac06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.754+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f01c00ab790 con 0x7f01c814a2a0 2023-11-24T08:12:48.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.754+0000 7f01cccdf640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01ac06b3e0 0x7f01ac06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:48.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.755+0000 7f01cccdf640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01ac06b3e0 0x7f01ac06d8a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009910 tx=0x7f01bc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:48.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:48.759+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f01c80ad460 con 0x7f01c814a2a0 2023-11-24T08:12:49.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.020+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f01c80ad460 con 0x7f01c814a2a0 2023-11-24T08:12:49.651 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.650+0000 7f01c57fa640 1 -- 172.21.15.106:0/2284818905 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v241) v1 ==== 167+0+0 (secure 0 0 0) 0x7f01c80ad460 con 0x7f01c814a2a0 2023-11-24T08:12:49.654 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01ac06b3e0 msgr2=0x7f01ac06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01ac06b3e0 0x7f01ac06d8a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f01bc009910 tx=0x7f01bc008040 comp rx=0 tx=0).stop 2023-11-24T08:12:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 msgr2=0x7f01c8099730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 0x7f01c8099730 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f01c000a9f0 tx=0x7f01c0001360 comp rx=0 tx=0).stop 2023-11-24T08:12:49.655 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 shutdown_connections 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f01c814a2a0 0x7f01c8099730 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01c80ad740 0x7f01c809c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f01ac06b3e0 0x7f01ac06d8a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 --2- 172.21.15.106:0/2284818905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f01b8004d00 0x7f01c80ac010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.653+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 >> 172.21.15.106:0/2284818905 conn(0x7f01c806a920 msgr2=0x7f01c80ce3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.654+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 shutdown_connections 2023-11-24T08:12:49.656 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.654+0000 7f01cef43640 1 -- 172.21.15.106:0/2284818905 wait complete. 2023-11-24T08:12:49.676 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:12:49.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.889+0000 7fc225153640 1 Processor -- start 2023-11-24T08:12:49.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.889+0000 7fc225153640 1 -- start start 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.890+0000 7fc225153640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc220146eb0 0x7fc220161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.890+0000 7fc225153640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2200cd2e0 0x7fc220070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.890+0000 7fc225153640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc220161a10 con 0x7fc22006e160 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc225153640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc220161cf0 con 0x7fc220146eb0 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc225153640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc220161fd0 con 0x7fc2200cd2e0 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc220146eb0 0x7fc220161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc217fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc22006e160 0x7fc2200ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35390/0 (socket says 172.21.15.106:35390) 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21ed9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fc220146eb0 0x7fc220161150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58982/0 (socket says 172.21.15.106:58982) 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc217fff640 1 -- 172.21.15.106:0/629696078 learned_addr learned my addr 172.21.15.106:0/629696078 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:49.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21f59e640 1 --2- 172.21.15.106:0/629696078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2200cd2e0 0x7fc220070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:49.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21ed9d640 1 -- 172.21.15.106:0/629696078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2200cd2e0 msgr2=0x7fc220070e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:49.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/629696078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2200cd2e0 0x7fc220070e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21ed9d640 1 -- 172.21.15.106:0/629696078 >> v1:172.21.15.106:6789/0 conn(0x7fc22006e160 legacy=0x7fc2200ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:49.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21ed9d640 1 -- 172.21.15.106:0/629696078 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc220070660 con 0x7fc220146eb0 2023-11-24T08:12:49.894 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.891+0000 7fc21e59c640 1 -- 172.21.15.106:0/629696078 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3007150595 0 0) 0x7fc220161a10 con 0x7fc22006e160 2023-11-24T08:12:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/629696078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc220161150 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc2100097a0 tx=0x7fc21002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fd1f0b5810a2aa server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:49.895 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc21e59c640 1 -- 172.21.15.106:0/629696078 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210031830 con 0x7fc220146eb0 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc21e59c640 1 -- 172.21.15.106:0/629696078 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc210031990 con 0x7fc220146eb0 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc21e59c640 1 -- 172.21.15.106:0/629696078 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc210031d20 con 0x7fc220146eb0 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc225153640 1 -- 172.21.15.106:0/629696078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 msgr2=0x7fc220161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc225153640 1 --2- 172.21.15.106:0/629696078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc220161150 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fc2100097a0 tx=0x7fc21002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc225153640 1 -- 172.21.15.106:0/629696078 shutdown_connections 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc225153640 1 --2- 172.21.15.106:0/629696078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc2200cd2e0 0x7fc220070e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc225153640 1 --2- 172.21.15.106:0/629696078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc220161150 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.893+0000 7fc225153640 1 -- 172.21.15.106:0/629696078 >> 172.21.15.106:0/629696078 conn(0x7fc2200b0c50 msgr2=0x7fc2200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.894+0000 7fc225153640 1 -- 172.21.15.106:0/629696078 shutdown_connections 2023-11-24T08:12:49.896 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.894+0000 7fc225153640 1 -- 172.21.15.106:0/629696078 wait complete. 2023-11-24T08:12:49.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.895+0000 7fc225153640 1 Processor -- start 2023-11-24T08:12:49.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.895+0000 7fc225153640 1 -- start start 2023-11-24T08:12:49.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.896+0000 7fc225153640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22006e160 0x7fc22015c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:49.898 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.896+0000 7fc225153640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2200cd2e0 0x7fc220142df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.896+0000 7fc225153640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc22015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.896+0000 7fc225153640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2200b2b30 con 0x7fc220146eb0 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.896+0000 7fc225153640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc220143330 con 0x7fc2200cd2e0 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.896+0000 7fc225153640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc22015e460 con 0x7fc22006e160 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc22015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc22015df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:58998/0 (socket says 172.21.15.106:58998) 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 -- 172.21.15.106:0/266528851 learned_addr learned my addr 172.21.15.106:0/266528851 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 -- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22006e160 msgr2=0x7fc22015c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22006e160 0x7fc22015c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc217fff640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2200cd2e0 0x7fc220142df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 -- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2200cd2e0 msgr2=0x7fc220142df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2200cd2e0 0x7fc220142df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:49.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 -- 172.21.15.106:0/266528851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc210009400 con 0x7fc220146eb0 2023-11-24T08:12:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.897+0000 7fc21f59e640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc22015df20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc21800cd70 tx=0x7fc21800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.898+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218003b90 con 0x7fc220146eb0 2023-11-24T08:12:49.900 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.898+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc218003cf0 con 0x7fc220146eb0 2023-11-24T08:12:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.898+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218004240 con 0x7fc220146eb0 2023-11-24T08:12:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.898+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2200b1eb0 con 0x7fc220146eb0 2023-11-24T08:12:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.898+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2200b2470 con 0x7fc220146eb0 2023-11-24T08:12:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.899+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22004e8b0 con 0x7fc220146eb0 2023-11-24T08:12:49.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.900+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fc2180043a0 con 0x7fc220146eb0 2023-11-24T08:12:49.902 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.901+0000 7fc2177fe640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006b3e0 0x7fc20006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.901+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc218012070 con 0x7fc220146eb0 2023-11-24T08:12:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.902+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006b3e0 0x7fc20006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:49.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.902+0000 7fc21ed9d640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006b3e0 0x7fc20006d8a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc210036000 tx=0x7fc210030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:49.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:49.906+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc2180552b0 con 0x7fc220146eb0 2023-11-24T08:12:50.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.095+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc220143560 con 0x7fc220146eb0 2023-11-24T08:12:50.103 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.102+0000 7fc2177fe640 1 -- 172.21.15.106:0/266528851 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc2180588d0 con 0x7fc220146eb0 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006b3e0 msgr2=0x7fc20006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006b3e0 0x7fc20006d8a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc210036000 tx=0x7fc210030000 comp rx=0 tx=0).stop 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 msgr2=0x7fc22015df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc22015df20 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fc21800cd70 tx=0x7fc21800a660 comp rx=0 tx=0).stop 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 shutdown_connections 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc220146eb0 0x7fc22015df20 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2200cd2e0 0x7fc220142df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc20006b3e0 0x7fc20006d8a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 --2- 172.21.15.106:0/266528851 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc22006e160 0x7fc22015c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 >> 172.21.15.106:0/266528851 conn(0x7fc2200b0c50 msgr2=0x7fc2200b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 shutdown_connections 2023-11-24T08:12:50.107 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.105+0000 7fc225153640 1 -- 172.21.15.106:0/266528851 wait complete. 2023-11-24T08:12:50.126 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:12:50.126 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:12:50.225 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:12:50.354 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:12:50.354 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:50.354 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:50.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:12:50.411 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:50.411 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:50.471 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:12:50.471 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:12:50.471 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:50.531 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:12:50.531 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:12:50.531 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:12:50.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:12:50.594 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-24T08:12:50.594 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:12:50.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.814+0000 7f7fba91d640 1 Processor -- start 2023-11-24T08:12:50.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.814+0000 7f7fba91d640 1 -- start start 2023-11-24T08:12:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fba91d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7fb40cc150 0x7f7fb4164af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fba91d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40b3a40 con 0x7f7fb4159ce0 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fba91d640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fb40cdfa0 con 0x7f7fb40b21d0 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fba91d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fb40cd9e0 con 0x7f7fb40cc150 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fb3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f7fb4159ce0 0x7f7fb406de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35404/0 (socket says 172.21.15.106:35404) 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fb3fff640 1 -- 172.21.15.106:0/2584864944 learned_addr learned my addr 172.21.15.106:0/2584864944 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.816+0000 7f7fb8eba640 1 --2- 172.21.15.106:0/2584864944 >> v2:172.21.15.156:3300/0 conn(0x7f7fb40cc150 0x7f7fb4164af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179198905 0 0) 0x7f7fb40b3a40 con 0x7f7fb4159ce0 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7fa0003520 con 0x7f7fb4159ce0 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3370882871 0 0) 0x7f7fa0003520 con 0x7f7fb4159ce0 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 >> v2:172.21.15.156:3300/0 conn(0x7f7fb40cc150 msgr2=0x7f7fb4164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:50.822 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 --2- 172.21.15.106:0/2584864944 >> v2:172.21.15.156:3300/0 conn(0x7f7fb40cc150 0x7f7fb4164af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 >> v1:172.21.15.156:6789/0 conn(0x7f7fb40b21d0 legacy=0x7f7fb40cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40cdcc0 con 0x7f7fb4159ce0 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f7fa4001670 con 0x7f7fb4159ce0 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 >> v1:172.21.15.106:6789/0 conn(0x7f7fb4159ce0 legacy=0x7f7fb406de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0003970 0x7f7fa0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa00047c0 0x7f7fa0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fa0005180 0x7f7fa000ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb40cd9e0 con 0x7f7fa0005180 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa000d130 con 0x7f7fa00047c0 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.817+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa00059d0 con 0x7f7fa0003970 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fb8eba640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fa0005180 0x7f7fa000ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fb8eba640 1 -- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0003970 msgr2=0x7f7fa0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:50.823 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fb8eba640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0003970 0x7f7fa0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fb8eba640 1 -- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa00047c0 msgr2=0x7f7fa0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fabfff640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa00047c0 0x7f7fa0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fb8eba640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa00047c0 0x7f7fa0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fb8eba640 1 -- 172.21.15.106:0/2584864944 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa0003520 con 0x7f7fa0005180 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.818+0000 7f7fabfff640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fa00047c0 0x7f7fa0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.819+0000 7f7fb8eba640 1 -- 172.21.15.106:0/2584864944 reap_dead start 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.819+0000 7f7fb8eba640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fa0005180 0x7f7fa000ca90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009f90 tx=0x7f7fa00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976f6f6ccc3d6363 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.819+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac02cc20 con 0x7f7fa0005180 2023-11-24T08:12:50.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.819+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fac00f040 con 0x7f7fa0005180 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.819+0000 7f7fb37fe640 1 -- 172.21.15.106:0/2584864944 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fac003990 con 0x7f7fa0005180 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 -- 172.21.15.106:0/2584864944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fa0005180 msgr2=0x7f7fa000ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fa0005180 0x7f7fa000ca90 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f7fac009f90 tx=0x7f7fa00057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 -- 172.21.15.106:0/2584864944 shutdown_connections 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 --2- 172.21.15.106:0/2584864944 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fa0005180 0x7f7fa000ca90 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 -- 172.21.15.106:0/2584864944 >> 172.21.15.106:0/2584864944 conn(0x7f7fb406a920 msgr2=0x7f7fb406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 -- 172.21.15.106:0/2584864944 shutdown_connections 2023-11-24T08:12:50.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.820+0000 7f7fba91d640 1 -- 172.21.15.106:0/2584864944 wait complete. 2023-11-24T08:12:50.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.822+0000 7f7fba91d640 1 Processor -- start 2023-11-24T08:12:50.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.822+0000 7f7fba91d640 1 -- start start 2023-11-24T08:12:50.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fba91d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0005180 0x7f7fb4098ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fba91d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 0x7f7fb4153550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fba91d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fb41555c0 0x7f7fb4153a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fba91d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41540c0 con 0x7f7fb4155160 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fba91d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb41543a0 con 0x7f7fb41555c0 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fba91d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fb4154680 con 0x7f7fa0005180 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 0x7f7fb4153550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 0x7f7fb4153550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59014/0 (socket says 172.21.15.106:59014) 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 -- 172.21.15.106:0/1234696917 learned_addr learned my addr 172.21.15.106:0/1234696917 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fb3fff640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0005180 0x7f7fb4098ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 -- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0005180 msgr2=0x7f7fb4098ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0005180 0x7f7fb4098ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 -- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fb41555c0 msgr2=0x7f7fb4153a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fb8eba640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fb41555c0 0x7f7fb4153a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fb41555c0 0x7f7fb4153a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.823+0000 7f7fabfff640 1 -- 172.21.15.106:0/1234696917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb4154960 con 0x7f7fb4155160 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.824+0000 7f7fabfff640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 0x7f7fb4153550 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c000f60 tx=0x7f7f9c00d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.824+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c005480 con 0x7f7fb4155160 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.824+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c003610 con 0x7f7fb4155160 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.824+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0039c0 con 0x7f7fb4155160 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.824+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fb4154c40 con 0x7f7fb4155160 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.824+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fb40ad010 con 0x7f7fb4155160 2023-11-24T08:12:50.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.825+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fb4154e00 con 0x7f7fb4155160 2023-11-24T08:12:50.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.826+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f7f9c003b20 con 0x7f7fb4155160 2023-11-24T08:12:50.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.827+0000 7f7fb17fa640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7f9406b3e0 0x7f7f9406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:50.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.827+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7f9c089dd0 con 0x7f7fb4155160 2023-11-24T08:12:50.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.828+0000 7f7fb3fff640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7f9406b3e0 0x7f7f9406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:50.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.828+0000 7f7fb3fff640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7f9406b3e0 0x7f7f9406d8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000ff0 tx=0x7f7fa4009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:50.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:50.832+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7f9c054820 con 0x7f7fb4155160 2023-11-24T08:12:51.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.061+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7fb40cd030 con 0x7f7fb4155160 2023-11-24T08:12:51.063 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.062+0000 7f7fb17fa640 1 -- 172.21.15.106:0/1234696917 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f7f9c001230 con 0x7f7fb4155160 2023-11-24T08:12:51.064 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:51.064 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":46,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4907,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/4241158407","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":4241158407},{"type":"v1","addr":"172.21.15.156:6834","nonce":4241158407}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":4910,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/3259058141","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":3259058141},{"type":"v1","addr":"172.21.15.156:6835","nonce":3259058141}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":38},{"gid":6513,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/3324878854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":3324878854},{"type":"v1","addr":"172.21.15.106:6836","nonce":3324878854}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":37},{"gid":6939,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/3349342011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":3349342011},{"type":"v1","addr":"172.21.15.106:6837","nonce":3349342011}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":46}],"filesystems":[]} 2023-11-24T08:12:51.064 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 46 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7f9406b3e0 msgr2=0x7f7f9406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7f9406b3e0 0x7f7f9406d8a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000ff0 tx=0x7f7fa4009040 comp rx=0 tx=0).stop 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 msgr2=0x7f7fb4153550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 0x7f7fb4153550 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c000f60 tx=0x7f7f9c00d9f0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 shutdown_connections 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7fb41555c0 0x7f7fb4153a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7fb4155160 0x7f7fb4153550 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7f9406b3e0 0x7f7f9406d8a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 --2- 172.21.15.106:0/1234696917 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7fa0005180 0x7f7fb4098ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.067 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 >> 172.21.15.106:0/1234696917 conn(0x7f7fb406a920 msgr2=0x7f7fb406de80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 shutdown_connections 2023-11-24T08:12:51.068 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.065+0000 7f7fba91d640 1 -- 172.21.15.106:0/1234696917 wait complete. 2023-11-24T08:12:51.086 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-24T08:12:51.086 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-24T08:12:51.087 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:12:51.087 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:51.087 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-24T08:12:51.087 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-24T08:12:51.087 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:12:51.087 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:51.088 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-24T08:12:51.088 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-24T08:12:51.088 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:12:51.088 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:51.088 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-24T08:12:51.088 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-24T08:12:51.088 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:12:51.088 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:12:51.103 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:12:51.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-24T08:12:51.106 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:12:51.107 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-24T08:12:51.110 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:12:51.110 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-24T08:12:51.114 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:12:51.114 DEBUG:teuthology.orchestra.run.smithi156:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-24T08:12:51.154 INFO:tasks.ceph.mds.c:Started 2023-11-24T08:12:51.154 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:51.162 INFO:tasks.ceph.mds.a:Started 2023-11-24T08:12:51.162 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:51.163 INFO:tasks.ceph.mds.b:Started 2023-11-24T08:12:51.163 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:51.170 INFO:tasks.ceph.mds.d:Started 2023-11-24T08:12:51.170 DEBUG:teuthology.parallel:result is None 2023-11-24T08:12:51.170 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-24T08:12:51.313 INFO:tasks.ceph.mds.c.smithi106.stdout:starting mds.c at 2023-11-24T08:12:51.314 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:12:51.329 INFO:tasks.ceph.mds.d.smithi156.stdout:starting mds.d at 2023-11-24T08:12:51.330 INFO:tasks.ceph.mds.b.smithi156.stdout:starting mds.b at 2023-11-24T08:12:51.331 INFO:tasks.ceph.mds.b.smithi156.stderr:2023-11-24T08:12:51.329+0000 7f308fe3c640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-24T08:12:51.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.357+0000 7f8cc0048640 1 Processor -- start 2023-11-24T08:12:51.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.357+0000 7f8cc0048640 1 -- start start 2023-11-24T08:12:51.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cc0048640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cc0048640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8cb80c49f0 0x7f8cb806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cc0048640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8cb806dc40 0x7f8cb80cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cc0048640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb806e0f0 con 0x7f8cb806dc40 2023-11-24T08:12:51.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cc0048640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb815ec90 con 0x7f8cb80c49f0 2023-11-24T08:12:51.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cc0048640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80cd530 con 0x7f8cb8072160 2023-11-24T08:12:51.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cbe5e5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8cb806dc40 0x7f8cb80cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cbe5e5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8cb806dc40 0x7f8cb80cd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59040/0 (socket says 172.21.15.106:59040) 2023-11-24T08:12:51.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.358+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/1694204034 learned_addr learned my addr 172.21.15.106:0/1694204034 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:51.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbd5e3640 1 --2- 172.21.15.106:0/1694204034 >> v2:172.21.15.156:3300/0 conn(0x7f8cb80c49f0 0x7f8cb806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbdde4640 1 --2- 172.21.15.106:0/1694204034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/1694204034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 msgr2=0x7f8cb815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbe5e5640 1 --2- 172.21.15.106:0/1694204034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/1694204034 >> v2:172.21.15.156:3300/0 conn(0x7f8cb80c49f0 msgr2=0x7f8cb806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbe5e5640 1 --2- 172.21.15.106:0/1694204034 >> v2:172.21.15.156:3300/0 conn(0x7f8cb80c49f0 0x7f8cb806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.359+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/1694204034 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb80c4ed0 con 0x7f8cb806dc40 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.360+0000 7f8cbe5e5640 1 --2- 172.21.15.106:0/1694204034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb806dc40 0x7f8cb80cd750 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80ccb10 tx=0x7f8cb002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d08fb69dd434d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.360+0000 7f8cbcde2640 1 -- 172.21.15.106:0/1694204034 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb002ca70 con 0x7f8cb806dc40 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.360+0000 7f8cbcde2640 1 -- 172.21.15.106:0/1694204034 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb002cbd0 con 0x7f8cb806dc40 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.360+0000 7f8cbcde2640 1 -- 172.21.15.106:0/1694204034 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00343f0 con 0x7f8cb806dc40 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 -- 172.21.15.106:0/1694204034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb806dc40 msgr2=0x7f8cb80cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 --2- 172.21.15.106:0/1694204034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb806dc40 0x7f8cb80cd750 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8cb80ccb10 tx=0x7f8cb002bed0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 -- 172.21.15.106:0/1694204034 shutdown_connections 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 --2- 172.21.15.106:0/1694204034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb806dc40 0x7f8cb80cd750 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 --2- 172.21.15.106:0/1694204034 >> v2:172.21.15.156:3300/0 conn(0x7f8cb80c49f0 0x7f8cb806d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 --2- 172.21.15.106:0/1694204034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 -- 172.21.15.106:0/1694204034 >> 172.21.15.106:0/1694204034 conn(0x7f8cb80b0c50 msgr2=0x7f8cb80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 -- 172.21.15.106:0/1694204034 shutdown_connections 2023-11-24T08:12:51.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.361+0000 7f8cc0048640 1 -- 172.21.15.106:0/1694204034 wait complete. 2023-11-24T08:12:51.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.363+0000 7f8cc0048640 1 Processor -- start 2023-11-24T08:12:51.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.363+0000 7f8cc0048640 1 -- start start 2023-11-24T08:12:51.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cc0048640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb806dc40 0x7f8cb80ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cc0048640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cc0048640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 0x7f8cb8152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cc0048640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80acc00 con 0x7f8cb80c49f0 2023-11-24T08:12:51.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cc0048640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb80c84d0 con 0x7f8cb806dc40 2023-11-24T08:12:51.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cc0048640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb815cc60 con 0x7f8cb8072160 2023-11-24T08:12:51.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cbe5e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 0x7f8cb8152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cbe5e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 0x7f8cb8152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59046/0 (socket says 172.21.15.106:59046) 2023-11-24T08:12:51.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.364+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/995325146 learned_addr learned my addr 172.21.15.106:0/995325146 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:51.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbdde4640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb806dc40 0x7f8cb80ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbd5e3640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 msgr2=0x7f8cb815c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbe5e5640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb806dc40 msgr2=0x7f8cb80ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbe5e5640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb806dc40 0x7f8cb80ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbe5e5640 1 -- 172.21.15.106:0/995325146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb0009b30 con 0x7f8cb80c49f0 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cbe5e5640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 0x7f8cb8152010 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0036040 tx=0x7f8cb0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:51.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0009510 con 0x7f8cb80c49f0 2023-11-24T08:12:51.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb003e450 con 0x7f8cb80c49f0 2023-11-24T08:12:51.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb003e800 con 0x7f8cb80c49f0 2023-11-24T08:12:51.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.365+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb80c7b30 con 0x7f8cb80c49f0 2023-11-24T08:12:51.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.366+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb80c80f0 con 0x7f8cb80c49f0 2023-11-24T08:12:51.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.367+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb80c8610 con 0x7f8cb80c49f0 2023-11-24T08:12:51.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.368+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8cb003e960 con 0x7f8cb80c49f0 2023-11-24T08:12:51.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.369+0000 7f8cae7fc640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c9c06b410 0x7f8c9c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.369+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8cb00acad0 con 0x7f8cb80c49f0 2023-11-24T08:12:51.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.369+0000 7f8cbdde4640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c9c06b410 0x7f8c9c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.370+0000 7f8cbdde4640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c9c06b410 0x7f8c9c06d8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006fd0 tx=0x7f8cb4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:51.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.373+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8cb80c8610 con 0x7f8cb80c49f0 2023-11-24T08:12:51.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.573+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8cb80c8610 con 0x7f8cb80c49f0 2023-11-24T08:12:51.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.675+0000 7f8cae7fc640 1 -- 172.21.15.106:0/995325146 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v83) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8cb80c8610 con 0x7f8cb80c49f0 2023-11-24T08:12:51.677 INFO:teuthology.orchestra.run.smithi106.stderr: removed all blocklist entries 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c9c06b410 msgr2=0x7f8c9c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c9c06b410 0x7f8c9c06d8d0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4006fd0 tx=0x7f8cb4008040 comp rx=0 tx=0).stop 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 msgr2=0x7f8cb8152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 0x7f8cb8152010 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0036040 tx=0x7f8cb0004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 shutdown_connections 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8cb80c49f0 0x7f8cb8152010 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c9c06b410 0x7f8c9c06d8d0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8cb8072160 0x7f8cb815c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 --2- 172.21.15.106:0/995325146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8cb806dc40 0x7f8cb80ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 >> 172.21.15.106:0/995325146 conn(0x7f8cb80b0c50 msgr2=0x7f8cb80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:51.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 shutdown_connections 2023-11-24T08:12:51.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.679+0000 7f8cc0048640 1 -- 172.21.15.106:0/995325146 wait complete. 2023-11-24T08:12:51.701 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-24T08:12:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.897+0000 7f42a01c9640 1 Processor -- start 2023-11-24T08:12:51.899 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.898+0000 7f42a01c9640 1 -- start start 2023-11-24T08:12:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.899+0000 7f42a01c9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f42980b6be0 0x7f42980d2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.899+0000 7f42a01c9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980d3120 con 0x7f42980d27a0 2023-11-24T08:12:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.899+0000 7f42a01c9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42980b2930 con 0x7f429815d520 2023-11-24T08:12:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.899+0000 7f42a01c9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42980d1420 con 0x7f42980b6be0 2023-11-24T08:12:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.899+0000 7f429d764640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f42980d27a0 0x7f42980b64d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35422/0 (socket says 172.21.15.106:35422) 2023-11-24T08:12:51.901 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.899+0000 7f429d764640 1 -- 172.21.15.106:0/3254554110 learned_addr learned my addr 172.21.15.106:0/3254554110 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429e766640 1 --2- 172.21.15.106:0/3254554110 >> v2:172.21.15.156:3300/0 conn(0x7f42980b6be0 0x7f42980d2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272033751 0 0) 0x7f42980d3120 con 0x7f42980d27a0 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4288003520 con 0x7f42980d27a0 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 437758076 0 0) 0x7f4288003520 con 0x7f42980d27a0 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 >> v2:172.21.15.156:3300/0 conn(0x7f42980b6be0 msgr2=0x7f42980d2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 --2- 172.21.15.106:0/3254554110 >> v2:172.21.15.156:3300/0 conn(0x7f42980b6be0 0x7f42980d2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.903 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 >> v1:172.21.15.156:6789/0 conn(0x7f429815d520 legacy=0x7f42980b8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42980d1700 con 0x7f42980d27a0 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f4284001670 con 0x7f42980d27a0 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 >> v1:172.21.15.106:6789/0 conn(0x7f42980d27a0 legacy=0x7f42980b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288003970 0x7f4288003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42880047c0 0x7f4288004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4288005180 0x7f428800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980d1420 con 0x7f4288005180 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428800c030 con 0x7f42880047c0 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.900+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428800c1f0 con 0x7f4288003970 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429e766640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4288005180 0x7f428800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429e766640 1 -- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288003970 msgr2=0x7f4288003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429df65640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288003970 0x7f4288003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429d764640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42880047c0 0x7f4288004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429e766640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288003970 0x7f4288003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429e766640 1 -- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42880047c0 msgr2=0x7f4288004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429e766640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42880047c0 0x7f4288004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429e766640 1 -- 172.21.15.106:0/3254554110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288003520 con 0x7f4288005180 2023-11-24T08:12:51.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.901+0000 7f429d764640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42880047c0 0x7f4288004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.902+0000 7f429e766640 1 -- 172.21.15.106:0/3254554110 reap_dead start 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.902+0000 7f429e766640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4288005180 0x7f428800b980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f42980b6460 tx=0x7f42880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d236dc76d6c27e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.902+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429002c680 con 0x7f4288005180 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429002c800 con 0x7f4288005180 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f429cf63640 1 -- 172.21.15.106:0/3254554110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429002cbb0 con 0x7f4288005180 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 -- 172.21.15.106:0/3254554110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4288005180 msgr2=0x7f428800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4288005180 0x7f428800b980 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f42980b6460 tx=0x7f42880057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 -- 172.21.15.106:0/3254554110 shutdown_connections 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 --2- 172.21.15.106:0/3254554110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4288005180 0x7f428800b980 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 -- 172.21.15.106:0/3254554110 >> 172.21.15.106:0/3254554110 conn(0x7f4298061340 msgr2=0x7f4298061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:51.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 -- 172.21.15.106:0/3254554110 shutdown_connections 2023-11-24T08:12:51.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.903+0000 7f42a01c9640 1 -- 172.21.15.106:0/3254554110 wait complete. 2023-11-24T08:12:51.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.905+0000 7f42a01c9640 1 Processor -- start 2023-11-24T08:12:51.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.905+0000 7f42a01c9640 1 -- start start 2023-11-24T08:12:51.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f42a01c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288005180 0x7f42981f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f42a01c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4298158040 0x7f42981568e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f42a01c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 0x7f42980cc900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f42a01c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f429815a4a0 con 0x7f42980cc480 2023-11-24T08:12:51.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f42a01c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42980cce40 con 0x7f4298158040 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f42a01c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42981f7330 con 0x7f4288005180 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.906+0000 7f429e766640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 0x7f42980cc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 0x7f42980cc900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59060/0 (socket says 172.21.15.106:59060) 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 -- 172.21.15.106:0/1188932376 learned_addr learned my addr 172.21.15.106:0/1188932376 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429d764640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4298158040 0x7f42981568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 -- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288005180 msgr2=0x7f42981f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429df65640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288005180 0x7f42981f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288005180 0x7f42981f6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 -- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4298158040 msgr2=0x7f42981568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4298158040 0x7f42981568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:51.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 -- 172.21.15.106:0/1188932376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298158570 con 0x7f42980cc480 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429d764640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4298158040 0x7f42981568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f429e766640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 0x7f42980cc900 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f429002c2f0 tx=0x7f4290004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.907+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290034910 con 0x7f42980cc480 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.908+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4290034a70 con 0x7f42980cc480 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.908+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4290034de0 con 0x7f42980cc480 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.908+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4298158850 con 0x7f42980cc480 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.908+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f429815f510 con 0x7f42980cc480 2023-11-24T08:12:51.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.909+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429815f730 con 0x7f42980cc480 2023-11-24T08:12:51.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.910+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f429003b040 con 0x7f42980cc480 2023-11-24T08:12:51.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.910+0000 7f42967fc640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427c06b3e0 0x7f427c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:51.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.911+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f42900aa8f0 con 0x7f42980cc480 2023-11-24T08:12:51.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.911+0000 7f429df65640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427c06b3e0 0x7f427c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:51.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.911+0000 7f429df65640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427c06b3e0 0x7f427c06d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f428c000f80 tx=0x7f428c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:51.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:51.915+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f42900755c0 con 0x7f42980cc480 2023-11-24T08:12:52.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.151+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f429804e8b0 con 0x7f42980cc480 2023-11-24T08:12:52.154 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.152+0000 7f42967fc640 1 -- 172.21.15.106:0/1188932376 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4290046090 con 0x7f42980cc480 2023-11-24T08:12:52.154 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:52.154 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "auth_dump": [ 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.a", 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBlssD3DxAAMQjKxFeoQZ8yHxieHM+a4A==", 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.155 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.b", 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl1/ZVLxAA/UYZ65HGyjFGmrMP1PJdhg==", 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.156 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.c", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBl2jBmHhAAvf9qPf7sV3A+RrJhNuDC3Q==", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mds.d", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlOO7jBBAAQUv6g46Kd6qCMpvS6I5Ajw==", 2023-11-24T08:12:52.157 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.0", 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDaWWBlN3lJMhAAPw9t+wgrbbrgm2UjpIkbbA==", 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.158 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.1", 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDeWWBl/UkIARAAcYgPgjyHBh7TC/xVOr7XxA==", 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.159 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.2", 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDhWWBlY/JACxAAOkrZhFhQCjWiH4Z5tD5fVA==", 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.3", 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDkWWBlkimLFBAA5PgsZk3yonUP50bvq7Wibw==", 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.160 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.4", 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDpWWBlRKa7BBAAFvHvnBgKM9wigkbeS4eJTw==", 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.161 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.5", 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDsWWBlRts0HRAAdUoOFMOQRA306btayXmjaw==", 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.162 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.6", 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDvWWBlB0CFNBAAH3W8vGlAT8fGdB+rCP+2TA==", 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "osd.7", 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDzWWBlStYuERAARE6fsm0NdI0IJ2h+hkeTug==", 2023-11-24T08:12:52.163 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow profile osd", 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile osd", 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.0", 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ==", 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow r", 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-24T08:12:52.164 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.1", 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBl5fdJJhAAV6oTQukcBkje97asJYQvNg==", 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.165 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.2", 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlbhCSGxAAtlheeoUlh5x4djzUPzghbg==", 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.3", 2023-11-24T08:12:52.166 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDWWWBlaK8DMRAA9PNXur/sUwnFwuiHX++n+w==", 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow", 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow r", 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow rw", 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow rwx" 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "client.admin", 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBludbKAhAARxKqsuG6+yx4M/G/4IloBw==", 2023-11-24T08:12:52.167 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "mgr": "allow *", 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow *", 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.x", 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDVWWBliaT1AxAAuULtkbyXymxX2g7uuYsjbQ==", 2023-11-24T08:12:52.168 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: }, 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: "entity": "mgr.y", 2023-11-24T08:12:52.169 INFO:teuthology.orchestra.run.smithi106.stdout: "key": "AQDUWWBluyYzNBAA/TPUnXeslBQa+oWXqOvKhA==", 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: "caps": { 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: "mds": "allow *", 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: "mon": "allow profile mgr", 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: "osd": "allow *" 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout: ] 2023-11-24T08:12:52.170 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:12:52.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427c06b3e0 msgr2=0x7f427c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427c06b3e0 0x7f427c06d8a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f428c000f80 tx=0x7f428c009040 comp rx=0 tx=0).stop 2023-11-24T08:12:52.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 msgr2=0x7f42980cc900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 0x7f42980cc900 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f429002c2f0 tx=0x7f4290004730 comp rx=0 tx=0).stop 2023-11-24T08:12:52.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 shutdown_connections 2023-11-24T08:12:52.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f42980cc480 0x7f42980cc900 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4298158040 0x7f42981568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427c06b3e0 0x7f427c06d8a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 --2- 172.21.15.106:0/1188932376 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4288005180 0x7f42981f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 >> 172.21.15.106:0/1188932376 conn(0x7f4298061340 msgr2=0x7f42980b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:52.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 shutdown_connections 2023-11-24T08:12:52.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.155+0000 7f42a01c9640 1 -- 172.21.15.106:0/1188932376 wait complete. 2023-11-24T08:12:52.175 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:52.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.381+0000 7fd14e49b640 1 Processor -- start 2023-11-24T08:12:52.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.381+0000 7fd14e49b640 1 -- start start 2023-11-24T08:12:52.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14e49b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fd14815df80 0x7fd1480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14e49b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd1480cd470 0x7fd14815b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14e49b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd14815e3f0 con 0x7fd1480b2250 2023-11-24T08:12:52.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14e49b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14815a990 con 0x7fd1480cd470 2023-11-24T08:12:52.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14e49b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd14815ab80 con 0x7fd14815df80 2023-11-24T08:12:52.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14ca38640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fd1480cd470 0x7fd14815b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd147fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fd1480b2250 0x7fd1480b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:35432/0 (socket says 172.21.15.106:35432) 2023-11-24T08:12:52.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd147fff640 1 -- 172.21.15.106:0/1420981467 learned_addr learned my addr 172.21.15.106:0/1420981467 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:52.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14ca38640 1 -- 172.21.15.106:0/1420981467 >> v2:172.21.15.156:3300/0 conn(0x7fd14815df80 msgr2=0x7fd1480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:52.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14ca38640 1 --2- 172.21.15.106:0/1420981467 >> v2:172.21.15.156:3300/0 conn(0x7fd14815df80 0x7fd1480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14ca38640 1 -- 172.21.15.106:0/1420981467 >> v1:172.21.15.106:6789/0 conn(0x7fd1480b2250 legacy=0x7fd1480b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.383+0000 7fd14ca38640 1 -- 172.21.15.106:0/1420981467 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480b2d40 con 0x7fd1480cd470 2023-11-24T08:12:52.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.384+0000 7fd14ca38640 1 --2- 172.21.15.106:0/1420981467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480cd470 0x7fd14815b080 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd1480ae5c0 tx=0x7fd14002bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d2bcfac3a2e2ec server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd146ffd640 1 -- 172.21.15.106:0/1420981467 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd14002cc70 con 0x7fd1480cd470 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd146ffd640 1 -- 172.21.15.106:0/1420981467 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd14002cdd0 con 0x7fd1480cd470 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd146ffd640 1 -- 172.21.15.106:0/1420981467 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd140034660 con 0x7fd1480cd470 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 -- 172.21.15.106:0/1420981467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480cd470 msgr2=0x7fd14815b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 --2- 172.21.15.106:0/1420981467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480cd470 0x7fd14815b080 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fd1480ae5c0 tx=0x7fd14002bec0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 -- 172.21.15.106:0/1420981467 shutdown_connections 2023-11-24T08:12:52.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 --2- 172.21.15.106:0/1420981467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480cd470 0x7fd14815b080 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 --2- 172.21.15.106:0/1420981467 >> v2:172.21.15.156:3300/0 conn(0x7fd14815df80 0x7fd1480ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 -- 172.21.15.106:0/1420981467 >> 172.21.15.106:0/1420981467 conn(0x7fd14806a920 msgr2=0x7fd14806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:52.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 -- 172.21.15.106:0/1420981467 shutdown_connections 2023-11-24T08:12:52.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.385+0000 7fd14e49b640 1 -- 172.21.15.106:0/1420981467 wait complete. 2023-11-24T08:12:52.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.387+0000 7fd14e49b640 1 Processor -- start 2023-11-24T08:12:52.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.387+0000 7fd14e49b640 1 -- start start 2023-11-24T08:12:52.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd14e49b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 0x7fd1481492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd14e49b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1480cd470 0x7fd148152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd14e49b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14815df80 0x7fd14815ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd14e49b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1481507b0 con 0x7fd1480b2250 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd14e49b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480adcd0 con 0x7fd1480cd470 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd14e49b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd148150a00 con 0x7fd14815df80 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd147fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 0x7fd1481492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd147fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 0x7fd1481492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59070/0 (socket says 172.21.15.106:59070) 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.388+0000 7fd147fff640 1 -- 172.21.15.106:0/4067435901 learned_addr learned my addr 172.21.15.106:0/4067435901 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd147fff640 1 -- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14815df80 msgr2=0x7fd14815ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd147fff640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14815df80 0x7fd14815ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd147fff640 1 -- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1480cd470 msgr2=0x7fd148152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd1477fe640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1480cd470 0x7fd148152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd147fff640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1480cd470 0x7fd148152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd147fff640 1 -- 172.21.15.106:0/4067435901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd140009a70 con 0x7fd1480b2250 2023-11-24T08:12:52.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd1477fe640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1480cd470 0x7fd148152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:52.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd147fff640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 0x7fd1481492d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c4a0 tx=0x7fd13800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:52.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138005ce0 con 0x7fd1480b2250 2023-11-24T08:12:52.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd138005e40 con 0x7fd1480b2250 2023-11-24T08:12:52.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd148150ce0 con 0x7fd1480b2250 2023-11-24T08:12:52.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.389+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138007270 con 0x7fd1480b2250 2023-11-24T08:12:52.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.390+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14815d9f0 con 0x7fd1480b2250 2023-11-24T08:12:52.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.391+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd148067470 con 0x7fd1480b2250 2023-11-24T08:12:52.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.391+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fd138007490 con 0x7fd1480b2250 2023-11-24T08:12:52.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.392+0000 7fd144ff9640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13406b3e0 0x7fd13406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.392+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd1380890f0 con 0x7fd1480b2250 2023-11-24T08:12:52.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.393+0000 7fd1477fe640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13406b3e0 0x7fd13406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.393+0000 7fd1477fe640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13406b3e0 0x7fd13406d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd1480ccf80 tx=0x7fd13c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:52.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.396+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fd138053d80 con 0x7fd1480b2250 2023-11-24T08:12:52.591 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.589+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1480cea00 con 0x7fd1480b2250 2023-11-24T08:12:52.593 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.591+0000 7fd144ff9640 1 -- 172.21.15.106:0/4067435901 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+12906 (secure 0 0 0) 0x7fd138017090 con 0x7fd1480b2250 2023-11-24T08:12:52.593 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:52.593 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":85,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:51.687029+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:52.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13406b3e0 msgr2=0x7fd13406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13406b3e0 0x7fd13406d8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fd1480ccf80 tx=0x7fd13c008040 comp rx=0 tx=0).stop 2023-11-24T08:12:52.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 msgr2=0x7fd1481492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 0x7fd1481492d0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fd13800c4a0 tx=0x7fd13800c960 comp rx=0 tx=0).stop 2023-11-24T08:12:52.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 shutdown_connections 2023-11-24T08:12:52.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fd13406b3e0 0x7fd13406d8a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fd14815df80 0x7fd14815ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd1480cd470 0x7fd148152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 --2- 172.21.15.106:0/4067435901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fd1480b2250 0x7fd1481492d0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 >> 172.21.15.106:0/4067435901 conn(0x7fd14806a920 msgr2=0x7fd148158590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:52.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 shutdown_connections 2023-11-24T08:12:52.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.594+0000 7fd14e49b640 1 -- 172.21.15.106:0/4067435901 wait complete. 2023-11-24T08:12:52.615 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:12:52.615 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:12:52.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.831+0000 7f1805cdf640 1 Processor -- start 2023-11-24T08:12:52.833 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.831+0000 7f1805cdf640 1 -- start start 2023-11-24T08:12:52.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.832+0000 7f1805cdf640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f18000b0b40 0x7f18000b0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.834 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.832+0000 7f1805cdf640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f18000cce40 0x7f18000cd2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.832+0000 7f1805cdf640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18000b14d0 con 0x7f180015d310 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.832+0000 7f1805cdf640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f180006cd10 con 0x7f18000b0b40 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f1805cdf640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18000ae5d0 con 0x7f18000cce40 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f18000b0b40 0x7f18000b0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f18000b0b40 0x7f18000b0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59084/0 (socket says 172.21.15.106:59084) 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 -- 172.21.15.106:0/2664290173 learned_addr learned my addr 172.21.15.106:0/2664290173 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:52.835 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f18054de640 1 --2- 172.21.15.106:0/2664290173 >> v2:172.21.15.156:3300/0 conn(0x7f18000cce40 0x7f18000cd2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 -- 172.21.15.106:0/2664290173 >> v2:172.21.15.156:3300/0 conn(0x7f18000cce40 msgr2=0x7f18000cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:52.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 --2- 172.21.15.106:0/2664290173 >> v2:172.21.15.156:3300/0 conn(0x7f18000cce40 0x7f18000cd2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 -- 172.21.15.106:0/2664290173 >> v1:172.21.15.156:6789/0 conn(0x7f180015d310 legacy=0x7f18000ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:52.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.833+0000 7f17fffff640 1 -- 172.21.15.106:0/2664290173 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18000ae8b0 con 0x7f18000b0b40 2023-11-24T08:12:52.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.834+0000 7f17fffff640 1 --2- 172.21.15.106:0/2664290173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f18000b0f90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18000cb240 tx=0x7f17ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d9021848cd7a98 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:52.836 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.835+0000 7f17ff7fe640 1 -- 172.21.15.106:0/2664290173 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec031870 con 0x7f18000b0b40 2023-11-24T08:12:52.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.835+0000 7f17ff7fe640 1 -- 172.21.15.106:0/2664290173 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17ec0319d0 con 0x7f18000b0b40 2023-11-24T08:12:52.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.835+0000 7f17ff7fe640 1 -- 172.21.15.106:0/2664290173 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ec031d60 con 0x7f18000b0b40 2023-11-24T08:12:52.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.835+0000 7f1805cdf640 1 -- 172.21.15.106:0/2664290173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 msgr2=0x7f18000b0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.835+0000 7f1805cdf640 1 --2- 172.21.15.106:0/2664290173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f18000b0f90 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f18000cb240 tx=0x7f17ec02b910 comp rx=0 tx=0).stop 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.836+0000 7f1805cdf640 1 -- 172.21.15.106:0/2664290173 shutdown_connections 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.836+0000 7f1805cdf640 1 --2- 172.21.15.106:0/2664290173 >> v2:172.21.15.156:3300/0 conn(0x7f18000cce40 0x7f18000cd2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.836+0000 7f1805cdf640 1 --2- 172.21.15.106:0/2664290173 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f18000b0f90 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.836+0000 7f1805cdf640 1 -- 172.21.15.106:0/2664290173 >> 172.21.15.106:0/2664290173 conn(0x7f18000cbb40 msgr2=0x7f18000cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.836+0000 7f1805cdf640 1 -- 172.21.15.106:0/2664290173 shutdown_connections 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.836+0000 7f1805cdf640 1 -- 172.21.15.106:0/2664290173 wait complete. 2023-11-24T08:12:52.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.837+0000 7f1805cdf640 1 Processor -- start 2023-11-24T08:12:52.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 -- start start 2023-11-24T08:12:52.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f180009bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f18000cce40 0x7f1800155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f180015d310 0x7f18000c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180009c0d0 con 0x7f18000b0b40 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180009aaf0 con 0x7f180015d310 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.838+0000 7f1805cdf640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180009add0 con 0x7f18000cce40 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f180009bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f180009bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59090/0 (socket says 172.21.15.106:59090) 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 -- 172.21.15.106:0/1871451483 learned_addr learned my addr 172.21.15.106:0/1871451483 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 -- 172.21.15.106:0/1871451483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f18000cce40 msgr2=0x7f1800155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f18000cce40 0x7f1800155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 -- 172.21.15.106:0/1871451483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f180015d310 msgr2=0x7f18000c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:52.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f180015d310 0x7f18000c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:52.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 -- 172.21.15.106:0/1871451483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17ec0094d0 con 0x7f18000b0b40 2023-11-24T08:12:52.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.839+0000 7f1804cdd640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f180009bb90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f17f400b3f0 tx=0x7f17f400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.840+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4011830 con 0x7f18000b0b40 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.840+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17f4011990 con 0x7f18000b0b40 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.840+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180009b0b0 con 0x7f18000b0b40 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.840+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17f4011d40 con 0x7f18000b0b40 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.840+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180009b670 con 0x7f18000b0b40 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.841+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180004e840 con 0x7f18000b0b40 2023-11-24T08:12:52.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.842+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f17f401a400 con 0x7f18000b0b40 2023-11-24T08:12:52.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.843+0000 7f17fd7fa640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f17e406b3e0 0x7f17e406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:52.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.843+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f17f4089b70 con 0x7f18000b0b40 2023-11-24T08:12:52.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.843+0000 7f17fffff640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f17e406b3e0 0x7f17e406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:52.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.844+0000 7f17fffff640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f17e406b3e0 0x7f17e406d8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f17ec036000 tx=0x7f17ec030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:52.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:52.848+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f17f4054700 con 0x7f18000b0b40 2023-11-24T08:12:53.039 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.038+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f180006e870 con 0x7f18000b0b40 2023-11-24T08:12:53.040 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.039+0000 7f17fd7fa640 1 -- 172.21.15.106:0/1871451483 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f17f40150a0 con 0x7f18000b0b40 2023-11-24T08:12:53.041 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:12:53.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f17e406b3e0 msgr2=0x7f17e406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f17e406b3e0 0x7f17e406d8a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f17ec036000 tx=0x7f17ec030000 comp rx=0 tx=0).stop 2023-11-24T08:12:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 msgr2=0x7f180009bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f180009bb90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f17f400b3f0 tx=0x7f17f400b8b0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 shutdown_connections 2023-11-24T08:12:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f180015d310 0x7f18000c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f17e406b3e0 0x7f17e406d8a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.041+0000 7f1805cdf640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f18000cce40 0x7f1800155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.042+0000 7f1805cdf640 1 --2- 172.21.15.106:0/1871451483 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f18000b0b40 0x7f180009bb90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.042+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 >> 172.21.15.106:0/1871451483 conn(0x7f18000cbb40 msgr2=0x7f18000afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:53.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.042+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 shutdown_connections 2023-11-24T08:12:53.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.042+0000 7f1805cdf640 1 -- 172.21.15.106:0/1871451483 wait complete. 2023-11-24T08:12:53.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-24T08:12:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.274+0000 7f43e8a90640 1 Processor -- start 2023-11-24T08:12:53.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.274+0000 7f43e8a90640 1 -- start start 2023-11-24T08:12:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.275+0000 7f43e8a90640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e415c7f0 0x7f43e406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.275+0000 7f43e8a90640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43e40ce6c0 0x7f43e40cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.275+0000 7f43e8a90640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43e40cece0 con 0x7f43e40cf450 2023-11-24T08:12:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.275+0000 7f43e8a90640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43e4156010 con 0x7f43e40ce6c0 2023-11-24T08:12:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.275+0000 7f43e8a90640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e406ce30 con 0x7f43e415c7f0 2023-11-24T08:12:53.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43e40ce6c0 0x7f43e40cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f43e40ce6c0 0x7f43e40cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59100/0 (socket says 172.21.15.106:59100) 2023-11-24T08:12:53.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 -- 172.21.15.106:0/3285478868 learned_addr learned my addr 172.21.15.106:0/3285478868 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:53.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 -- 172.21.15.106:0/3285478868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e415c7f0 msgr2=0x7f43e406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:53.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 --2- 172.21.15.106:0/3285478868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e415c7f0 0x7f43e406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 -- 172.21.15.106:0/3285478868 >> v1:172.21.15.156:6789/0 conn(0x7f43e40cf450 legacy=0x7f43e40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.276+0000 7f43e2d9d640 1 -- 172.21.15.106:0/3285478868 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e406d040 con 0x7f43e40ce6c0 2023-11-24T08:12:53.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.277+0000 7f43e2d9d640 1 --2- 172.21.15.106:0/3285478868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e40ce6c0 0x7f43e40cf8d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009e50 tx=0x7f43dc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28ee09e9a40bd40 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e1d9b640 1 -- 172.21.15.106:0/3285478868 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc02ca70 con 0x7f43e40ce6c0 2023-11-24T08:12:53.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e1d9b640 1 -- 172.21.15.106:0/3285478868 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc02cbd0 con 0x7f43e40ce6c0 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e1d9b640 1 -- 172.21.15.106:0/3285478868 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc0343f0 con 0x7f43e40ce6c0 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 -- 172.21.15.106:0/3285478868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e40ce6c0 msgr2=0x7f43e40cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 --2- 172.21.15.106:0/3285478868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e40ce6c0 0x7f43e40cf8d0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009e50 tx=0x7f43dc02bed0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 -- 172.21.15.106:0/3285478868 shutdown_connections 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 --2- 172.21.15.106:0/3285478868 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e40ce6c0 0x7f43e40cf8d0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 --2- 172.21.15.106:0/3285478868 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e415c7f0 0x7f43e406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 -- 172.21.15.106:0/3285478868 >> 172.21.15.106:0/3285478868 conn(0x7f43e406a920 msgr2=0x7f43e406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 -- 172.21.15.106:0/3285478868 shutdown_connections 2023-11-24T08:12:53.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.278+0000 7f43e8a90640 1 -- 172.21.15.106:0/3285478868 wait complete. 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.280+0000 7f43e8a90640 1 Processor -- start 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.280+0000 7f43e8a90640 1 -- start start 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e8a90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e40ce6c0 0x7f43e41f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e8a90640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e40cf450 0x7f43e41f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e8a90640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 0x7f43e4150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e8a90640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40ad510 con 0x7f43e415c7f0 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e8a90640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43e41f4370 con 0x7f43e40ce6c0 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e8a90640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43e40ad230 con 0x7f43e40cf450 2023-11-24T08:12:53.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 0x7f43e4150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e2d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 0x7f43e4150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59112/0 (socket says 172.21.15.106:59112) 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.281+0000 7f43e2d9d640 1 -- 172.21.15.106:0/1139085228 learned_addr learned my addr 172.21.15.106:0/1139085228 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e2d9d640 1 -- 172.21.15.106:0/1139085228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e40cf450 msgr2=0x7f43e41f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e2d9d640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e40cf450 0x7f43e41f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e2d9d640 1 -- 172.21.15.106:0/1139085228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e40ce6c0 msgr2=0x7f43e41f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e2d9d640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e40ce6c0 0x7f43e41f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e2d9d640 1 -- 172.21.15.106:0/1139085228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43dc009b30 con 0x7f43e415c7f0 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e2d9d640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 0x7f43e4150e70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f43dc036040 tx=0x7f43dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc009510 con 0x7f43e415c7f0 2023-11-24T08:12:53.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43dc03e450 con 0x7f43e415c7f0 2023-11-24T08:12:53.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43e40b22b0 con 0x7f43e415c7f0 2023-11-24T08:12:53.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43dc03e800 con 0x7f43e415c7f0 2023-11-24T08:12:53.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.282+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43e40b2870 con 0x7f43e415c7f0 2023-11-24T08:12:53.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.283+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43e40b2110 con 0x7f43e415c7f0 2023-11-24T08:12:53.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.284+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f43dc03e960 con 0x7f43e415c7f0 2023-11-24T08:12:53.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.286+0000 7f43db7fe640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c406b3e0 0x7f43c406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.286+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f43dc0acb90 con 0x7f43e415c7f0 2023-11-24T08:12:53.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.286+0000 7f43e259c640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c406b3e0 0x7f43c406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.286+0000 7f43e259c640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c406b3e0 0x7f43c406d8a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009670 tx=0x7f43d4006bf0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:53.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.290+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f43e40b2110 con 0x7f43e415c7f0 2023-11-24T08:12:53.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.487+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f43e40b2110 con 0x7f43e415c7f0 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.488+0000 7f43db7fe640 1 -- 172.21.15.106:0/1139085228 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v85) v1 ==== 84+0+65 (secure 0 0 0) 0x7f43dc043060 con 0x7f43e415c7f0 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stdout: "poolnum": 1, 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stdout: "poolname": ".mgr" 2023-11-24T08:12:53.490 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:12:53.491 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:12:53.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c406b3e0 msgr2=0x7f43c406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c406b3e0 0x7f43c406d8a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009670 tx=0x7f43d4006bf0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 msgr2=0x7f43e4150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 0x7f43e4150e70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f43dc036040 tx=0x7f43dc004a40 comp rx=0 tx=0).stop 2023-11-24T08:12:53.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 shutdown_connections 2023-11-24T08:12:53.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f43e415c7f0 0x7f43e4150e70 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.493 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f43c406b3e0 0x7f43c406d8a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f43e40cf450 0x7f43e41f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 --2- 172.21.15.106:0/1139085228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43e40ce6c0 0x7f43e41f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 >> 172.21.15.106:0/1139085228 conn(0x7f43e406a920 msgr2=0x7f43e40cdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:53.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.491+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 shutdown_connections 2023-11-24T08:12:53.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.492+0000 7f43e8a90640 1 -- 172.21.15.106:0/1139085228 wait complete. 2023-11-24T08:12:53.513 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-24T08:12:53.514 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-24T08:12:53.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.724+0000 7fe32f347640 1 Processor -- start 2023-11-24T08:12:53.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.724+0000 7fe32f347640 1 -- start start 2023-11-24T08:12:53.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.725+0000 7fe32f347640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe32809a0e0 0x7fe32815cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.725+0000 7fe32f347640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3280b3a30 con 0x7fe3280b0390 2023-11-24T08:12:53.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.725+0000 7fe32f347640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32815d8b0 con 0x7fe32806b610 2023-11-24T08:12:53.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.725+0000 7fe32f347640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3280d0740 con 0x7fe32809a0e0 2023-11-24T08:12:53.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.726+0000 7fe32db44640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe32809a0e0 0x7fe32815cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.726+0000 7fe32db44640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe32809a0e0 0x7fe32815cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59120/0 (socket says 172.21.15.106:59120) 2023-11-24T08:12:53.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.726+0000 7fe32db44640 1 -- 172.21.15.106:0/1065644425 learned_addr learned my addr 172.21.15.106:0/1065644425 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:53.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.726+0000 7fe32db44640 1 -- 172.21.15.106:0/1065644425 >> v1:172.21.15.156:6789/0 conn(0x7fe32806b610 legacy=0x7fe32815d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:53.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.726+0000 7fe32db44640 1 -- 172.21.15.106:0/1065644425 >> v1:172.21.15.106:6789/0 conn(0x7fe3280b0390 legacy=0x7fe3280999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.726+0000 7fe32db44640 1 -- 172.21.15.106:0/1065644425 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3280cf6a0 con 0x7fe32809a0e0 2023-11-24T08:12:53.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.727+0000 7fe32db44640 1 --2- 172.21.15.106:0/1065644425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32809a0e0 0x7fe32815cc60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe318009870 tx=0x7fe31802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f59d9370a229f8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:53.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32d343640 1 -- 172.21.15.106:0/1065644425 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318031870 con 0x7fe32809a0e0 2023-11-24T08:12:53.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32d343640 1 -- 172.21.15.106:0/1065644425 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3180319d0 con 0x7fe32809a0e0 2023-11-24T08:12:53.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32d343640 1 -- 172.21.15.106:0/1065644425 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318031d60 con 0x7fe32809a0e0 2023-11-24T08:12:53.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 -- 172.21.15.106:0/1065644425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32809a0e0 msgr2=0x7fe32815cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 --2- 172.21.15.106:0/1065644425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32809a0e0 0x7fe32815cc60 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fe318009870 tx=0x7fe31802b910 comp rx=0 tx=0).stop 2023-11-24T08:12:53.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 -- 172.21.15.106:0/1065644425 shutdown_connections 2023-11-24T08:12:53.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 --2- 172.21.15.106:0/1065644425 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32809a0e0 0x7fe32815cc60 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 -- 172.21.15.106:0/1065644425 >> 172.21.15.106:0/1065644425 conn(0x7fe3280b2ae0 msgr2=0x7fe3280b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:53.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 -- 172.21.15.106:0/1065644425 shutdown_connections 2023-11-24T08:12:53.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.728+0000 7fe32f347640 1 -- 172.21.15.106:0/1065644425 wait complete. 2023-11-24T08:12:53.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.730+0000 7fe32f347640 1 Processor -- start 2023-11-24T08:12:53.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.730+0000 7fe32f347640 1 -- start start 2023-11-24T08:12:53.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32f347640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 0x7fe32814f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32f347640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32809a0e0 0x7fe3280c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32f347640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3280b0390 0x7fe3280c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32f347640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32814fa40 con 0x7fe32806b610 2023-11-24T08:12:53.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32f347640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281f3430 con 0x7fe3280b0390 2023-11-24T08:12:53.736 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32f347640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3281e9ab0 con 0x7fe32809a0e0 2023-11-24T08:12:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32e345640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 0x7fe32814f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32e345640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 0x7fe32814f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59130/0 (socket says 172.21.15.106:59130) 2023-11-24T08:12:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.731+0000 7fe32e345640 1 -- 172.21.15.106:0/3310158576 learned_addr learned my addr 172.21.15.106:0/3310158576 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32e345640 1 -- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32809a0e0 msgr2=0x7fe3280c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.737 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32db44640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32809a0e0 0x7fe3280c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32eb46640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3280b0390 0x7fe3280c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32e345640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32809a0e0 0x7fe3280c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32e345640 1 -- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3280b0390 msgr2=0x7fe3280c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:53.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32e345640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3280b0390 0x7fe3280c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:53.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32e345640 1 -- 172.21.15.106:0/3310158576 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3180094d0 con 0x7fe32806b610 2023-11-24T08:12:53.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32eb46640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3280b0390 0x7fe3280c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:53.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe32e345640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 0x7fe32814f500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c450 tx=0x7fe32400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:53.739 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.732+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3240052d0 con 0x7fe32806b610 2023-11-24T08:12:53.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.733+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe324005430 con 0x7fe32806b610 2023-11-24T08:12:53.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.733+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3281e9210 con 0x7fe32806b610 2023-11-24T08:12:53.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.733+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe324012870 con 0x7fe32806b610 2023-11-24T08:12:53.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.733+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3281e97d0 con 0x7fe32806b610 2023-11-24T08:12:53.740 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.734+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe328067990 con 0x7fe32806b610 2023-11-24T08:12:53.741 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.735+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe3240129d0 con 0x7fe32806b610 2023-11-24T08:12:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.735+0000 7fe31effd640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe30c06b3e0 0x7fe30c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.736+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe3240898b0 con 0x7fe32806b610 2023-11-24T08:12:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.736+0000 7fe32db44640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe30c06b3e0 0x7fe30c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:53.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.737+0000 7fe32db44640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe30c06b3e0 0x7fe30c06d8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe318036000 tx=0x7fe31803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:53.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.740+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe324054400 con 0x7fe32806b610 2023-11-24T08:12:53.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:53.931+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe32804e840 con 0x7fe32806b610 2023-11-24T08:12:54.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.690+0000 7fe31effd640 1 -- 172.21.15.106:0/3310158576 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v86) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe324057a20 con 0x7fe32806b610 2023-11-24T08:12:54.692 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' created 2023-11-24T08:12:54.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe30c06b3e0 msgr2=0x7fe30c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe30c06b3e0 0x7fe30c06d8a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe318036000 tx=0x7fe31803c040 comp rx=0 tx=0).stop 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 msgr2=0x7fe32814f500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 0x7fe32814f500 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fe32400c450 tx=0x7fe32400c910 comp rx=0 tx=0).stop 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 shutdown_connections 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3280b0390 0x7fe3280c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe30c06b3e0 0x7fe30c06d8a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.697 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe32809a0e0 0x7fe3280c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 --2- 172.21.15.106:0/3310158576 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe32806b610 0x7fe32814f500 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 >> 172.21.15.106:0/3310158576 conn(0x7fe3280b2ae0 msgr2=0x7fe32815ce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:54.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 shutdown_connections 2023-11-24T08:12:54.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.695+0000 7fe32f347640 1 -- 172.21.15.106:0/3310158576 wait complete. 2023-11-24T08:12:54.725 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-24T08:12:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.929+0000 7f5e5874c640 1 Processor -- start 2023-11-24T08:12:54.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.930+0000 7f5e5874c640 1 -- start start 2023-11-24T08:12:54.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.931+0000 7f5e5874c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e501660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.931+0000 7f5e5874c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5e500c9650 0x7f5e500c9ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.931+0000 7f5e5874c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e501655f0 con 0x7f5e500ccfb0 2023-11-24T08:12:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.931+0000 7f5e5874c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e500cc380 con 0x7f5e500c9650 2023-11-24T08:12:54.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.931+0000 7f5e5874c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50162b30 con 0x7f5e500ccbd0 2023-11-24T08:12:54.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.932+0000 7f5e55ce7640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f5e500ccfb0 0x7f5e50162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37258/0 (socket says 172.21.15.106:37258) 2023-11-24T08:12:54.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.932+0000 7f5e55ce7640 1 -- 172.21.15.106:0/3964044417 learned_addr learned my addr 172.21.15.106:0/3964044417 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:54.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.932+0000 7f5e564e8640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e501660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.932+0000 7f5e56ce9640 1 --2- 172.21.15.106:0/3964044417 >> v2:172.21.15.156:3300/0 conn(0x7f5e500c9650 0x7f5e500c9ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.932+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1367865078 0 0) 0x7f5e501655f0 con 0x7f5e500ccfb0 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.932+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e38003520 con 0x7f5e500ccfb0 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3374303734 0 0) 0x7f5e38003520 con 0x7f5e500ccfb0 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 msgr2=0x7f5e501660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e501660b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 >> v2:172.21.15.156:3300/0 conn(0x7f5e500c9650 msgr2=0x7f5e500c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 --2- 172.21.15.106:0/3964044417 >> v2:172.21.15.156:3300/0 conn(0x7f5e500c9650 0x7f5e500c9ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50162e10 con 0x7f5e500ccfb0 2023-11-24T08:12:54.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f5e44001670 con 0x7f5e500ccfb0 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 >> v1:172.21.15.106:6789/0 conn(0x7f5e500ccfb0 legacy=0x7f5e50162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38003970 0x7f5e38003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e380047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e38005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500cc380 con 0x7f5e38004d00 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e50162b30 con 0x7f5e38003970 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3800b980 con 0x7f5e500ccbd0 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e56ce9640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e38005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e564e8640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38003970 0x7f5e38003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e55ce7640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e380047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e56ce9640 1 -- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 msgr2=0x7f5e380047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.933+0000 7f5e56ce9640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e380047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.934+0000 7f5e56ce9640 1 -- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38003970 msgr2=0x7f5e38003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.934+0000 7f5e56ce9640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38003970 0x7f5e38003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.934+0000 7f5e56ce9640 1 -- 172.21.15.106:0/3964044417 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38003520 con 0x7f5e38004d00 2023-11-24T08:12:54.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.934+0000 7f5e55ce7640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e500ccbd0 0x7f5e380047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:54.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e56ce9640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e38005180 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009fc0 tx=0x7f5e380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535c7dbded15ef07 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4802cc30 con 0x7f5e38004d00 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4800f040 con 0x7f5e38004d00 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e554e6640 1 -- 172.21.15.106:0/3964044417 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e48034440 con 0x7f5e38004d00 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e5874c640 1 -- 172.21.15.106:0/3964044417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 msgr2=0x7f5e38005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e5874c640 1 --2- 172.21.15.106:0/3964044417 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e38005180 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f5e48009fc0 tx=0x7f5e380057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.935+0000 7f5e56ce9640 1 -- 172.21.15.106:0/3964044417 reap_dead start 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.936+0000 7f5e5874c640 1 -- 172.21.15.106:0/3964044417 shutdown_connections 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.936+0000 7f5e5874c640 1 -- 172.21.15.106:0/3964044417 >> 172.21.15.106:0/3964044417 conn(0x7f5e500b0c50 msgr2=0x7f5e500b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.936+0000 7f5e5874c640 1 -- 172.21.15.106:0/3964044417 shutdown_connections 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.936+0000 7f5e5874c640 1 -- 172.21.15.106:0/3964044417 wait complete. 2023-11-24T08:12:54.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.937+0000 7f5e5874c640 1 Processor -- start 2023-11-24T08:12:54.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.938+0000 7f5e5874c640 1 -- start start 2023-11-24T08:12:54.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e5874c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e50163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e5874c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e5009c640 0x7f5e500c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e5874c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e500acf80 0x7f5e500ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e5874c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5009cac0 con 0x7f5e38004d00 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e5874c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e500ad940 con 0x7f5e500acf80 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e5874c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e501636a0 con 0x7f5e5009c640 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e50163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e50163050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38512/0 (socket says 172.21.15.106:38512) 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 -- 172.21.15.106:0/1745952517 learned_addr learned my addr 172.21.15.106:0/1745952517 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 -- 172.21.15.106:0/1745952517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e5009c640 msgr2=0x7f5e500c8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e5009c640 0x7f5e500c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 -- 172.21.15.106:0/1745952517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e500acf80 msgr2=0x7f5e500ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:54.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e500acf80 0x7f5e500ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.939+0000 7f5e564e8640 1 -- 172.21.15.106:0/1745952517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e50163980 con 0x7f5e38004d00 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.940+0000 7f5e564e8640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e50163050 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5e40011600 tx=0x7f5e40011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.940+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40015870 con 0x7f5e38004d00 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.940+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e400159d0 con 0x7f5e38004d00 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.940+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e40015d40 con 0x7f5e38004d00 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.940+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e50163c60 con 0x7f5e38004d00 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.940+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e500c8790 con 0x7f5e38004d00 2023-11-24T08:12:54.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.941+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5004e8b0 con 0x7f5e38004d00 2023-11-24T08:12:54.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.943+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5e40013430 con 0x7f5e38004d00 2023-11-24T08:12:54.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.943+0000 7f5e4effd640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5e3c06b410 0x7f5e3c06d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:54.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.943+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5e4008a520 con 0x7f5e38004d00 2023-11-24T08:12:54.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.944+0000 7f5e55ce7640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5e3c06b410 0x7f5e3c06d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:54.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.944+0000 7f5e55ce7640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5e3c06b410 0x7f5e3c06d8d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5e440017a0 tx=0x7f5e44009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:54.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:54.947+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5e40055180 con 0x7f5e38004d00 2023-11-24T08:12:55.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.142+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5e5009a4e0 con 0x7f5e38004d00 2023-11-24T08:12:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.705+0000 7f5e4effd640 1 -- 172.21.15.106:0/1745952517 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v87) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5e400587a0 con 0x7f5e38004d00 2023-11-24T08:12:55.707 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' created 2023-11-24T08:12:55.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5e3c06b410 msgr2=0x7f5e3c06d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:55.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5e3c06b410 0x7f5e3c06d8d0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5e440017a0 tx=0x7f5e44009040 comp rx=0 tx=0).stop 2023-11-24T08:12:55.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 msgr2=0x7f5e50163050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:55.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e50163050 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f5e40011600 tx=0x7f5e40011ac0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 shutdown_connections 2023-11-24T08:12:55.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e500acf80 0x7f5e500ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5e3c06b410 0x7f5e3c06d8d0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5e5009c640 0x7f5e500c8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 --2- 172.21.15.106:0/1745952517 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5e38004d00 0x7f5e50163050 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.709+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 >> 172.21.15.106:0/1745952517 conn(0x7f5e500b0c50 msgr2=0x7f5e500b0160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:55.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.710+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 shutdown_connections 2023-11-24T08:12:55.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.710+0000 7f5e5874c640 1 -- 172.21.15.106:0/1745952517 wait complete. 2023-11-24T08:12:55.744 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-24T08:12:55.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.929+0000 7f44ef761640 1 Processor -- start 2023-11-24T08:12:55.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.929+0000 7f44ef761640 1 -- start start 2023-11-24T08:12:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.930+0000 7f44ef761640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44e806de10 0x7f44e815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.930+0000 7f44ef761640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f44e80c49f0 0x7f44e80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.930+0000 7f44ef761640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44e815ec90 con 0x7f44e8072160 2023-11-24T08:12:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.930+0000 7f44ef761640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80cd530 con 0x7f44e806de10 2023-11-24T08:12:55.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.930+0000 7f44ef761640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44e80c4ed0 con 0x7f44e80c49f0 2023-11-24T08:12:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44e806de10 0x7f44e815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f44e806de10 0x7f44e815e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38528/0 (socket says 172.21.15.106:38528) 2023-11-24T08:12:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 -- 172.21.15.106:0/2191272236 learned_addr learned my addr 172.21.15.106:0/2191272236 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:55.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44edcfe640 1 --2- 172.21.15.106:0/2191272236 >> v2:172.21.15.156:3300/0 conn(0x7f44e80c49f0 0x7f44e80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:55.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 -- 172.21.15.106:0/2191272236 >> v2:172.21.15.156:3300/0 conn(0x7f44e80c49f0 msgr2=0x7f44e80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:55.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 --2- 172.21.15.106:0/2191272236 >> v2:172.21.15.156:3300/0 conn(0x7f44e80c49f0 0x7f44e80705c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 -- 172.21.15.106:0/2191272236 >> v1:172.21.15.156:6789/0 conn(0x7f44e8072160 legacy=0x7f44e806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:55.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.931+0000 7f44eccfc640 1 -- 172.21.15.106:0/2191272236 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e80c51b0 con 0x7f44e806de10 2023-11-24T08:12:55.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.932+0000 7f44eccfc640 1 --2- 172.21.15.106:0/2191272236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e806de10 0x7f44e815e750 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f44e8072a30 tx=0x7f44d802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2667ed144affe5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44dffff640 1 -- 172.21.15.106:0/2191272236 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8031890 con 0x7f44e806de10 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44dffff640 1 -- 172.21.15.106:0/2191272236 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d80319f0 con 0x7f44e806de10 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44dffff640 1 -- 172.21.15.106:0/2191272236 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d8031d80 con 0x7f44e806de10 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 -- 172.21.15.106:0/2191272236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e806de10 msgr2=0x7f44e815e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 --2- 172.21.15.106:0/2191272236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e806de10 0x7f44e815e750 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f44e8072a30 tx=0x7f44d802b930 comp rx=0 tx=0).stop 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 -- 172.21.15.106:0/2191272236 shutdown_connections 2023-11-24T08:12:55.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 --2- 172.21.15.106:0/2191272236 >> v2:172.21.15.156:3300/0 conn(0x7f44e80c49f0 0x7f44e80705c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 --2- 172.21.15.106:0/2191272236 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e806de10 0x7f44e815e750 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 -- 172.21.15.106:0/2191272236 >> 172.21.15.106:0/2191272236 conn(0x7f44e80b0c50 msgr2=0x7f44e80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 -- 172.21.15.106:0/2191272236 shutdown_connections 2023-11-24T08:12:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.933+0000 7f44ef761640 1 -- 172.21.15.106:0/2191272236 wait complete. 2023-11-24T08:12:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.935+0000 7f44ef761640 1 Processor -- start 2023-11-24T08:12:55.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.935+0000 7f44ef761640 1 -- start start 2023-11-24T08:12:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44ef761640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44e806de10 0x7f44e815b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:55.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44ef761640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e8072160 0x7f44e81585c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44ef761640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 0x7f44e809b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44ef761640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e815be60 con 0x7f44e80c49f0 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44ef761640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e815b490 con 0x7f44e8072160 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44ef761640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44e8158b00 con 0x7f44e806de10 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.936+0000 7f44edcfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 0x7f44e809b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 0x7f44e809b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38532/0 (socket says 172.21.15.106:38532) 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 -- 172.21.15.106:0/690064622 learned_addr learned my addr 172.21.15.106:0/690064622 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44ed4fd640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44e806de10 0x7f44e815b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44eccfc640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e8072160 0x7f44e81585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 -- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44e806de10 msgr2=0x7f44e815b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:55.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44e806de10 0x7f44e815b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 -- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e8072160 msgr2=0x7f44e81585c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:55.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e8072160 0x7f44e81585c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:55.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 -- 172.21.15.106:0/690064622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d80094d0 con 0x7f44e80c49f0 2023-11-24T08:12:55.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44eccfc640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e8072160 0x7f44e81585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:55.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.937+0000 7f44edcfe640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 0x7f44e809b7b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c5f0 tx=0x7f44e400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.938+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4004220 con 0x7f44e80c49f0 2023-11-24T08:12:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.938+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e4004380 con 0x7f44e80c49f0 2023-11-24T08:12:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.938+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e4003850 con 0x7f44e80c49f0 2023-11-24T08:12:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.938+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e815ae80 con 0x7f44e80c49f0 2023-11-24T08:12:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.938+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e815d180 con 0x7f44e80c49f0 2023-11-24T08:12:55.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.939+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e815b640 con 0x7f44e80c49f0 2023-11-24T08:12:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.940+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f44e40039b0 con 0x7f44e80c49f0 2023-11-24T08:12:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.941+0000 7f44ddffb640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f44cc06b3e0 0x7f44cc06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.941+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f44e4089df0 con 0x7f44e80c49f0 2023-11-24T08:12:55.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.941+0000 7f44ed4fd640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f44cc06b3e0 0x7f44cc06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:55.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.942+0000 7f44ed4fd640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f44cc06b3e0 0x7f44cc06d8a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44e0001db0 tx=0x7f44e0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:55.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:55.945+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f44e815b640 con 0x7f44e80c49f0 2023-11-24T08:12:56.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.171+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f44e815b640 con 0x7f44e80c49f0 2023-11-24T08:12:56.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.710+0000 7f44ddffb640 1 -- 172.21.15.106:0/690064622 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v49) v1 ==== 174+0+0 (secure 0 0 0) 0x7f44e815b640 con 0x7f44e80c49f0 2023-11-24T08:12:56.711 INFO:teuthology.orchestra.run.smithi106.stderr:new fs with metadata pool 12 and data pool 13 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f44cc06b3e0 msgr2=0x7f44cc06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f44cc06b3e0 0x7f44cc06d8a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f44e0001db0 tx=0x7f44e0009040 comp rx=0 tx=0).stop 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 msgr2=0x7f44e809b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 0x7f44e809b7b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f44e400c5f0 tx=0x7f44e400cab0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 shutdown_connections 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f44e80c49f0 0x7f44e809b7b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f44e8072160 0x7f44e81585c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f44cc06b3e0 0x7f44cc06d8a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 --2- 172.21.15.106:0/690064622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f44e806de10 0x7f44e815b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.713+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 >> 172.21.15.106:0/690064622 conn(0x7f44e80b0c50 msgr2=0x7f44e80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:56.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.714+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 shutdown_connections 2023-11-24T08:12:56.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.714+0000 7f44ef761640 1 -- 172.21.15.106:0/690064622 wait complete. 2023-11-24T08:12:56.757 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.935+0000 7f817ae9b640 1 Processor -- start 2023-11-24T08:12:56.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.935+0000 7f817ae9b640 1 -- start start 2023-11-24T08:12:56.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.936+0000 7f817ae9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f8174161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.936+0000 7f817ae9b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f817415e390 0x7f81740cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.936+0000 7f817ae9b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81740ce0b0 con 0x7f81740cf450 2023-11-24T08:12:56.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.936+0000 7f817ae9b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81740cfe10 con 0x7f817415e390 2023-11-24T08:12:56.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f817ae9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8174161960 con 0x7f8174157c70 2023-11-24T08:12:56.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f816bfff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f81740cf450 0x7f817415dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37268/0 (socket says 172.21.15.106:37268) 2023-11-24T08:12:56.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f816bfff640 1 -- 172.21.15.106:0/1436799769 learned_addr learned my addr 172.21.15.106:0/1436799769 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962763349 0 0) 0x7f81740ce0b0 con 0x7f81740cf450 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f8179438640 1 --2- 172.21.15.106:0/1436799769 >> v2:172.21.15.156:3300/0 conn(0x7f817415e390 0x7f81740cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f815c003520 con 0x7f81740cf450 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.937+0000 7f8178c37640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f8174161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 686237506 0 0) 0x7f815c003520 con 0x7f81740cf450 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 msgr2=0x7f8174161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f8174161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 >> v2:172.21.15.156:3300/0 conn(0x7f817415e390 msgr2=0x7f81740cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 --2- 172.21.15.106:0/1436799769 >> v2:172.21.15.156:3300/0 conn(0x7f817415e390 0x7f81740cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f817415e8d0 con 0x7f81740cf450 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f8164001670 con 0x7f81740cf450 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 >> v1:172.21.15.106:6789/0 conn(0x7f81740cf450 legacy=0x7f817415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f815c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f815c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c004d00 0x7f815c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81740cfe10 con 0x7f815c003fa0 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174161960 con 0x7f815c004d00 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f815c00bb40 con 0x7f8174157c70 2023-11-24T08:12:56.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f816bfff640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f815c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.938+0000 7f8179438640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c004d00 0x7f815c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f8178c37640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f815c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f816bfff640 1 -- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 msgr2=0x7f815c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f816bfff640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f815c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f816bfff640 1 -- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c004d00 msgr2=0x7f815c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f816bfff640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815c004d00 0x7f815c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f816bfff640 1 -- 172.21.15.106:0/1436799769 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c003520 con 0x7f815c003fa0 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.939+0000 7f8178c37640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8174157c70 0x7f815c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:56.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.940+0000 7f816bfff640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f815c0047c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f816400c240 tx=0x7f815c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fb11ee02c2ab1d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:56.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.940+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f816402cb00 con 0x7f815c003fa0 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.940+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f816402cc60 con 0x7f815c003fa0 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.940+0000 7f816b7fe640 1 -- 172.21.15.106:0/1436799769 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8164005af0 con 0x7f815c003fa0 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.940+0000 7f817ae9b640 1 -- 172.21.15.106:0/1436799769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 msgr2=0x7f815c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.940+0000 7f817ae9b640 1 --2- 172.21.15.106:0/1436799769 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f815c0047c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f816400c240 tx=0x7f815c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.941+0000 7f8179438640 1 -- 172.21.15.106:0/1436799769 reap_dead start 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.941+0000 7f817ae9b640 1 -- 172.21.15.106:0/1436799769 shutdown_connections 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.941+0000 7f817ae9b640 1 -- 172.21.15.106:0/1436799769 >> 172.21.15.106:0/1436799769 conn(0x7f817406a920 msgr2=0x7f817406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.941+0000 7f817ae9b640 1 -- 172.21.15.106:0/1436799769 shutdown_connections 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.941+0000 7f817ae9b640 1 -- 172.21.15.106:0/1436799769 wait complete. 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.942+0000 7f817ae9b640 1 Processor -- start 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 -- start start 2023-11-24T08:12:56.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f8174147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81740af9b0 0x7f817414c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81741464f0 0x7f81740cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174146e60 con 0x7f815c003fa0 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8174147140 con 0x7f81740af9b0 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.943+0000 7f817ae9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8174147420 con 0x7f81741464f0 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f8174147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f8174147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38548/0 (socket says 172.21.15.106:38548) 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 -- 172.21.15.106:0/3975770449 learned_addr learned my addr 172.21.15.106:0/3975770449 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 -- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81741464f0 msgr2=0x7f81740cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81741464f0 0x7f81740cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 -- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81740af9b0 msgr2=0x7f817414c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:56.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f816bfff640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81740af9b0 0x7f817414c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81740af9b0 0x7f817414c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:56.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 -- 172.21.15.106:0/3975770449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174147700 con 0x7f815c003fa0 2023-11-24T08:12:56.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.944+0000 7f8178c37640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f8174147f00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8170009fd0 tx=0x7f817000f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.945+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81700145f0 con 0x7f815c003fa0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.945+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8170014750 con 0x7f815c003fa0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.945+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81741479e0 con 0x7f815c003fa0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.945+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170014ae0 con 0x7f815c003fa0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.945+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8174146970 con 0x7f815c003fa0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.946+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8174146c20 con 0x7f815c003fa0 2023-11-24T08:12:56.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.947+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8170014c40 con 0x7f815c003fa0 2023-11-24T08:12:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.948+0000 7f81697fa640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f816006b3e0 0x7f816006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.948+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f817008aed0 con 0x7f815c003fa0 2023-11-24T08:12:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.948+0000 7f816bfff640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f816006b3e0 0x7f816006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:56.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.949+0000 7f816bfff640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f816006b3e0 0x7f816006d8a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8164034000 tx=0x7f8164007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:56.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:56.953+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8174146c20 con 0x7f815c003fa0 2023-11-24T08:12:57.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.138+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8174146c20 con 0x7f815c003fa0 2023-11-24T08:12:57.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.141+0000 7f81697fa640 1 -- 172.21.15.106:0/3975770449 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f8174146c20 con 0x7f815c003fa0 2023-11-24T08:12:57.143 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:57.143 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":88,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:56.173410+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f816006b3e0 msgr2=0x7f816006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f816006b3e0 0x7f816006d8a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8164034000 tx=0x7f8164007000 comp rx=0 tx=0).stop 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 msgr2=0x7f8174147f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f8174147f00 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f8170009fd0 tx=0x7f817000f5e0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 shutdown_connections 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f816006b3e0 0x7f816006d8a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81741464f0 0x7f81740cb3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81740af9b0 0x7f817414c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.145 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 --2- 172.21.15.106:0/3975770449 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f815c003fa0 0x7f8174147f00 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 >> 172.21.15.106:0/3975770449 conn(0x7f817406a920 msgr2=0x7f81740b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:57.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 shutdown_connections 2023-11-24T08:12:57.146 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.143+0000 7f817ae9b640 1 -- 172.21.15.106:0/3975770449 wait complete. 2023-11-24T08:12:57.161 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:12:57.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.364+0000 7f980bfff640 1 Processor -- start 2023-11-24T08:12:57.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.364+0000 7f980bfff640 1 -- start start 2023-11-24T08:12:57.367 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.366+0000 7f980bfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 0x7f980c15f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.366+0000 7f980bfff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c15bf80 con 0x7f980c15c410 2023-11-24T08:12:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.366+0000 7f980bfff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c0b1560 con 0x7f980c0b1de0 2023-11-24T08:12:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.366+0000 7f980bfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0b1280 con 0x7f980c144b30 2023-11-24T08:12:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.366+0000 7f980a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f980c15c410 0x7f980c0b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37274/0 (socket says 172.21.15.106:37274) 2023-11-24T08:12:57.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.366+0000 7f980a7fc640 1 -- 172.21.15.106:0/1554907445 learned_addr learned my addr 172.21.15.106:0/1554907445 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:57.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f980affd640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 0x7f980c15f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636619507 0 0) 0x7f980c15bf80 con 0x7f980c15c410 2023-11-24T08:12:57.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f4003520 con 0x7f980c15c410 2023-11-24T08:12:57.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3738706170 0 0) 0x7f97f4003520 con 0x7f980c15c410 2023-11-24T08:12:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 msgr2=0x7f980c15f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 0x7f980c15f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 >> v1:172.21.15.156:6789/0 conn(0x7f980c0b1de0 legacy=0x7f980c0b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c15b930 con 0x7f980c15c410 2023-11-24T08:12:57.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f9800001670 con 0x7f980c15c410 2023-11-24T08:12:57.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 >> v1:172.21.15.106:6789/0 conn(0x7f980c15c410 legacy=0x7f980c0b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97f40039a0 0x7f97f4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97f40047c0 0x7f97f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 0x7f97f4005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0b1280 con 0x7f97f40039a0 2023-11-24T08:12:57.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f400bb10 con 0x7f97f40047c0 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f400bcd0 con 0x7f980c144b30 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.367+0000 7f980affd640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97f40039a0 0x7f97f4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.368+0000 7f980affd640 1 -- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 msgr2=0x7f97f4005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.368+0000 7f980affd640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f980c144b30 0x7f97f4005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.368+0000 7f980affd640 1 -- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97f40047c0 msgr2=0x7f97f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.368+0000 7f980affd640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97f40047c0 0x7f97f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.368+0000 7f980affd640 1 -- 172.21.15.106:0/1554907445 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f4003520 con 0x7f97f40039a0 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.369+0000 7f980affd640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97f40039a0 0x7f97f4003dc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00ba30 tx=0x7f97f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9373ce85e22aa787 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.369+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc032680 con 0x7f97f40039a0 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.369+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc0327e0 con 0x7f97f40039a0 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.369+0000 7f9809ffb640 1 -- 172.21.15.106:0/1554907445 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc032b70 con 0x7f97f40039a0 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.369+0000 7f980bfff640 1 -- 172.21.15.106:0/1554907445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97f40039a0 msgr2=0x7f97f4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.369+0000 7f980bfff640 1 --2- 172.21.15.106:0/1554907445 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f97f40039a0 0x7f97f4003dc0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f97fc00ba30 tx=0x7f97f40057f0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.370+0000 7f980b7fe640 1 -- 172.21.15.106:0/1554907445 reap_dead start 2023-11-24T08:12:57.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.370+0000 7f980bfff640 1 -- 172.21.15.106:0/1554907445 shutdown_connections 2023-11-24T08:12:57.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.370+0000 7f980bfff640 1 -- 172.21.15.106:0/1554907445 >> 172.21.15.106:0/1554907445 conn(0x7f980c0cb830 msgr2=0x7f980c0cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:57.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.370+0000 7f980bfff640 1 -- 172.21.15.106:0/1554907445 shutdown_connections 2023-11-24T08:12:57.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.370+0000 7f980bfff640 1 -- 172.21.15.106:0/1554907445 wait complete. 2023-11-24T08:12:57.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.371+0000 7f980bfff640 1 Processor -- start 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 -- start start 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97f40039a0 0x7f980c070cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f980c15e320 0x7f980c15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 0x7f980c0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c13c690 con 0x7f980c0ca1a0 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0cab60 con 0x7f980c15e320 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.372+0000 7f980bfff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c0714e0 con 0x7f97f40039a0 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 0x7f980c0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 0x7f980c0ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38572/0 (socket says 172.21.15.106:38572) 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 -- 172.21.15.106:0/2495876381 learned_addr learned my addr 172.21.15.106:0/2495876381 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:57.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980a7fc640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f980c15e320 0x7f980c15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980affd640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97f40039a0 0x7f980c070cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 -- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97f40039a0 msgr2=0x7f980c070cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97f40039a0 0x7f980c070cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 -- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f980c15e320 msgr2=0x7f980c15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f980c15e320 0x7f980c15ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 -- 172.21.15.106:0/2495876381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c0717c0 con 0x7f980c0ca1a0 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.373+0000 7f980b7fe640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 0x7f980c0ca620 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f980c0cc7a0 tx=0x7f980400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.374+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9804005d40 con 0x7f980c0ca1a0 2023-11-24T08:12:57.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.374+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980400ced0 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.374+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980400a850 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.374+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c071aa0 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.374+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c15b190 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.375+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c15ca60 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.376+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f980400a9b0 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.377+0000 7f97f37fe640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f97f806b3e0 0x7f97f806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.377+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9804089150 con 0x7f980c0ca1a0 2023-11-24T08:12:57.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.377+0000 7f980affd640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f97f806b3e0 0x7f97f806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.378+0000 7f980affd640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f97f806b3e0 0x7f97f806d8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f97fc037000 tx=0x7f97fc030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:57.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.381+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f98040bbe90 con 0x7f980c0ca1a0 2023-11-24T08:12:57.563 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.561+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f980c04e840 con 0x7f980c0ca1a0 2023-11-24T08:12:57.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.563+0000 7f97f37fe640 1 -- 172.21.15.106:0/2495876381 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+16621 (secure 0 0 0) 0x7f98040539e0 con 0x7f980c0ca1a0 2023-11-24T08:12:57.565 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:12:57.565 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":88,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:56.173410+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f97f806b3e0 msgr2=0x7f97f806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f97f806b3e0 0x7f97f806d8a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f97fc037000 tx=0x7f97fc030000 comp rx=0 tx=0).stop 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 msgr2=0x7f980c0ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 0x7f980c0ca620 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f980c0cc7a0 tx=0x7f980400cb90 comp rx=0 tx=0).stop 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 shutdown_connections 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f980c0ca1a0 0x7f980c0ca620 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f980c15e320 0x7f980c15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f97f806b3e0 0x7f97f806d8a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 --2- 172.21.15.106:0/2495876381 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f97f40039a0 0x7f980c070cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 >> 172.21.15.106:0/2495876381 conn(0x7f980c0cb830 msgr2=0x7f980c0b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:57.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 shutdown_connections 2023-11-24T08:12:57.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.566+0000 7f980bfff640 1 -- 172.21.15.106:0/2495876381 wait complete. 2023-11-24T08:12:57.585 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-24T08:12:57.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.800+0000 7ffab21c5640 1 Processor -- start 2023-11-24T08:12:57.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.800+0000 7ffab21c5640 1 -- start start 2023-11-24T08:12:57.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffab21c5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ffaac0ccbd0 0x7ffaac1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffab21c5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac1655f0 con 0x7ffaac0ccfb0 2023-11-24T08:12:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffab21c5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaac0cc380 con 0x7ffaac0c9650 2023-11-24T08:12:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffab21c5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaac162b30 con 0x7ffaac0ccbd0 2023-11-24T08:12:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffaaaffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ffaac0ccfb0 0x7ffaac162000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37280/0 (socket says 172.21.15.106:37280) 2023-11-24T08:12:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffaaaffd640 1 -- 172.21.15.106:0/2696860937 learned_addr learned my addr 172.21.15.106:0/2696860937 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:12:57.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.802+0000 7ffaab7fe640 1 --2- 172.21.15.106:0/2696860937 >> v2:172.21.15.106:3300/0 conn(0x7ffaac0ccbd0 0x7ffaac1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.803+0000 7ffaaa7fc640 1 -- 172.21.15.106:0/2696860937 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 10319767 0 0) 0x7ffaac1655f0 con 0x7ffaac0ccfb0 2023-11-24T08:12:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.803+0000 7ffaaa7fc640 1 -- 172.21.15.106:0/2696860937 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa94003520 con 0x7ffaac0ccfb0 2023-11-24T08:12:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.803+0000 7ffaab7fe640 1 -- 172.21.15.106:0/2696860937 >> v1:172.21.15.156:6789/0 conn(0x7ffaac0c9650 legacy=0x7ffaac0c9ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.803+0000 7ffaab7fe640 1 -- 172.21.15.106:0/2696860937 >> v1:172.21.15.106:6789/0 conn(0x7ffaac0ccfb0 legacy=0x7ffaac162000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.803+0000 7ffaab7fe640 1 -- 172.21.15.106:0/2696860937 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac162e10 con 0x7ffaac0ccbd0 2023-11-24T08:12:57.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.804+0000 7ffaab7fe640 1 --2- 172.21.15.106:0/2696860937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac1660b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097a0 tx=0x7ffa9c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d18a1d403efa95f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:57.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.804+0000 7ffaaa7fc640 1 -- 172.21.15.106:0/2696860937 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c030610 con 0x7ffaac0ccbd0 2023-11-24T08:12:57.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.804+0000 7ffaaa7fc640 1 -- 172.21.15.106:0/2696860937 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c030770 con 0x7ffaac0ccbd0 2023-11-24T08:12:57.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.804+0000 7ffaaa7fc640 1 -- 172.21.15.106:0/2696860937 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c030b00 con 0x7ffaac0ccbd0 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 -- 172.21.15.106:0/2696860937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0ccbd0 msgr2=0x7ffaac1660b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 --2- 172.21.15.106:0/2696860937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac1660b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0097a0 tx=0x7ffa9c02b960 comp rx=0 tx=0).stop 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 -- 172.21.15.106:0/2696860937 shutdown_connections 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 --2- 172.21.15.106:0/2696860937 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac1660b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 -- 172.21.15.106:0/2696860937 >> 172.21.15.106:0/2696860937 conn(0x7ffaac0b0c50 msgr2=0x7ffaac0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 -- 172.21.15.106:0/2696860937 shutdown_connections 2023-11-24T08:12:57.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.805+0000 7ffab21c5640 1 -- 172.21.15.106:0/2696860937 wait complete. 2023-11-24T08:12:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.807+0000 7ffab21c5640 1 Processor -- start 2023-11-24T08:12:57.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.807+0000 7ffab21c5640 1 -- start start 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffab21c5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 0x7ffaac15bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffab21c5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac15e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffab21c5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaac0ccfb0 0x7ffaac158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffab21c5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac164c10 con 0x7ffaac0c9650 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffab21c5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac070d20 con 0x7ffaac0ccbd0 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffab21c5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaac164930 con 0x7ffaac0ccfb0 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 0x7ffaac15bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 0x7ffaac15bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38590/0 (socket says 172.21.15.106:38590) 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffaab7fe640 1 -- 172.21.15.106:0/1515851940 learned_addr learned my addr 172.21.15.106:0/1515851940 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffaaaffd640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac15e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.808+0000 7ffaab7fe640 1 -- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaac0ccfb0 msgr2=0x7ffaac158fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffaab7fe640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaac0ccfb0 0x7ffaac158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffaab7fe640 1 -- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac0ccbd0 msgr2=0x7ffaac15e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffaab7fe640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac15e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffaab7fe640 1 -- 172.21.15.106:0/1515851940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa9c009400 con 0x7ffaac0c9650 2023-11-24T08:12:57.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffaaaffd640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:12:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffaab7fe640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 0x7ffaac15bb00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c02bf70 tx=0x7ffa9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c03e070 con 0x7ffaac0c9650 2023-11-24T08:12:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c030e10 con 0x7ffaac0c9650 2023-11-24T08:12:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c039c30 con 0x7ffaac0c9650 2023-11-24T08:12:57.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaac070660 con 0x7ffaac0c9650 2023-11-24T08:12:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.809+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaac15ee00 con 0x7ffaac0c9650 2023-11-24T08:12:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.811+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaac0704c0 con 0x7ffaac0c9650 2023-11-24T08:12:57.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.811+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7ffa9c0423f0 con 0x7ffaac0c9650 2023-11-24T08:12:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.813+0000 7ffa93fff640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa9806b3e0 0x7ffa9806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.813+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffa9c0abc20 con 0x7ffaac0c9650 2023-11-24T08:12:57.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.813+0000 7ffaaaffd640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa9806b3e0 0x7ffa9806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:57.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.814+0000 7ffaaaffd640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa9806b3e0 0x7ffa9806d8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0b0a10 tx=0x7ffaa0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:57.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:57.817+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7ffaac0704c0 con 0x7ffaac0c9650 2023-11-24T08:12:58.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.043+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ffaac0704c0 con 0x7ffaac0c9650 2023-11-24T08:12:58.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.752+0000 7ffa93fff640 1 -- 172.21.15.106:0/1515851940 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v52) v1 ==== 118+0+0 (secure 0 0 0) 0x7ffaac0704c0 con 0x7ffaac0c9650 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.755+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa9806b3e0 msgr2=0x7ffa9806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.755+0000 7ffab21c5640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa9806b3e0 0x7ffa9806d8a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0b0a10 tx=0x7ffaa0009040 comp rx=0 tx=0).stop 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.755+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 msgr2=0x7ffaac15bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.755+0000 7ffab21c5640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 0x7ffaac15bb00 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c02bf70 tx=0x7ffa9c0046c0 comp rx=0 tx=0).stop 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 shutdown_connections 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ffa9806b3e0 0x7ffa9806d8a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:58.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ffaac0ccfb0 0x7ffaac158fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:58.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffaac0ccbd0 0x7ffaac15e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:58.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 --2- 172.21.15.106:0/1515851940 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ffaac0c9650 0x7ffaac15bb00 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:58.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 >> 172.21.15.106:0/1515851940 conn(0x7ffaac0b0c50 msgr2=0x7ffaac0aeb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:58.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 shutdown_connections 2023-11-24T08:12:58.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.756+0000 7ffab21c5640 1 -- 172.21.15.106:0/1515851940 wait complete. 2023-11-24T08:12:58.778 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-24T08:12:58.779 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-24T08:12:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.998+0000 7f1046d7c640 1 Processor -- start 2023-11-24T08:12:59.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:58.998+0000 7f1046d7c640 1 -- start start 2023-11-24T08:12:59.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f1046d7c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10400b0fb0 0x7f104006ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f1046d7c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104015ed20 con 0x7f104015b260 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f1046d7c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f104015f8a0 con 0x7f10400b0b40 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f1046d7c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f104015b640 con 0x7f10400b0fb0 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f104657b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10400b0fb0 0x7f104006ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f104657b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10400b0fb0 0x7f104006ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38598/0 (socket says 172.21.15.106:38598) 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f1045d7a640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f104015b260 0x7f104006e040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37294/0 (socket says 172.21.15.106:37294) 2023-11-24T08:12:59.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.000+0000 7f104657b640 1 -- 172.21.15.106:0/1794573895 learned_addr learned my addr 172.21.15.106:0/1794573895 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:59.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.001+0000 7f104657b640 1 -- 172.21.15.106:0/1794573895 >> v1:172.21.15.156:6789/0 conn(0x7f10400b0b40 legacy=0x7f104006c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.001+0000 7f104657b640 1 -- 172.21.15.106:0/1794573895 >> v1:172.21.15.106:6789/0 conn(0x7f104015b260 legacy=0x7f104006e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.001+0000 7f104657b640 1 -- 172.21.15.106:0/1794573895 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104006e750 con 0x7f10400b0fb0 2023-11-24T08:12:59.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.002+0000 7f104657b640 1 --2- 172.21.15.106:0/1794573895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0fb0 0x7f104006ccc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1038009e50 tx=0x7f103802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4896edf837a3813 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:59.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.002+0000 7f1044d78640 1 -- 172.21.15.106:0/1794573895 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103802ca80 con 0x7f10400b0fb0 2023-11-24T08:12:59.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.002+0000 7f1044d78640 1 -- 172.21.15.106:0/1794573895 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103802cbe0 con 0x7f10400b0fb0 2023-11-24T08:12:59.004 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.002+0000 7f1044d78640 1 -- 172.21.15.106:0/1794573895 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380343f0 con 0x7f10400b0fb0 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 -- 172.21.15.106:0/1794573895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0fb0 msgr2=0x7f104006ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 --2- 172.21.15.106:0/1794573895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0fb0 0x7f104006ccc0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1038009e50 tx=0x7f103802bee0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 -- 172.21.15.106:0/1794573895 shutdown_connections 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 --2- 172.21.15.106:0/1794573895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0fb0 0x7f104006ccc0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 -- 172.21.15.106:0/1794573895 >> 172.21.15.106:0/1794573895 conn(0x7f10400cbb40 msgr2=0x7f10400cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 -- 172.21.15.106:0/1794573895 shutdown_connections 2023-11-24T08:12:59.005 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.003+0000 7f1046d7c640 1 -- 172.21.15.106:0/1794573895 wait complete. 2023-11-24T08:12:59.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.005+0000 7f1046d7c640 1 Processor -- start 2023-11-24T08:12:59.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.005+0000 7f1046d7c640 1 -- start start 2023-11-24T08:12:59.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1046d7c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 0x7f1040099c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:59.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1046d7c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10400b0fb0 0x7f10400c83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1046d7c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f104015b260 0x7f10400c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1046d7c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400ca670 con 0x7f10400b0b40 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1046d7c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c9a60 con 0x7f104015b260 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1046d7c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10400c9d40 con 0x7f10400b0fb0 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 0x7f1040099c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 0x7f1040099c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38614/0 (socket says 172.21.15.106:38614) 2023-11-24T08:12:59.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 -- 172.21.15.106:0/3341046947 learned_addr learned my addr 172.21.15.106:0/3341046947 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:12:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 -- 172.21.15.106:0/3341046947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10400b0fb0 msgr2=0x7f10400c83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10400b0fb0 0x7f10400c83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 -- 172.21.15.106:0/3341046947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f104015b260 msgr2=0x7f10400c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f104015b260 0x7f10400c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.006+0000 7f1045d7a640 1 -- 172.21.15.106:0/3341046947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038009b30 con 0x7f10400b0b40 2023-11-24T08:12:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.007+0000 7f1045d7a640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 0x7f1040099c40 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f103c00c3e0 tx=0x7f103c00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.007+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0074e0 con 0x7f10400b0b40 2023-11-24T08:12:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.007+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c007640 con 0x7f10400b0b40 2023-11-24T08:12:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.007+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0049e0 con 0x7f10400b0b40 2023-11-24T08:12:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.007+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10400ca020 con 0x7f10400b0b40 2023-11-24T08:12:59.010 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.007+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10400c9710 con 0x7f10400b0b40 2023-11-24T08:12:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.008+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104004e840 con 0x7f10400b0b40 2023-11-24T08:12:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.009+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f103c004b40 con 0x7f10400b0b40 2023-11-24T08:12:59.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.010+0000 7f10367fc640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f102406b3e0 0x7f102406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:12:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.010+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f103c089a90 con 0x7f10400b0b40 2023-11-24T08:12:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.011+0000 7f1045579640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f102406b3e0 0x7f102406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:12:59.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.011+0000 7f1045579640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f102406b3e0 0x7f102406d8a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1030000f60 tx=0x7f10300079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:12:59.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.015+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f103c054290 con 0x7f10400b0b40 2023-11-24T08:12:59.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.242+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f104015f500 con 0x7f10400b0b40 2023-11-24T08:12:59.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.764+0000 7f10367fc640 1 -- 172.21.15.106:0/3341046947 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v53) v1 ==== 122+0+0 (secure 0 0 0) 0x7f103c0578b0 con 0x7f10400b0b40 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f102406b3e0 msgr2=0x7f102406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f102406b3e0 0x7f102406d8a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1030000f60 tx=0x7f10300079b0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 msgr2=0x7f1040099c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 0x7f1040099c40 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f103c00c3e0 tx=0x7f103c00c8a0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 shutdown_connections 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f104015b260 0x7f10400c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.770 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f102406b3e0 0x7f102406d8a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10400b0fb0 0x7f10400c83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 --2- 172.21.15.106:0/3341046947 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10400b0b40 0x7f1040099c40 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:12:59.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.768+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 >> 172.21.15.106:0/3341046947 conn(0x7f10400cbb40 msgr2=0x7f10400adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:12:59.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.769+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 shutdown_connections 2023-11-24T08:12:59.771 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:12:59.769+0000 7f1046d7c640 1 -- 172.21.15.106:0/3341046947 wait complete. 2023-11-24T08:12:59.791 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-24T08:12:59.791 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:00.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.007+0000 7f8c71cee640 1 Processor -- start 2023-11-24T08:13:00.009 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.007+0000 7f8c71cee640 1 -- start start 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c71cee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8c6c15c7f0 0x7f8c6c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c71cee640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c0cece0 con 0x7f8c6c0cf450 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c71cee640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6c06d040 con 0x7f8c6c0ce6c0 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c71cee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c0ceaf0 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c6b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8c6c15c7f0 0x7f8c6c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c6affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8c6c0cf450 0x7f8c6c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37296/0 (socket says 172.21.15.106:37296) 2023-11-24T08:13:00.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.009+0000 7f8c6affd640 1 -- 172.21.15.106:0/1350157407 learned_addr learned my addr 172.21.15.106:0/1350157407 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:00.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.010+0000 7f8c6b7fe640 1 -- 172.21.15.106:0/1350157407 >> v1:172.21.15.156:6789/0 conn(0x7f8c6c0ce6c0 legacy=0x7f8c6c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:00.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.010+0000 7f8c6b7fe640 1 -- 172.21.15.106:0/1350157407 >> v1:172.21.15.106:6789/0 conn(0x7f8c6c0cf450 legacy=0x7f8c6c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.010+0000 7f8c6a7fc640 1 -- 172.21.15.106:0/1350157407 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 695584418 0 0) 0x7f8c6c0cece0 con 0x7f8c6c0cf450 2023-11-24T08:13:00.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.010+0000 7f8c6b7fe640 1 -- 172.21.15.106:0/1350157407 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c06d320 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.011+0000 7f8c6b7fe640 1 --2- 172.21.15.106:0/1350157407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c06db20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009d90 tx=0x7f8c5c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc4e47eefda7fbb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.011+0000 7f8c6a7fc640 1 -- 172.21.15.106:0/1350157407 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c02cc60 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.011+0000 7f8c6a7fc640 1 -- 172.21.15.106:0/1350157407 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c02cdc0 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.013 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.011+0000 7f8c6a7fc640 1 -- 172.21.15.106:0/1350157407 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c034660 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 -- 172.21.15.106:0/1350157407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 msgr2=0x7f8c6c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 --2- 172.21.15.106:0/1350157407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c06db20 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009d90 tx=0x7f8c5c02beb0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 -- 172.21.15.106:0/1350157407 shutdown_connections 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 --2- 172.21.15.106:0/1350157407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c06db20 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 -- 172.21.15.106:0/1350157407 >> 172.21.15.106:0/1350157407 conn(0x7f8c6c06a920 msgr2=0x7f8c6c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 -- 172.21.15.106:0/1350157407 shutdown_connections 2023-11-24T08:13:00.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.012+0000 7f8c71cee640 1 -- 172.21.15.106:0/1350157407 wait complete. 2023-11-24T08:13:00.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.014+0000 7f8c71cee640 1 Processor -- start 2023-11-24T08:13:00.015 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.014+0000 7f8c71cee640 1 -- start start 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c71cee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c6c0ce6c0 0x7f8c6c154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c71cee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c6c0cf450 0x7f8c6c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c71cee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c15d230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c71cee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c155040 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c71cee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0ae110 con 0x7f8c6c0ce6c0 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c71cee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0ac160 con 0x7f8c6c0cf450 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c15d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c6bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c15d230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38628/0 (socket says 172.21.15.106:38628) 2023-11-24T08:13:00.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.015+0000 7f8c6bfff640 1 -- 172.21.15.106:0/469412164 learned_addr learned my addr 172.21.15.106:0/469412164 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6affd640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c6c0cf450 0x7f8c6c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6bfff640 1 -- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c6c0cf450 msgr2=0x7f8c6c0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6b7fe640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c6c0ce6c0 0x7f8c6c154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6bfff640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c6c0cf450 0x7f8c6c0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6bfff640 1 -- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c6c0ce6c0 msgr2=0x7f8c6c154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6bfff640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c6c0ce6c0 0x7f8c6c154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6bfff640 1 -- 172.21.15.106:0/469412164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c009a70 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6affd640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c6c0cf450 0x7f8c6c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c6bfff640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c15d230 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c410 tx=0x7f8c6400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c64005230 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c64005390 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c640117d0 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.016+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c0ac440 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.017+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c155860 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.018+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c0abfc0 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.019+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8c64011930 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.019+0000 7f8c63fff640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c4c06b3e0 0x7f8c4c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.020+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c64089500 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.020+0000 7f8c6b7fe640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c4c06b3e0 0x7f8c4c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.022 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.020+0000 7f8c6b7fe640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c4c06b3e0 0x7f8c4c06d8a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036000 tx=0x7f8c5c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.024+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8c6c0abfc0 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.256+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c6c0ac960 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.257+0000 7f8c63fff640 1 -- 172.21.15.106:0/469412164 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f8c6c0ac960 con 0x7f8c6c15c7f0 2023-11-24T08:13:00.259 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:00.259 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:56.174099+0000","modified":"2023-11-24T08:12:59.244264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1896010072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1896010072},{"type":"v1","addr":"172.21.15.106:6837","nonce":1896010072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:13:00.259 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 53 2023-11-24T08:13:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c4c06b3e0 msgr2=0x7f8c4c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c4c06b3e0 0x7f8c4c06d8a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c036000 tx=0x7f8c5c030000 comp rx=0 tx=0).stop 2023-11-24T08:13:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 msgr2=0x7f8c6c15d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c15d230 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400c410 tx=0x7f8c6400c8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 shutdown_connections 2023-11-24T08:13:00.262 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8c6c15c7f0 0x7f8c6c15d230 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8c4c06b3e0 0x7f8c4c06d8a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8c6c0cf450 0x7f8c6c0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 --2- 172.21.15.106:0/469412164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c6c0ce6c0 0x7f8c6c154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 >> 172.21.15.106:0/469412164 conn(0x7f8c6c06a920 msgr2=0x7f8c6c0cb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:00.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 shutdown_connections 2023-11-24T08:13:00.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.260+0000 7f8c71cee640 1 -- 172.21.15.106:0/469412164 wait complete. 2023-11-24T08:13:00.279 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.483+0000 7f09e4ffb640 1 Processor -- start 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.483+0000 7f09e4ffb640 1 -- start start 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.484+0000 7f09e4ffb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 0x7f09e015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.484+0000 7f09e4ffb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09e00b2a70 0x7f09e015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.484+0000 7f09e4ffb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09e015ab80 con 0x7f09e00b2630 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.484+0000 7f09e4ffb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09e015a990 con 0x7f09e00b2a70 2023-11-24T08:13:00.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.484+0000 7f09e4ffb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e015e680 con 0x7f09e00b2250 2023-11-24T08:13:00.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.485+0000 7f09de59c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 0x7f09e015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.485+0000 7f09ded9d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f09e00b2a70 0x7f09e015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.485+0000 7f09de59c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 0x7f09e015df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:56726/0 (socket says 172.21.15.106:56726) 2023-11-24T08:13:00.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.485+0000 7f09de59c640 1 -- 172.21.15.106:0/1310810439 learned_addr learned my addr 172.21.15.106:0/1310810439 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:00.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.486+0000 7f09de59c640 1 -- 172.21.15.106:0/1310810439 >> v2:172.21.15.156:3300/0 conn(0x7f09e00b2a70 msgr2=0x7f09e015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:00.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.486+0000 7f09de59c640 1 --2- 172.21.15.106:0/1310810439 >> v2:172.21.15.156:3300/0 conn(0x7f09e00b2a70 0x7f09e015b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.486+0000 7f09de59c640 1 -- 172.21.15.106:0/1310810439 >> v1:172.21.15.156:6789/0 conn(0x7f09e00b2630 legacy=0x7f09e00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.486+0000 7f09de59c640 1 -- 172.21.15.106:0/1310810439 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e015e960 con 0x7f09e00b2250 2023-11-24T08:13:00.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.487+0000 7f09de59c640 1 --2- 172.21.15.106:0/1310810439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 0x7f09e015df80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009870 tx=0x7f09d002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b8d7edb1f10509 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.488+0000 7f09ddd9b640 1 -- 172.21.15.106:0/1310810439 <== mon.2 v2:172.21.15.156:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0030840 con 0x7f09e00b2250 2023-11-24T08:13:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.488+0000 7f09ddd9b640 1 -- 172.21.15.106:0/1310810439 <== mon.2 v2:172.21.15.156:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d00309a0 con 0x7f09e00b2250 2023-11-24T08:13:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.488+0000 7f09ddd9b640 1 -- 172.21.15.106:0/1310810439 <== mon.2 v2:172.21.15.156:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0030d30 con 0x7f09e00b2250 2023-11-24T08:13:00.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.488+0000 7f09e4ffb640 1 -- 172.21.15.106:0/1310810439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 msgr2=0x7f09e015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/1310810439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 0x7f09e015df80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009870 tx=0x7f09d002b980 comp rx=0 tx=0).stop 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 -- 172.21.15.106:0/1310810439 shutdown_connections 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/1310810439 >> v2:172.21.15.156:3300/0 conn(0x7f09e00b2a70 0x7f09e015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/1310810439 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2250 0x7f09e015df80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 -- 172.21.15.106:0/1310810439 >> 172.21.15.106:0/1310810439 conn(0x7f09e006a920 msgr2=0x7f09e006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 -- 172.21.15.106:0/1310810439 shutdown_connections 2023-11-24T08:13:00.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.489+0000 7f09e4ffb640 1 -- 172.21.15.106:0/1310810439 wait complete. 2023-11-24T08:13:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.491+0000 7f09e4ffb640 1 Processor -- start 2023-11-24T08:13:00.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.491+0000 7f09e4ffb640 1 -- start start 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09e4ffb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 0x7f09e014da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09e4ffb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2630 0x7f09e0154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09e4ffb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e00b2a70 0x7f09e0152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09e4ffb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0153ee0 con 0x7f09e00b2250 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09e4ffb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0152900 con 0x7f09e00b2a70 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09e4ffb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0153920 con 0x7f09e00b2630 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 0x7f09e014da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 0x7f09e014da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38638/0 (socket says 172.21.15.106:38638) 2023-11-24T08:13:00.494 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 -- 172.21.15.106:0/3405903288 learned_addr learned my addr 172.21.15.106:0/3405903288 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09d7fff640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2630 0x7f09e0154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09ded9d640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e00b2a70 0x7f09e0152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 -- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2630 msgr2=0x7f09e0154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2630 0x7f09e0154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 -- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e00b2a70 msgr2=0x7f09e0152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e00b2a70 0x7f09e0152d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.492+0000 7f09de59c640 1 -- 172.21.15.106:0/3405903288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00094d0 con 0x7f09e00b2250 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.493+0000 7f09de59c640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 0x7f09e014da00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f09d0000c00 tx=0x7f09d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.493+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0039c90 con 0x7f09e00b2250 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.493+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09d0039df0 con 0x7f09e00b2250 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.493+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09d0043640 con 0x7f09e00b2250 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.493+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e00cd7c0 con 0x7f09e00b2250 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.493+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e0152590 con 0x7f09e00b2250 2023-11-24T08:13:00.496 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.494+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e01523f0 con 0x7f09e00b2250 2023-11-24T08:13:00.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.495+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f09d00437a0 con 0x7f09e00b2250 2023-11-24T08:13:00.497 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.496+0000 7f09d6ffd640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f09c006b410 0x7f09c006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.496+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09d003e070 con 0x7f09e00b2250 2023-11-24T08:13:00.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.496+0000 7f09d7fff640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f09c006b410 0x7f09c006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.497+0000 7f09d7fff640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f09c006b410 0x7f09c006d8d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f09e01f6b20 tx=0x7f09c8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.501+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f09e01523f0 con 0x7f09e00b2250 2023-11-24T08:13:00.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.692+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09e01523f0 con 0x7f09e00b2250 2023-11-24T08:13:00.696 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.695+0000 7f09d6ffd640 1 -- 172.21.15.106:0/3405903288 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16608 (secure 0 0 0) 0x7f09e01523f0 con 0x7f09e00b2250 2023-11-24T08:13:00.697 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:00.697 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:57.727441+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:00.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.697+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f09c006b410 msgr2=0x7f09c006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.699 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.697+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f09c006b410 0x7f09c006d8d0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f09e01f6b20 tx=0x7f09c8009040 comp rx=0 tx=0).stop 2023-11-24T08:13:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 msgr2=0x7f09e014da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 0x7f09e014da00 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f09d0000c00 tx=0x7f09d0004730 comp rx=0 tx=0).stop 2023-11-24T08:13:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 shutdown_connections 2023-11-24T08:13:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f09e00b2a70 0x7f09e0152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f09c006b410 0x7f09c006d8d0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f09e00b2630 0x7f09e0154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 --2- 172.21.15.106:0/3405903288 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f09e00b2250 0x7f09e014da00 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 >> 172.21.15.106:0/3405903288 conn(0x7f09e006a920 msgr2=0x7f09e00cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 shutdown_connections 2023-11-24T08:13:00.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.698+0000 7f09e4ffb640 1 -- 172.21.15.106:0/3405903288 wait complete. 2023-11-24T08:13:00.718 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-24T08:13:00.719 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:00.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.928+0000 7f2118a90640 1 Processor -- start 2023-11-24T08:13:00.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.928+0000 7f2118a90640 1 -- start start 2023-11-24T08:13:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2118a90640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f21140b2250 0x7f211415df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2118a90640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2630 0x7f21140ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2118a90640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21140cd470 0x7f211415b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2118a90640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211415a990 con 0x7f21140cd470 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2118a90640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211415ab80 con 0x7f21140b2250 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2118a90640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211415e680 con 0x7f21140b2630 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2112d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21140cd470 0x7f211415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2112d9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f21140cd470 0x7f211415b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38654/0 (socket says 172.21.15.106:38654) 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.930+0000 7f2112d9d640 1 -- 172.21.15.106:0/3539099100 learned_addr learned my addr 172.21.15.106:0/3539099100 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f211259c640 1 --2- 172.21.15.106:0/3539099100 >> v2:172.21.15.156:3300/0 conn(0x7f21140b2250 0x7f211415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f210bfff640 1 --2- 172.21.15.106:0/3539099100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2630 0x7f21140ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f2112d9d640 1 -- 172.21.15.106:0/3539099100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2630 msgr2=0x7f21140ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f2112d9d640 1 --2- 172.21.15.106:0/3539099100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2630 0x7f21140ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f2112d9d640 1 -- 172.21.15.106:0/3539099100 >> v2:172.21.15.156:3300/0 conn(0x7f21140b2250 msgr2=0x7f211415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f2112d9d640 1 --2- 172.21.15.106:0/3539099100 >> v2:172.21.15.156:3300/0 conn(0x7f21140b2250 0x7f211415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.931+0000 7f2112d9d640 1 -- 172.21.15.106:0/3539099100 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211415e960 con 0x7f21140cd470 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.932+0000 7f2112d9d640 1 --2- 172.21.15.106:0/3539099100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140cd470 0x7f211415b080 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2114072a70 tx=0x7f210c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467b1be15edb80ee server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.932+0000 7f2111d9b640 1 -- 172.21.15.106:0/3539099100 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c02ca70 con 0x7f21140cd470 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.932+0000 7f2111d9b640 1 -- 172.21.15.106:0/3539099100 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f210c02cbd0 con 0x7f21140cd470 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.932+0000 7f2111d9b640 1 -- 172.21.15.106:0/3539099100 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f210c0343f0 con 0x7f21140cd470 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 -- 172.21.15.106:0/3539099100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140cd470 msgr2=0x7f211415b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 --2- 172.21.15.106:0/3539099100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140cd470 0x7f211415b080 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f2114072a70 tx=0x7f210c02bed0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 -- 172.21.15.106:0/3539099100 shutdown_connections 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 --2- 172.21.15.106:0/3539099100 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140cd470 0x7f211415b080 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 --2- 172.21.15.106:0/3539099100 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2630 0x7f21140ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 --2- 172.21.15.106:0/3539099100 >> v2:172.21.15.156:3300/0 conn(0x7f21140b2250 0x7f211415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 -- 172.21.15.106:0/3539099100 >> 172.21.15.106:0/3539099100 conn(0x7f211406a920 msgr2=0x7f211406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 -- 172.21.15.106:0/3539099100 shutdown_connections 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.933+0000 7f2118a90640 1 -- 172.21.15.106:0/3539099100 wait complete. 2023-11-24T08:13:00.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.935+0000 7f2118a90640 1 Processor -- start 2023-11-24T08:13:00.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.935+0000 7f2118a90640 1 -- start start 2023-11-24T08:13:00.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f2118a90640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2250 0x7f2114155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f2118a90640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 0x7f211414d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f2118a90640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21140cd470 0x7f211415bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f2118a90640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21141558a0 con 0x7f21140b2630 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f2118a90640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2114155b80 con 0x7f21140cd470 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f2118a90640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2114155e60 con 0x7f21140b2250 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f210bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 0x7f211414d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f210bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 0x7f211414d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38656/0 (socket says 172.21.15.106:38656) 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.936+0000 7f210bfff640 1 -- 172.21.15.106:0/370991808 learned_addr learned my addr 172.21.15.106:0/370991808 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f2112d9d640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21140cd470 0x7f211415bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f211259c640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2250 0x7f2114155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210bfff640 1 -- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2250 msgr2=0x7f2114155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210bfff640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2250 0x7f2114155050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210bfff640 1 -- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21140cd470 msgr2=0x7f211415bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210bfff640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21140cd470 0x7f211415bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210bfff640 1 -- 172.21.15.106:0/370991808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c009b30 con 0x7f21140b2630 2023-11-24T08:13:00.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210bfff640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 0x7f211414d310 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c7f0 tx=0x7f20fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.937+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc004220 con 0x7f21140b2630 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.938+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc0043a0 con 0x7f21140b2630 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.938+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc007860 con 0x7f21140b2630 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.938+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2114156140 con 0x7f21140b2630 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.938+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2114155590 con 0x7f21140b2630 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.939+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2114156660 con 0x7f21140b2630 2023-11-24T08:13:00.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.940+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f20fc007bd0 con 0x7f21140b2630 2023-11-24T08:13:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.941+0000 7f210affd640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20f406b3e0 0x7f20f406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:00.942 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.941+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20fc08a200 con 0x7f21140b2630 2023-11-24T08:13:00.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.941+0000 7f211259c640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20f406b3e0 0x7f20f406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:00.943 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.942+0000 7f211259c640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20f406b3e0 0x7f20f406d8a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2104006fd0 tx=0x7f2104008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:00.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:00.945+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2114156660 con 0x7f21140b2630 2023-11-24T08:13:01.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.118+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2114156660 con 0x7f20f406b3e0 2023-11-24T08:13:01.132 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.131+0000 7f210affd640 1 -- 172.21.15.106:0/370991808 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+275382 (secure 0 0 0) 0x7f2114156660 con 0x7f20f406b3e0 2023-11-24T08:13:01.136 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:01.137 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20f406b3e0 msgr2=0x7f20f406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20f406b3e0 0x7f20f406d8a0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f2104006fd0 tx=0x7f2104008040 comp rx=0 tx=0).stop 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 msgr2=0x7f211414d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 0x7f211414d310 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f20fc00c7f0 tx=0x7f20fc00ccb0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 shutdown_connections 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f21140cd470 0x7f211415bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f21140b2630 0x7f211414d310 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20f406b3e0 0x7f20f406d8a0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 --2- 172.21.15.106:0/370991808 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f21140b2250 0x7f2114155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 >> 172.21.15.106:0/370991808 conn(0x7f211406a920 msgr2=0x7f21141591c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 shutdown_connections 2023-11-24T08:13:01.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.135+0000 7f2118a90640 1 -- 172.21.15.106:0/370991808 wait complete. 2023-11-24T08:13:01.154 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2023-11-24T08:12:59.350795+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258788,"kb_used_data":9336,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715932,"statfs":{"total":773094113280,"available":772829114368,"internally_reserved":0,"allocated":9560064,"data_stored":4737940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":20,"apply_latency_ms":20,"commit_latency_ns":20000000,"apply_latency_ns":20000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.657431"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.705206+0000","last_change":"2023-11-24T08:12:54.705206+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.705206+0000","last_undegraded":"2023-11-24T08:12:54.705206+0000","last_fullsized":"2023-11-24T08:12:54.705206+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.702074+0000","last_change":"2023-11-24T08:12:54.702074+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.702074+0000","last_undegraded":"2023-11-24T08:12:54.702074+0000","last_fullsized":"2023-11-24T08:12:54.702074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.704118+0000","last_change":"2023-11-24T08:12:54.704118+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.704118+0000","last_undegraded":"2023-11-24T08:12:54.704118+0000","last_fullsized":"2023-11-24T08:12:54.704118+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.704405+0000","last_change":"2023-11-24T08:12:54.704405+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.704405+0000","last_undegraded":"2023-11-24T08:12:54.704405+0000","last_fullsized":"2023-11-24T08:12:54.704405+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.699728+0000","last_change":"2023-11-24T08:12:54.699728+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.699728+0000","last_undegraded":"2023-11-24T08:12:54.699728+0000","last_fullsized":"2023-11-24T08:12:54.699728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.700985+0000","last_change":"2023-11-24T08:12:54.700985+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.700985+0000","last_undegraded":"2023-11-24T08:12:54.700985+0000","last_fullsized":"2023-11-24T08:12:54.700985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.704095+0000","last_change":"2023-11-24T08:12:54.704095+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.704095+0000","last_undegraded":"2023-11-24T08:12:54.704095+0000","last_fullsized":"2023-11-24T08:12:54.704095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698005+0000","last_change":"2023-11-24T08:12:54.698005+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698005+0000","last_undegraded":"2023-11-24T08:12:54.698005+0000","last_fullsized":"2023-11-24T08:12:54.698005+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.701460+0000","last_change":"2023-11-24T08:12:54.701460+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.701460+0000","last_undegraded":"2023-11-24T08:12:54.701460+0000","last_fullsized":"2023-11-24T08:12:54.701460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698550+0000","last_change":"2023-11-24T08:12:54.698550+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698550+0000","last_undegraded":"2023-11-24T08:12:54.698550+0000","last_fullsized":"2023-11-24T08:12:54.698550+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.700679+0000","last_change":"2023-11-24T08:12:54.700679+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.700679+0000","last_undegraded":"2023-11-24T08:12:54.700679+0000","last_fullsized":"2023-11-24T08:12:54.700679+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.707823+0000","last_change":"2023-11-24T08:12:54.707823+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.707823+0000","last_undegraded":"2023-11-24T08:12:54.707823+0000","last_fullsized":"2023-11-24T08:12:54.707823+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698016+0000","last_change":"2023-11-24T08:12:54.698016+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698016+0000","last_undegraded":"2023-11-24T08:12:54.698016+0000","last_fullsized":"2023-11-24T08:12:54.698016+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.702633+0000","last_change":"2023-11-24T08:12:54.702633+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.702633+0000","last_undegraded":"2023-11-24T08:12:54.702633+0000","last_fullsized":"2023-11-24T08:12:54.702633+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.708218+0000","last_change":"2023-11-24T08:12:54.708218+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.708218+0000","last_undegraded":"2023-11-24T08:12:54.708218+0000","last_fullsized":"2023-11-24T08:12:54.708218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.703595+0000","last_change":"2023-11-24T08:12:54.703595+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.703595+0000","last_undegraded":"2023-11-24T08:12:54.703595+0000","last_fullsized":"2023-11-24T08:12:54.703595+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698612+0000","last_change":"2023-11-24T08:12:54.698612+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698612+0000","last_undegraded":"2023-11-24T08:12:54.698612+0000","last_fullsized":"2023-11-24T08:12:54.698612+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.704448+0000","last_change":"2023-11-24T08:12:54.704448+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.704448+0000","last_undegraded":"2023-11-24T08:12:54.704448+0000","last_fullsized":"2023-11-24T08:12:54.704448+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.703602+0000","last_change":"2023-11-24T08:12:54.703602+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.703602+0000","last_undegraded":"2023-11-24T08:12:54.703602+0000","last_fullsized":"2023-11-24T08:12:54.703602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.702322+0000","last_change":"2023-11-24T08:12:54.702322+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.702322+0000","last_undegraded":"2023-11-24T08:12:54.702322+0000","last_fullsized":"2023-11-24T08:12:54.702322+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.707359+0000","last_change":"2023-11-24T08:12:54.707359+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.707359+0000","last_undegraded":"2023-11-24T08:12:54.707359+0000","last_fullsized":"2023-11-24T08:12:54.707359+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.703523+0000","last_change":"2023-11-24T08:12:54.703523+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.703523+0000","last_undegraded":"2023-11-24T08:12:54.703523+0000","last_fullsized":"2023-11-24T08:12:54.703523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.702348+0000","last_change":"2023-11-24T08:12:54.702348+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.702348+0000","last_undegraded":"2023-11-24T08:12:54.702348+0000","last_fullsized":"2023-11-24T08:12:54.702348+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.697920+0000","last_change":"2023-11-24T08:12:54.697920+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.697920+0000","last_undegraded":"2023-11-24T08:12:54.697920+0000","last_fullsized":"2023-11-24T08:12:54.697920+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698095+0000","last_change":"2023-11-24T08:12:54.698095+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698095+0000","last_undegraded":"2023-11-24T08:12:54.698095+0000","last_fullsized":"2023-11-24T08:12:54.698095+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.699542+0000","last_change":"2023-11-24T08:12:54.699542+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.699542+0000","last_undegraded":"2023-11-24T08:12:54.699542+0000","last_fullsized":"2023-11-24T08:12:54.699542+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698472+0000","last_change":"2023-11-24T08:12:54.698472+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698472+0000","last_undegraded":"2023-11-24T08:12:54.698472+0000","last_fullsized":"2023-11-24T08:12:54.698472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.703768+0000","last_change":"2023-11-24T08:12:54.703768+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.703768+0000","last_undegraded":"2023-11-24T08:12:54.703768+0000","last_fullsized":"2023-11-24T08:12:54.703768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.699755+0000","last_change":"2023-11-24T08:12:54.699755+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.699755+0000","last_undegraded":"2023-11-24T08:12:54.699755+0000","last_fullsized":"2023-11-24T08:12:54.699755+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.698419+0000","last_change":"2023-11-24T08:12:54.698419+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.698419+0000","last_undegraded":"2023-11-24T08:12:54.698419+0000","last_fullsized":"2023-11-24T08:12:54.698419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.699181+0000","last_change":"2023-11-24T08:12:54.699181+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.699181+0000","last_undegraded":"2023-11-24T08:12:54.699181+0000","last_fullsized":"2023-11-24T08:12:54.699181+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-24T08:12:55.692275+0000","last_change":"2023-11-24T08:12:55.692275+0000","last_active":"2023-11-24T08:12:55.692275+0000","last_peered":"2023-11-24T08:12:55.692275+0000","last_clean":"2023-11-24T08:12:55.692275+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:55.692275+0000","last_undegraded":"2023-11-24T08:12:55.692275+0000","last_fullsized":"2023-11-24T08:12:55.692275+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":86,"state":"creating+peering","last_fresh":"2023-11-24T08:12:54.710173+0000","last_change":"2023-11-24T08:12:54.710173+0000","last_active":"2023-11-24T08:12:54.680581+0000","last_peered":"2023-11-24T08:12:54.680581+0000","last_clean":"2023-11-24T08:12:54.680581+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-24T08:12:54.710173+0000","last_undegraded":"2023-11-24T08:12:54.710173+0000","last_fullsized":"2023-11-24T08:12:54.710173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":215,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-11-24T08:12:51.698726+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:51.698726+0000","last_peered":"2023-11-24T08:12:51.698726+0000","last_clean":"2023-11-24T08:12:51.698726+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:51.698726+0000","last_undegraded":"2023-11-24T08:12:51.698726+0000","last_fullsized":"2023-11-24T08:12:51.698726+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542199,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1646592,"data_stored":1037927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.196,"5min":1.073,"15min":1.0129999999999999},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8580000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.401}]},{"osd":1,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.75800000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.89700000000000002},{"interface":"front","average":{"1min":1.196,"5min":1.181,"15min":1.1679999999999999},"min":{"1min":0.90500000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7150000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":0.83299999999999996}]},{"osd":2,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.135,"15min":1.073},"min":{"1min":0.77300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.821,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":0.89700000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.954,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.357}]},{"osd":3,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.234},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5860000000000001,"5min":3.387,"15min":3.387},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.276,"15min":1.23},"min":{"1min":0.98699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.865,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.26}]},{"osd":4,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.2210000000000001},"min":{"1min":0.93799999999999994,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9259999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.3939999999999999,"15min":1.377},"min":{"1min":0.97999999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.909,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.321}]},{"osd":5,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":0.91800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7909999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.512},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3959999999999999,"15min":1.3640000000000001},"min":{"1min":1.081,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.984,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.673}]},{"osd":6,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.542,"15min":1.4950000000000001},"min":{"1min":1.4510000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.073,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.728},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7909999999999999,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.756}]}]},{"osd":6,"up_from":14,"seq":60129542198,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1040384,"data_stored":441742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4099999999999999,"5min":2.085,"15min":2.085},"last":1.379},{"interface":"front","average":{"1min":1.236,"5min":1.2110000000000001,"15min":1.236},"min":{"1min":0.745,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.55}]},{"osd":1,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.629,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.665},{"interface":"front","average":{"1min":1.048,"5min":1.145,"15min":1.163},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.593,"5min":1.98,"15min":1.98},"last":1.2270000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4199999999999999,"5min":1.778,"15min":1.778},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6659999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.484}]},{"osd":3,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.335},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7250000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.72},{"interface":"front","average":{"1min":1.264,"5min":1.395,"15min":1.4330000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.0669999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3160000000000001,"15min":1.3600000000000001},"min":{"1min":0.71499999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.7709999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.623},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.581},"min":{"1min":0.84499999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1269999999999998,"5min":2.661,"15min":2.661},"last":1.7709999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.6890000000000001},"min":{"1min":1.0900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0049999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.861},{"interface":"front","average":{"1min":1.419,"5min":1.538,"15min":1.6339999999999999},"min":{"1min":1.0580000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.752,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.3260000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6930000000000001,"15min":1.7689999999999999},"min":{"1min":1.3460000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.2370000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.919},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.6759999999999999,"15min":1.7689999999999999},"min":{"1min":1.1439999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.4179999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542197,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":441742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":1.383}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.2370000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.409}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":1.01},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.88900000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.5529999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.196}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":1.3140000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.4470000000000001},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.355}]}]},{"osd":4,"up_from":14,"seq":60129542198,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":447559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0369999999999999,"15min":0.996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6180000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.74199999999999999,"5min":0.626,"15min":0.626},"max":{"1min":1.7909999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.4350000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.157,"15min":1.155},"min":{"1min":0.90300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2090000000000001,"15min":1.1699999999999999},"min":{"1min":0.81200000000000006,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6559999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.88900000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.88900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.746,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.21},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.79900000000000004,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3320000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2869999999999999},"min":{"1min":0.748,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.399,"15min":1.415},"min":{"1min":0.96199999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.901,"5min":2.746,"15min":2.746},"last":1.5409999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2889999999999999,"15min":1.3149999999999999},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.919,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4690000000000001,"15min":1.4630000000000001},"min":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.502}]},{"osd":6,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.411,"15min":1.4410000000000001},"min":{"1min":0.83999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.101,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.5840000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.89200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.5700000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.526,"15min":1.516},"min":{"1min":1.0820000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.0059999999999998,"5min":2.548,"15min":2.548},"last":1.633},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.5070000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.6830000000000001}]}]},{"osd":3,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1040384,"data_stored":441742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":1.0489999999999999,"15min":1.145},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3180000000000001,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.84499999999999997,"5min":1.0589999999999999,"15min":1.143},"min":{"1min":0.46100000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.085,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.86599999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.95899999999999996,"15min":0.995},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":1.4630000000000001,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.85499999999999998,"5min":1.0740000000000001,"15min":1.117},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.534,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.1559999999999999,"15min":1.22},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.6319999999999999,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.1220000000000001,"15min":1.1479999999999999},"min":{"1min":0.52000000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.5269999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.3160000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.1020000000000001,"5min":1.385,"15min":1.458},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.631,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":2.0659999999999998},{"interface":"front","average":{"1min":1.141,"5min":1.337,"15min":1.373},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.4670000000000001,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.137}]},{"osd":5,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.5069999999999999,"15min":1.5680000000000001},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.8,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.397},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.486,"15min":1.536},"min":{"1min":0.872,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8700000000000001,"5min":3.339,"15min":3.339},"last":1.411}]},{"osd":6,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.5609999999999999,"15min":1.6659999999999999},"min":{"1min":0.746,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.742,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.349},{"interface":"front","average":{"1min":1.379,"5min":1.522,"15min":1.5580000000000001},"min":{"1min":1.0249999999999999,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":1.681,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":2.0030000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:11:52 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.605,"15min":1.675},"min":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":1.603,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":2.105},{"interface":"front","average":{"1min":1.355,"5min":1.8320000000000001,"15min":2.016},"min":{"1min":0.82299999999999995,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7629999999999999,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.456}]}]},{"osd":2,"up_from":14,"seq":60129542198,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32196,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339644,"statfs":{"total":96636764160,"available":96603795456,"internally_reserved":0,"allocated":1040384,"data_stored":441742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.014}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.60799999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.4690000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.284},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.5329999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.9950000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.571}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":2.1869999999999998},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.746}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.8959999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":2.3479999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":2.1339999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":2.0790000000000002}]}]},{"osd":1,"up_from":14,"seq":60129542198,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1052672,"data_stored":447559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.476,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2809999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0109999999999999},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.60199999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1519999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6279999999999999,"5min":1.907,"15min":1.907},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.458,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.167,"15min":1.165},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.325},"min":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":0.92400000000000004}]},{"osd":5,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.248,"15min":1.296},"min":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.661,"15min":2.661},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6819999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":0.88800000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.321,"15min":1.349},"min":{"1min":0.78900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.633,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.827},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3400000000000001,"15min":1.3340000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.8460000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3999999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.351,"15min":1.363},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.008,"15min":2.008},"last":0.76700000000000002}]}]},{"osd":0,"up_from":14,"seq":60129542198,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32788,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339052,"statfs":{"total":96636764160,"available":96603189248,"internally_reserved":0,"allocated":1646592,"data_stored":1037927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":1.006,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5069999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":0.89000000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.48399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":0.94099999999999995}]},{"osd":3,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.05,"15min":1.0720000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4450000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88800000000000001,"15min":0.95199999999999996},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.78100000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.591,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.29},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5880000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1950000000000001,"15min":1.218},"min":{"1min":0.85799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.506,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2529999999999999,"15min":1.2669999999999999},"min":{"1min":0.79600000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.722,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.163}]},{"osd":6,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2490000000000001,"15min":1.266},"min":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.512,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2849999999999999,"15min":1.2869999999999999},"min":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9039999999999999,"5min":2.149,"15min":2.149},"last":1.1080000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.879,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7809999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.2509999999999999},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6890000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4330000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:01.159 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:01.408 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.407+0000 7f814f64b640 1 Processor -- start 2023-11-24T08:13:01.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.407+0000 7f814f64b640 1 -- start start 2023-11-24T08:13:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.408+0000 7f814f64b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f81480b2250 0x7f814815df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.408+0000 7f814f64b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 0x7f81480ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.408+0000 7f814f64b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f814815ab80 con 0x7f81480cd470 2023-11-24T08:13:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.408+0000 7f814f64b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f814815a990 con 0x7f81480b2250 2023-11-24T08:13:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.408+0000 7f814f64b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f814815e680 con 0x7f81480b2630 2023-11-24T08:13:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f814dbe8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f81480cd470 0x7f814815b080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37302/0 (socket says 172.21.15.106:37302) 2023-11-24T08:13:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f814dbe8640 1 -- 172.21.15.106:0/697221121 learned_addr learned my addr 172.21.15.106:0/697221121 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f814d3e7640 1 --2- 172.21.15.106:0/697221121 >> v2:172.21.15.156:3300/0 conn(0x7f81480b2250 0x7f814815df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f814cbe6640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 0x7f81480ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139885788 0 0) 0x7f814815ab80 con 0x7f81480cd470 2023-11-24T08:13:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8134003520 con 0x7f81480cd470 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.409+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3913446913 0 0) 0x7f8134003520 con 0x7f81480cd470 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 msgr2=0x7f81480ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 0x7f81480ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 >> v2:172.21.15.156:3300/0 conn(0x7f81480b2250 msgr2=0x7f814815df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 --2- 172.21.15.106:0/697221121 >> v2:172.21.15.156:3300/0 conn(0x7f81480b2250 0x7f814815df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814815e960 con 0x7f81480cd470 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f81400019f0 con 0x7f81480cd470 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 >> v1:172.21.15.106:6789/0 conn(0x7f81480cd470 legacy=0x7f814815b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8134003970 0x7f8134003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81340047c0 0x7f8134004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 0x7f8134005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814815e680 con 0x7f8134003970 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f813400dbf0 con 0x7f81340047c0 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f813400ddb0 con 0x7f81480b2630 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814d3e7640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8134003970 0x7f8134003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814cbe6640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81340047c0 0x7f8134004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814d3e7640 1 -- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 msgr2=0x7f8134005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814d3e7640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f81480b2630 0x7f8134005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814d3e7640 1 -- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81340047c0 msgr2=0x7f8134004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814d3e7640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81340047c0 0x7f8134004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.410+0000 7f814d3e7640 1 -- 172.21.15.106:0/697221121 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8134003520 con 0x7f8134003970 2023-11-24T08:13:01.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.411+0000 7f814d3e7640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8134003970 0x7f8134003d90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f814400ba00 tx=0x7f81340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4fe6b3bc28e6f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:01.415 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.412+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144032630 con 0x7f8134003970 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.412+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8144032790 con 0x7f8134003970 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.412+0000 7f813ffff640 1 -- 172.21.15.106:0/697221121 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8144032b20 con 0x7f8134003970 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.412+0000 7f814f64b640 1 -- 172.21.15.106:0/697221121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8134003970 msgr2=0x7f8134003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.412+0000 7f814f64b640 1 --2- 172.21.15.106:0/697221121 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8134003970 0x7f8134003d90 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f814400ba00 tx=0x7f81340057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.412+0000 7f814dbe8640 1 -- 172.21.15.106:0/697221121 reap_dead start 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.413+0000 7f814f64b640 1 -- 172.21.15.106:0/697221121 shutdown_connections 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.413+0000 7f814f64b640 1 -- 172.21.15.106:0/697221121 >> 172.21.15.106:0/697221121 conn(0x7f814806a920 msgr2=0x7f814806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.413+0000 7f814f64b640 1 -- 172.21.15.106:0/697221121 shutdown_connections 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.413+0000 7f814f64b640 1 -- 172.21.15.106:0/697221121 wait complete. 2023-11-24T08:13:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.414+0000 7f814f64b640 1 Processor -- start 2023-11-24T08:13:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.414+0000 7f814f64b640 1 -- start start 2023-11-24T08:13:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.415+0000 7f814f64b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8134003970 0x7f8148146980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.415+0000 7f814f64b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 0x7f8148099670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.415+0000 7f814f64b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148145020 0x7f81481470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.415+0000 7f814f64b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8148146ec0 con 0x7f8148147d30 2023-11-24T08:13:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.415+0000 7f814f64b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81481458e0 con 0x7f8148145020 2023-11-24T08:13:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.415+0000 7f814f64b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8148145bc0 con 0x7f8134003970 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 0x7f8148099670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 0x7f8148099670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38668/0 (socket says 172.21.15.106:38668) 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 -- 172.21.15.106:0/2091518833 learned_addr learned my addr 172.21.15.106:0/2091518833 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814d3e7640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8134003970 0x7f8148146980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814dbe8640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148145020 0x7f81481470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 -- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8134003970 msgr2=0x7f8148146980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8134003970 0x7f8148146980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 -- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148145020 msgr2=0x7f81481470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148145020 0x7f81481470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814cbe6640 1 -- 172.21.15.106:0/2091518833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8148145ea0 con 0x7f8148147d30 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.416+0000 7f814dbe8640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148145020 0x7f81481470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.417+0000 7f814cbe6640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 0x7f8148099670 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f813800e7e0 tx=0x7f813800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.417+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800c6d0 con 0x7f8148147d30 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.417+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f813800c830 con 0x7f8148147d30 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.417+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f813800cba0 con 0x7f8148147d30 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.417+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8148146180 con 0x7f8148147d30 2023-11-24T08:13:01.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.417+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8148099bb0 con 0x7f8148147d30 2023-11-24T08:13:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.418+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8148067470 con 0x7f8148147d30 2023-11-24T08:13:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.419+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f813800cd00 con 0x7f8148147d30 2023-11-24T08:13:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.420+0000 7f813dffb640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812c06b3e0 0x7f812c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.420+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8138089fd0 con 0x7f8148147d30 2023-11-24T08:13:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.420+0000 7f814d3e7640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812c06b3e0 0x7f812c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.421+0000 7f814d3e7640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812c06b3e0 0x7f812c06d8a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f81340057a0 tx=0x7f814403d040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:01.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.424+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f81380546c0 con 0x7f8148147d30 2023-11-24T08:13:01.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.611+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f814806eed0 con 0x7f812c06b3e0 2023-11-24T08:13:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.622+0000 7f813dffb640 1 -- 172.21.15.106:0/2091518833 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+287343 (secure 0 0 0) 0x7f814806eed0 con 0x7f812c06b3e0 2023-11-24T08:13:01.628 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:01.629 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:01.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812c06b3e0 msgr2=0x7f812c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812c06b3e0 0x7f812c06d8a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f81340057a0 tx=0x7f814403d040 comp rx=0 tx=0).stop 2023-11-24T08:13:01.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 msgr2=0x7f8148099670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 0x7f8148099670 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f813800e7e0 tx=0x7f813800eca0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 shutdown_connections 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8148145020 0x7f81481470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8148147d30 0x7f8148099670 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f812c06b3e0 0x7f812c06d8a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 --2- 172.21.15.106:0/2091518833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8134003970 0x7f8148146980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 >> 172.21.15.106:0/2091518833 conn(0x7f814806a920 msgr2=0x7f814806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 shutdown_connections 2023-11-24T08:13:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.628+0000 7f814f64b640 1 -- 172.21.15.106:0/2091518833 wait complete. 2023-11-24T08:13:01.647 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2023-11-24T08:13:01.351361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":156,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259180,"kb_used_data":9736,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715540,"statfs":{"total":773094113280,"available":772828712960,"internally_reserved":0,"allocated":9969664,"data_stored":4920828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001732"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.746900+0000","last_change":"2023-11-24T08:12:56.750647+0000","last_active":"2023-11-24T08:12:57.746900+0000","last_peered":"2023-11-24T08:12:57.746900+0000","last_clean":"2023-11-24T08:12:57.746900+0000","last_became_active":"2023-11-24T08:12:56.750294+0000","last_became_peered":"2023-11-24T08:12:56.750294+0000","last_unstale":"2023-11-24T08:12:57.746900+0000","last_undegraded":"2023-11-24T08:12:57.746900+0000","last_fullsized":"2023-11-24T08:12:57.746900+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:39:50.007568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481203+0000","last_change":"2023-11-24T08:12:56.753438+0000","last_active":"2023-11-24T08:12:58.481203+0000","last_peered":"2023-11-24T08:12:58.481203+0000","last_clean":"2023-11-24T08:12:58.481203+0000","last_became_active":"2023-11-24T08:12:56.753167+0000","last_became_peered":"2023-11-24T08:12:56.753167+0000","last_unstale":"2023-11-24T08:12:58.481203+0000","last_undegraded":"2023-11-24T08:12:58.481203+0000","last_fullsized":"2023-11-24T08:12:58.481203+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:08:47.248550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481902+0000","last_change":"2023-11-24T08:12:56.747991+0000","last_active":"2023-11-24T08:12:58.481902+0000","last_peered":"2023-11-24T08:12:58.481902+0000","last_clean":"2023-11-24T08:12:58.481902+0000","last_became_active":"2023-11-24T08:12:56.747722+0000","last_became_peered":"2023-11-24T08:12:56.747722+0000","last_unstale":"2023-11-24T08:12:58.481902+0000","last_undegraded":"2023-11-24T08:12:58.481902+0000","last_fullsized":"2023-11-24T08:12:58.481902+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:23.303036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.740033+0000","last_change":"2023-11-24T08:12:56.753354+0000","last_active":"2023-11-24T08:12:57.740033+0000","last_peered":"2023-11-24T08:12:57.740033+0000","last_clean":"2023-11-24T08:12:57.740033+0000","last_became_active":"2023-11-24T08:12:56.753084+0000","last_became_peered":"2023-11-24T08:12:56.753084+0000","last_unstale":"2023-11-24T08:12:57.740033+0000","last_undegraded":"2023-11-24T08:12:57.740033+0000","last_fullsized":"2023-11-24T08:12:57.740033+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:25.875254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.744234+0000","last_change":"2023-11-24T08:12:56.756795+0000","last_active":"2023-11-24T08:12:57.744234+0000","last_peered":"2023-11-24T08:12:57.744234+0000","last_clean":"2023-11-24T08:12:57.744234+0000","last_became_active":"2023-11-24T08:12:56.754839+0000","last_became_peered":"2023-11-24T08:12:56.754839+0000","last_unstale":"2023-11-24T08:12:57.744234+0000","last_undegraded":"2023-11-24T08:12:57.744234+0000","last_fullsized":"2023-11-24T08:12:57.744234+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:19:54.897051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480322+0000","last_change":"2023-11-24T08:12:56.759716+0000","last_active":"2023-11-24T08:12:58.480322+0000","last_peered":"2023-11-24T08:12:58.480322+0000","last_clean":"2023-11-24T08:12:58.480322+0000","last_became_active":"2023-11-24T08:12:56.759447+0000","last_became_peered":"2023-11-24T08:12:56.759447+0000","last_unstale":"2023-11-24T08:12:58.480322+0000","last_undegraded":"2023-11-24T08:12:58.480322+0000","last_fullsized":"2023-11-24T08:12:58.480322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:07.350872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.739649+0000","last_change":"2023-11-24T08:12:56.745786+0000","last_active":"2023-11-24T08:12:57.739649+0000","last_peered":"2023-11-24T08:12:57.739649+0000","last_clean":"2023-11-24T08:12:57.739649+0000","last_became_active":"2023-11-24T08:12:56.745513+0000","last_became_peered":"2023-11-24T08:12:56.745513+0000","last_unstale":"2023-11-24T08:12:57.739649+0000","last_undegraded":"2023-11-24T08:12:57.739649+0000","last_fullsized":"2023-11-24T08:12:57.739649+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:19:53.820154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480322+0000","last_change":"2023-11-24T08:12:56.760819+0000","last_active":"2023-11-24T08:12:58.480322+0000","last_peered":"2023-11-24T08:12:58.480322+0000","last_clean":"2023-11-24T08:12:58.480322+0000","last_became_active":"2023-11-24T08:12:56.760527+0000","last_became_peered":"2023-11-24T08:12:56.760527+0000","last_unstale":"2023-11-24T08:12:58.480322+0000","last_undegraded":"2023-11-24T08:12:58.480322+0000","last_fullsized":"2023-11-24T08:12:58.480322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:37:20.587190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481452+0000","last_change":"2023-11-24T08:12:56.759062+0000","last_active":"2023-11-24T08:12:58.481452+0000","last_peered":"2023-11-24T08:12:58.481452+0000","last_clean":"2023-11-24T08:12:58.481452+0000","last_became_active":"2023-11-24T08:12:56.758731+0000","last_became_peered":"2023-11-24T08:12:56.758731+0000","last_unstale":"2023-11-24T08:12:58.481452+0000","last_undegraded":"2023-11-24T08:12:58.481452+0000","last_fullsized":"2023-11-24T08:12:58.481452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:41:18.770546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.747323+0000","last_change":"2023-11-24T08:12:56.734933+0000","last_active":"2023-11-24T08:12:57.747323+0000","last_peered":"2023-11-24T08:12:57.747323+0000","last_clean":"2023-11-24T08:12:57.747323+0000","last_became_active":"2023-11-24T08:12:56.734354+0000","last_became_peered":"2023-11-24T08:12:56.734354+0000","last_unstale":"2023-11-24T08:12:57.747323+0000","last_undegraded":"2023-11-24T08:12:57.747323+0000","last_fullsized":"2023-11-24T08:12:57.747323+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:02.356865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482891+0000","last_change":"2023-11-24T08:12:56.746530+0000","last_active":"2023-11-24T08:12:58.482891+0000","last_peered":"2023-11-24T08:12:58.482891+0000","last_clean":"2023-11-24T08:12:58.482891+0000","last_became_active":"2023-11-24T08:12:56.745004+0000","last_became_peered":"2023-11-24T08:12:56.745004+0000","last_unstale":"2023-11-24T08:12:58.482891+0000","last_undegraded":"2023-11-24T08:12:58.482891+0000","last_fullsized":"2023-11-24T08:12:58.482891+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:58:10.649882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.744473+0000","last_change":"2023-11-24T08:12:56.734796+0000","last_active":"2023-11-24T08:12:57.744473+0000","last_peered":"2023-11-24T08:12:57.744473+0000","last_clean":"2023-11-24T08:12:57.744473+0000","last_became_active":"2023-11-24T08:12:56.734397+0000","last_became_peered":"2023-11-24T08:12:56.734397+0000","last_unstale":"2023-11-24T08:12:57.744473+0000","last_undegraded":"2023-11-24T08:12:57.744473+0000","last_fullsized":"2023-11-24T08:12:57.744473+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:00.633954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481670+0000","last_change":"2023-11-24T08:12:56.749127+0000","last_active":"2023-11-24T08:12:58.481670+0000","last_peered":"2023-11-24T08:12:58.481670+0000","last_clean":"2023-11-24T08:12:58.481670+0000","last_became_active":"2023-11-24T08:12:56.748864+0000","last_became_peered":"2023-11-24T08:12:56.748864+0000","last_unstale":"2023-11-24T08:12:58.481670+0000","last_undegraded":"2023-11-24T08:12:58.481670+0000","last_fullsized":"2023-11-24T08:12:58.481670+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:32.081416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.739684+0000","last_change":"2023-11-24T08:12:56.750597+0000","last_active":"2023-11-24T08:12:57.739684+0000","last_peered":"2023-11-24T08:12:57.739684+0000","last_clean":"2023-11-24T08:12:57.739684+0000","last_became_active":"2023-11-24T08:12:56.750312+0000","last_became_peered":"2023-11-24T08:12:56.750312+0000","last_unstale":"2023-11-24T08:12:57.739684+0000","last_undegraded":"2023-11-24T08:12:57.739684+0000","last_fullsized":"2023-11-24T08:12:57.739684+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:35.914318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481848+0000","last_change":"2023-11-24T08:12:56.737160+0000","last_active":"2023-11-24T08:12:58.481848+0000","last_peered":"2023-11-24T08:12:58.481848+0000","last_clean":"2023-11-24T08:12:58.481848+0000","last_became_active":"2023-11-24T08:12:56.736732+0000","last_became_peered":"2023-11-24T08:12:56.736732+0000","last_unstale":"2023-11-24T08:12:58.481848+0000","last_undegraded":"2023-11-24T08:12:58.481848+0000","last_fullsized":"2023-11-24T08:12:58.481848+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:47.828075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.743655+0000","last_change":"2023-11-24T08:12:56.751833+0000","last_active":"2023-11-24T08:12:57.743655+0000","last_peered":"2023-11-24T08:12:57.743655+0000","last_clean":"2023-11-24T08:12:57.743655+0000","last_became_active":"2023-11-24T08:12:56.751551+0000","last_became_peered":"2023-11-24T08:12:56.751551+0000","last_unstale":"2023-11-24T08:12:57.743655+0000","last_undegraded":"2023-11-24T08:12:57.743655+0000","last_fullsized":"2023-11-24T08:12:57.743655+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:33:01.023239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.747511+0000","last_change":"2023-11-24T08:12:56.755561+0000","last_active":"2023-11-24T08:12:57.747511+0000","last_peered":"2023-11-24T08:12:57.747511+0000","last_clean":"2023-11-24T08:12:57.747511+0000","last_became_active":"2023-11-24T08:12:56.755034+0000","last_became_peered":"2023-11-24T08:12:56.755034+0000","last_unstale":"2023-11-24T08:12:57.747511+0000","last_undegraded":"2023-11-24T08:12:57.747511+0000","last_fullsized":"2023-11-24T08:12:57.747511+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:20.248866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.740353+0000","last_change":"2023-11-24T08:12:56.742822+0000","last_active":"2023-11-24T08:12:57.740353+0000","last_peered":"2023-11-24T08:12:57.740353+0000","last_clean":"2023-11-24T08:12:57.740353+0000","last_became_active":"2023-11-24T08:12:56.742524+0000","last_became_peered":"2023-11-24T08:12:56.742524+0000","last_unstale":"2023-11-24T08:12:57.740353+0000","last_undegraded":"2023-11-24T08:12:57.740353+0000","last_fullsized":"2023-11-24T08:12:57.740353+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:33.869908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.484150+0000","last_change":"2023-11-24T08:12:56.738659+0000","last_active":"2023-11-24T08:12:58.484150+0000","last_peered":"2023-11-24T08:12:58.484150+0000","last_clean":"2023-11-24T08:12:58.484150+0000","last_became_active":"2023-11-24T08:12:56.738379+0000","last_became_peered":"2023-11-24T08:12:56.738379+0000","last_unstale":"2023-11-24T08:12:58.484150+0000","last_undegraded":"2023-11-24T08:12:58.484150+0000","last_fullsized":"2023-11-24T08:12:58.484150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:40:16.593012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.740119+0000","last_change":"2023-11-24T08:12:56.741452+0000","last_active":"2023-11-24T08:12:57.740119+0000","last_peered":"2023-11-24T08:12:57.740119+0000","last_clean":"2023-11-24T08:12:57.740119+0000","last_became_active":"2023-11-24T08:12:56.741164+0000","last_became_peered":"2023-11-24T08:12:56.741164+0000","last_unstale":"2023-11-24T08:12:57.740119+0000","last_undegraded":"2023-11-24T08:12:57.740119+0000","last_fullsized":"2023-11-24T08:12:57.740119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:30.015794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.739845+0000","last_change":"2023-11-24T08:12:56.740353+0000","last_active":"2023-11-24T08:12:57.739845+0000","last_peered":"2023-11-24T08:12:57.739845+0000","last_clean":"2023-11-24T08:12:57.739845+0000","last_became_active":"2023-11-24T08:12:56.740026+0000","last_became_peered":"2023-11-24T08:12:56.740026+0000","last_unstale":"2023-11-24T08:12:57.739845+0000","last_undegraded":"2023-11-24T08:12:57.739845+0000","last_fullsized":"2023-11-24T08:12:57.739845+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:52:29.933787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480818+0000","last_change":"2023-11-24T08:12:56.743084+0000","last_active":"2023-11-24T08:12:58.480818+0000","last_peered":"2023-11-24T08:12:58.480818+0000","last_clean":"2023-11-24T08:12:58.480818+0000","last_became_active":"2023-11-24T08:12:56.742574+0000","last_became_peered":"2023-11-24T08:12:56.742574+0000","last_unstale":"2023-11-24T08:12:58.480818+0000","last_undegraded":"2023-11-24T08:12:58.480818+0000","last_fullsized":"2023-11-24T08:12:58.480818+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:23:33.869908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.746918+0000","last_change":"2023-11-24T08:12:56.746929+0000","last_active":"2023-11-24T08:12:57.746918+0000","last_peered":"2023-11-24T08:12:57.746918+0000","last_clean":"2023-11-24T08:12:57.746918+0000","last_became_active":"2023-11-24T08:12:56.746619+0000","last_became_peered":"2023-11-24T08:12:56.746619+0000","last_unstale":"2023-11-24T08:12:57.746918+0000","last_undegraded":"2023-11-24T08:12:57.746918+0000","last_fullsized":"2023-11-24T08:12:57.746918+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:46.783763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.739442+0000","last_change":"2023-11-24T08:12:56.745571+0000","last_active":"2023-11-24T08:12:57.739442+0000","last_peered":"2023-11-24T08:12:57.739442+0000","last_clean":"2023-11-24T08:12:57.739442+0000","last_became_active":"2023-11-24T08:12:56.745301+0000","last_became_peered":"2023-11-24T08:12:56.745301+0000","last_unstale":"2023-11-24T08:12:57.739442+0000","last_undegraded":"2023-11-24T08:12:57.739442+0000","last_fullsized":"2023-11-24T08:12:57.739442+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:01:06.984182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480834+0000","last_change":"2023-11-24T08:12:56.753281+0000","last_active":"2023-11-24T08:12:58.480834+0000","last_peered":"2023-11-24T08:12:58.480834+0000","last_clean":"2023-11-24T08:12:58.480834+0000","last_became_active":"2023-11-24T08:12:56.752979+0000","last_became_peered":"2023-11-24T08:12:56.752979+0000","last_unstale":"2023-11-24T08:12:58.480834+0000","last_undegraded":"2023-11-24T08:12:58.480834+0000","last_fullsized":"2023-11-24T08:12:58.480834+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:44:32.552178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482228+0000","last_change":"2023-11-24T08:12:56.746121+0000","last_active":"2023-11-24T08:12:58.482228+0000","last_peered":"2023-11-24T08:12:58.482228+0000","last_clean":"2023-11-24T08:12:58.482228+0000","last_became_active":"2023-11-24T08:12:56.744972+0000","last_became_peered":"2023-11-24T08:12:56.744972+0000","last_unstale":"2023-11-24T08:12:58.482228+0000","last_undegraded":"2023-11-24T08:12:58.482228+0000","last_fullsized":"2023-11-24T08:12:58.482228+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:11:44.801065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.745148+0000","last_change":"2023-11-24T08:12:56.752519+0000","last_active":"2023-11-24T08:12:57.745148+0000","last_peered":"2023-11-24T08:12:57.745148+0000","last_clean":"2023-11-24T08:12:57.745148+0000","last_became_active":"2023-11-24T08:12:56.750795+0000","last_became_peered":"2023-11-24T08:12:56.750795+0000","last_unstale":"2023-11-24T08:12:57.745148+0000","last_undegraded":"2023-11-24T08:12:57.745148+0000","last_fullsized":"2023-11-24T08:12:57.745148+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:12.387169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.744788+0000","last_change":"2023-11-24T08:12:56.754147+0000","last_active":"2023-11-24T08:12:57.744788+0000","last_peered":"2023-11-24T08:12:57.744788+0000","last_clean":"2023-11-24T08:12:57.744788+0000","last_became_active":"2023-11-24T08:12:56.753812+0000","last_became_peered":"2023-11-24T08:12:56.753812+0000","last_unstale":"2023-11-24T08:12:57.744788+0000","last_undegraded":"2023-11-24T08:12:57.744788+0000","last_fullsized":"2023-11-24T08:12:57.744788+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:33.669052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481041+0000","last_change":"2023-11-24T08:12:56.739203+0000","last_active":"2023-11-24T08:12:58.481041+0000","last_peered":"2023-11-24T08:12:58.481041+0000","last_clean":"2023-11-24T08:12:58.481041+0000","last_became_active":"2023-11-24T08:12:56.738597+0000","last_became_peered":"2023-11-24T08:12:56.738597+0000","last_unstale":"2023-11-24T08:12:58.481041+0000","last_undegraded":"2023-11-24T08:12:58.481041+0000","last_fullsized":"2023-11-24T08:12:58.481041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:28:28.004140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481193+0000","last_change":"2023-11-24T08:12:56.742891+0000","last_active":"2023-11-24T08:12:58.481193+0000","last_peered":"2023-11-24T08:12:58.481193+0000","last_clean":"2023-11-24T08:12:58.481193+0000","last_became_active":"2023-11-24T08:12:56.742434+0000","last_became_peered":"2023-11-24T08:12:56.742434+0000","last_unstale":"2023-11-24T08:12:58.481193+0000","last_undegraded":"2023-11-24T08:12:58.481193+0000","last_fullsized":"2023-11-24T08:12:58.481193+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:59:48.056679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.743940+0000","last_change":"2023-11-24T08:12:56.758076+0000","last_active":"2023-11-24T08:12:57.743940+0000","last_peered":"2023-11-24T08:12:57.743940+0000","last_clean":"2023-11-24T08:12:57.743940+0000","last_became_active":"2023-11-24T08:12:56.757507+0000","last_became_peered":"2023-11-24T08:12:56.757507+0000","last_unstale":"2023-11-24T08:12:57.743940+0000","last_undegraded":"2023-11-24T08:12:57.743940+0000","last_fullsized":"2023-11-24T08:12:57.743940+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:37.926906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.744546+0000","last_change":"2023-11-24T08:12:56.742827+0000","last_active":"2023-11-24T08:12:57.744546+0000","last_peered":"2023-11-24T08:12:57.744546+0000","last_clean":"2023-11-24T08:12:57.744546+0000","last_became_active":"2023-11-24T08:12:56.742555+0000","last_became_peered":"2023-11-24T08:12:56.742555+0000","last_unstale":"2023-11-24T08:12:57.744546+0000","last_undegraded":"2023-11-24T08:12:57.744546+0000","last_fullsized":"2023-11-24T08:12:57.744546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:04:12.387169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.748527+0000","last_change":"2023-11-24T08:12:56.753747+0000","last_active":"2023-11-24T08:12:57.748527+0000","last_peered":"2023-11-24T08:12:57.748527+0000","last_clean":"2023-11-24T08:12:57.748527+0000","last_became_active":"2023-11-24T08:12:56.752406+0000","last_became_peered":"2023-11-24T08:12:56.752406+0000","last_unstale":"2023-11-24T08:12:57.748527+0000","last_undegraded":"2023-11-24T08:12:57.748527+0000","last_fullsized":"2023-11-24T08:12:57.748527+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:25:02.356865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.755006+0000","last_change":"2023-11-24T08:12:55.741000+0000","last_active":"2023-11-24T08:12:56.755006+0000","last_peered":"2023-11-24T08:12:56.755006+0000","last_clean":"2023-11-24T08:12:56.755006+0000","last_became_active":"2023-11-24T08:12:55.740165+0000","last_became_peered":"2023-11-24T08:12:55.740165+0000","last_unstale":"2023-11-24T08:12:56.755006+0000","last_undegraded":"2023-11-24T08:12:56.755006+0000","last_fullsized":"2023-11-24T08:12:56.755006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:38.927770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.747288+0000","last_change":"2023-11-24T08:12:56.739443+0000","last_active":"2023-11-24T08:12:57.747288+0000","last_peered":"2023-11-24T08:12:57.747288+0000","last_clean":"2023-11-24T08:12:57.747288+0000","last_became_active":"2023-11-24T08:12:56.739116+0000","last_became_peered":"2023-11-24T08:12:56.739116+0000","last_unstale":"2023-11-24T08:12:57.747288+0000","last_undegraded":"2023-11-24T08:12:57.747288+0000","last_fullsized":"2023-11-24T08:12:57.747288+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:31:20.248866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.752167+0000","last_change":"2023-11-24T08:12:55.740772+0000","last_active":"2023-11-24T08:12:56.752167+0000","last_peered":"2023-11-24T08:12:56.752167+0000","last_clean":"2023-11-24T08:12:56.752167+0000","last_became_active":"2023-11-24T08:12:55.740202+0000","last_became_peered":"2023-11-24T08:12:55.740202+0000","last_unstale":"2023-11-24T08:12:56.752167+0000","last_undegraded":"2023-11-24T08:12:56.752167+0000","last_fullsized":"2023-11-24T08:12:56.752167+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:34:38.927770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.747694+0000","last_change":"2023-11-24T08:12:56.747261+0000","last_active":"2023-11-24T08:12:57.747694+0000","last_peered":"2023-11-24T08:12:57.747694+0000","last_clean":"2023-11-24T08:12:57.747694+0000","last_became_active":"2023-11-24T08:12:56.746418+0000","last_became_peered":"2023-11-24T08:12:56.746418+0000","last_unstale":"2023-11-24T08:12:57.747694+0000","last_undegraded":"2023-11-24T08:12:57.747694+0000","last_fullsized":"2023-11-24T08:12:57.747694+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:41.508884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.754649+0000","last_change":"2023-11-24T08:12:55.743476+0000","last_active":"2023-11-24T08:12:56.754649+0000","last_peered":"2023-11-24T08:12:56.754649+0000","last_clean":"2023-11-24T08:12:56.754649+0000","last_became_active":"2023-11-24T08:12:55.743175+0000","last_became_peered":"2023-11-24T08:12:55.743175+0000","last_unstale":"2023-11-24T08:12:56.754649+0000","last_undegraded":"2023-11-24T08:12:56.754649+0000","last_fullsized":"2023-11-24T08:12:56.754649+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:32.934160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482119+0000","last_change":"2023-11-24T08:12:56.735776+0000","last_active":"2023-11-24T08:12:58.482119+0000","last_peered":"2023-11-24T08:12:58.482119+0000","last_clean":"2023-11-24T08:12:58.482119+0000","last_became_active":"2023-11-24T08:12:56.735435+0000","last_became_peered":"2023-11-24T08:12:56.735435+0000","last_unstale":"2023-11-24T08:12:58.482119+0000","last_undegraded":"2023-11-24T08:12:58.482119+0000","last_fullsized":"2023-11-24T08:12:58.482119+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:25:33.945854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.754759+0000","last_change":"2023-11-24T08:12:55.743662+0000","last_active":"2023-11-24T08:12:56.754759+0000","last_peered":"2023-11-24T08:12:56.754759+0000","last_clean":"2023-11-24T08:12:56.754759+0000","last_became_active":"2023-11-24T08:12:55.743388+0000","last_became_peered":"2023-11-24T08:12:55.743388+0000","last_unstale":"2023-11-24T08:12:56.754759+0000","last_undegraded":"2023-11-24T08:12:56.754759+0000","last_fullsized":"2023-11-24T08:12:56.754759+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:46.238956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481565+0000","last_change":"2023-11-24T08:12:56.742879+0000","last_active":"2023-11-24T08:12:58.481565+0000","last_peered":"2023-11-24T08:12:58.481565+0000","last_clean":"2023-11-24T08:12:58.481565+0000","last_became_active":"2023-11-24T08:12:56.742269+0000","last_became_peered":"2023-11-24T08:12:56.742269+0000","last_unstale":"2023-11-24T08:12:58.481565+0000","last_undegraded":"2023-11-24T08:12:58.481565+0000","last_fullsized":"2023-11-24T08:12:58.481565+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:45:18.047891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.754543+0000","last_change":"2023-11-24T08:12:55.742560+0000","last_active":"2023-11-24T08:12:56.754543+0000","last_peered":"2023-11-24T08:12:56.754543+0000","last_clean":"2023-11-24T08:12:56.754543+0000","last_became_active":"2023-11-24T08:12:55.742048+0000","last_became_peered":"2023-11-24T08:12:55.742048+0000","last_unstale":"2023-11-24T08:12:56.754543+0000","last_undegraded":"2023-11-24T08:12:56.754543+0000","last_fullsized":"2023-11-24T08:12:56.754543+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:47.371496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.744549+0000","last_change":"2023-11-24T08:12:56.753814+0000","last_active":"2023-11-24T08:12:57.744549+0000","last_peered":"2023-11-24T08:12:57.744549+0000","last_clean":"2023-11-24T08:12:57.744549+0000","last_became_active":"2023-11-24T08:12:56.753498+0000","last_became_peered":"2023-11-24T08:12:56.753498+0000","last_unstale":"2023-11-24T08:12:57.744549+0000","last_undegraded":"2023-11-24T08:12:57.744549+0000","last_fullsized":"2023-11-24T08:12:57.744549+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:36:00.633954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.753292+0000","last_change":"2023-11-24T08:12:55.741683+0000","last_active":"2023-11-24T08:12:56.753292+0000","last_peered":"2023-11-24T08:12:56.753292+0000","last_clean":"2023-11-24T08:12:56.753292+0000","last_became_active":"2023-11-24T08:12:55.741408+0000","last_became_peered":"2023-11-24T08:12:55.741408+0000","last_unstale":"2023-11-24T08:12:56.753292+0000","last_undegraded":"2023-11-24T08:12:56.753292+0000","last_fullsized":"2023-11-24T08:12:56.753292+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:29.130640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480615+0000","last_change":"2023-11-24T08:12:56.745345+0000","last_active":"2023-11-24T08:12:58.480615+0000","last_peered":"2023-11-24T08:12:58.480615+0000","last_clean":"2023-11-24T08:12:58.480615+0000","last_became_active":"2023-11-24T08:12:56.744834+0000","last_became_peered":"2023-11-24T08:12:56.744834+0000","last_unstale":"2023-11-24T08:12:58.480615+0000","last_undegraded":"2023-11-24T08:12:58.480615+0000","last_fullsized":"2023-11-24T08:12:58.480615+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:47.828075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.732582+0000","last_change":"2023-11-24T08:12:55.742424+0000","last_active":"2023-11-24T08:12:56.732582+0000","last_peered":"2023-11-24T08:12:56.732582+0000","last_clean":"2023-11-24T08:12:56.732582+0000","last_became_active":"2023-11-24T08:12:55.742140+0000","last_became_peered":"2023-11-24T08:12:55.742140+0000","last_unstale":"2023-11-24T08:12:56.732582+0000","last_undegraded":"2023-11-24T08:12:56.732582+0000","last_fullsized":"2023-11-24T08:12:56.732582+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:46.816380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480964+0000","last_change":"2023-11-24T08:12:56.736485+0000","last_active":"2023-11-24T08:12:58.480964+0000","last_peered":"2023-11-24T08:12:58.480964+0000","last_clean":"2023-11-24T08:12:58.480964+0000","last_became_active":"2023-11-24T08:12:56.735411+0000","last_became_peered":"2023-11-24T08:12:56.735411+0000","last_unstale":"2023-11-24T08:12:58.480964+0000","last_undegraded":"2023-11-24T08:12:58.480964+0000","last_fullsized":"2023-11-24T08:12:58.480964+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:41.508884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.730509+0000","last_change":"2023-11-24T08:12:55.739961+0000","last_active":"2023-11-24T08:12:56.730509+0000","last_peered":"2023-11-24T08:12:56.730509+0000","last_clean":"2023-11-24T08:12:56.730509+0000","last_became_active":"2023-11-24T08:12:55.739657+0000","last_became_peered":"2023-11-24T08:12:55.739657+0000","last_unstale":"2023-11-24T08:12:56.730509+0000","last_undegraded":"2023-11-24T08:12:56.730509+0000","last_fullsized":"2023-11-24T08:12:56.730509+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:40:34.902623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482595+0000","last_change":"2023-11-24T08:12:56.733155+0000","last_active":"2023-11-24T08:12:58.482595+0000","last_peered":"2023-11-24T08:12:58.482595+0000","last_clean":"2023-11-24T08:12:58.482595+0000","last_became_active":"2023-11-24T08:12:56.732817+0000","last_became_peered":"2023-11-24T08:12:56.732817+0000","last_unstale":"2023-11-24T08:12:58.482595+0000","last_undegraded":"2023-11-24T08:12:58.482595+0000","last_fullsized":"2023-11-24T08:12:58.482595+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:45.358902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.726332+0000","last_change":"2023-11-24T08:12:55.735828+0000","last_active":"2023-11-24T08:12:56.726332+0000","last_peered":"2023-11-24T08:12:56.726332+0000","last_clean":"2023-11-24T08:12:56.726332+0000","last_became_active":"2023-11-24T08:12:55.735515+0000","last_became_peered":"2023-11-24T08:12:55.735515+0000","last_unstale":"2023-11-24T08:12:56.726332+0000","last_undegraded":"2023-11-24T08:12:56.726332+0000","last_fullsized":"2023-11-24T08:12:56.726332+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:33:00.011544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481475+0000","last_change":"2023-11-24T08:12:56.760018+0000","last_active":"2023-11-24T08:12:58.481475+0000","last_peered":"2023-11-24T08:12:58.481475+0000","last_clean":"2023-11-24T08:12:58.481475+0000","last_became_active":"2023-11-24T08:12:56.759727+0000","last_became_peered":"2023-11-24T08:12:56.759727+0000","last_unstale":"2023-11-24T08:12:58.481475+0000","last_undegraded":"2023-11-24T08:12:58.481475+0000","last_fullsized":"2023-11-24T08:12:58.481475+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:14:35.228408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.723896+0000","last_change":"2023-11-24T08:12:55.738448+0000","last_active":"2023-11-24T08:12:56.723896+0000","last_peered":"2023-11-24T08:12:56.723896+0000","last_clean":"2023-11-24T08:12:56.723896+0000","last_became_active":"2023-11-24T08:12:55.738154+0000","last_became_peered":"2023-11-24T08:12:55.738154+0000","last_unstale":"2023-11-24T08:12:56.723896+0000","last_undegraded":"2023-11-24T08:12:56.723896+0000","last_fullsized":"2023-11-24T08:12:56.723896+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:23:15.059651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.483920+0000","last_change":"2023-11-24T08:12:56.748236+0000","last_active":"2023-11-24T08:12:58.483920+0000","last_peered":"2023-11-24T08:12:58.483920+0000","last_clean":"2023-11-24T08:12:58.483920+0000","last_became_active":"2023-11-24T08:12:56.747968+0000","last_became_peered":"2023-11-24T08:12:56.747968+0000","last_unstale":"2023-11-24T08:12:58.483920+0000","last_undegraded":"2023-11-24T08:12:58.483920+0000","last_fullsized":"2023-11-24T08:12:58.483920+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:28:24.141946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"88'3","reported_seq":15,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.760232+0000","last_change":"2023-11-24T08:12:55.737925+0000","last_active":"2023-11-24T08:12:56.760232+0000","last_peered":"2023-11-24T08:12:56.760232+0000","last_clean":"2023-11-24T08:12:56.760232+0000","last_became_active":"2023-11-24T08:12:55.737587+0000","last_became_peered":"2023-11-24T08:12:55.737587+0000","last_unstale":"2023-11-24T08:12:56.760232+0000","last_undegraded":"2023-11-24T08:12:56.760232+0000","last_fullsized":"2023-11-24T08:12:56.760232+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:47.371496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.745356+0000","last_change":"2023-11-24T08:12:56.756917+0000","last_active":"2023-11-24T08:12:57.745356+0000","last_peered":"2023-11-24T08:12:57.745356+0000","last_clean":"2023-11-24T08:12:57.745356+0000","last_became_active":"2023-11-24T08:12:56.755514+0000","last_became_peered":"2023-11-24T08:12:56.755514+0000","last_unstale":"2023-11-24T08:12:57.745356+0000","last_undegraded":"2023-11-24T08:12:57.745356+0000","last_fullsized":"2023-11-24T08:12:57.745356+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:59:33.669052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"88'6","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.764302+0000","last_change":"2023-11-24T08:12:55.741085+0000","last_active":"2023-11-24T08:12:56.764302+0000","last_peered":"2023-11-24T08:12:56.764302+0000","last_clean":"2023-11-24T08:12:56.764302+0000","last_became_active":"2023-11-24T08:12:55.739893+0000","last_became_peered":"2023-11-24T08:12:55.739893+0000","last_unstale":"2023-11-24T08:12:56.764302+0000","last_undegraded":"2023-11-24T08:12:56.764302+0000","last_fullsized":"2023-11-24T08:12:56.764302+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:22:25.667577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482077+0000","last_change":"2023-11-24T08:12:56.750348+0000","last_active":"2023-11-24T08:12:58.482077+0000","last_peered":"2023-11-24T08:12:58.482077+0000","last_clean":"2023-11-24T08:12:58.482077+0000","last_became_active":"2023-11-24T08:12:56.750069+0000","last_became_peered":"2023-11-24T08:12:56.750069+0000","last_unstale":"2023-11-24T08:12:58.482077+0000","last_undegraded":"2023-11-24T08:12:58.482077+0000","last_fullsized":"2023-11-24T08:12:58.482077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T10:11:44.801065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.756218+0000","last_change":"2023-11-24T08:12:55.740306+0000","last_active":"2023-11-24T08:12:56.756218+0000","last_peered":"2023-11-24T08:12:56.756218+0000","last_clean":"2023-11-24T08:12:56.756218+0000","last_became_active":"2023-11-24T08:12:55.737634+0000","last_became_peered":"2023-11-24T08:12:55.737634+0000","last_unstale":"2023-11-24T08:12:56.756218+0000","last_undegraded":"2023-11-24T08:12:56.756218+0000","last_fullsized":"2023-11-24T08:12:56.756218+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:43.012254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.481700+0000","last_change":"2023-11-24T08:12:56.744297+0000","last_active":"2023-11-24T08:12:58.481700+0000","last_peered":"2023-11-24T08:12:58.481700+0000","last_clean":"2023-11-24T08:12:58.481700+0000","last_became_active":"2023-11-24T08:12:56.744021+0000","last_became_peered":"2023-11-24T08:12:56.744021+0000","last_unstale":"2023-11-24T08:12:58.481700+0000","last_undegraded":"2023-11-24T08:12:58.481700+0000","last_fullsized":"2023-11-24T08:12:58.481700+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:47.250651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.716210+0000","last_change":"2023-11-24T08:12:55.739329+0000","last_active":"2023-11-24T08:12:56.716210+0000","last_peered":"2023-11-24T08:12:56.716210+0000","last_clean":"2023-11-24T08:12:56.716210+0000","last_became_active":"2023-11-24T08:12:55.738843+0000","last_became_peered":"2023-11-24T08:12:55.738843+0000","last_unstale":"2023-11-24T08:12:56.716210+0000","last_undegraded":"2023-11-24T08:12:56.716210+0000","last_fullsized":"2023-11-24T08:12:56.716210+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:39:48.995873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480003+0000","last_change":"2023-11-24T08:12:56.743656+0000","last_active":"2023-11-24T08:12:58.480003+0000","last_peered":"2023-11-24T08:12:58.480003+0000","last_clean":"2023-11-24T08:12:58.480003+0000","last_became_active":"2023-11-24T08:12:56.743407+0000","last_became_peered":"2023-11-24T08:12:56.743407+0000","last_unstale":"2023-11-24T08:12:58.480003+0000","last_undegraded":"2023-11-24T08:12:58.480003+0000","last_fullsized":"2023-11-24T08:12:58.480003+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:00:30.015794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.744785+0000","last_change":"2023-11-24T08:12:55.744025+0000","last_active":"2023-11-24T08:12:56.744785+0000","last_peered":"2023-11-24T08:12:56.744785+0000","last_clean":"2023-11-24T08:12:56.744785+0000","last_became_active":"2023-11-24T08:12:55.743148+0000","last_became_peered":"2023-11-24T08:12:55.743148+0000","last_unstale":"2023-11-24T08:12:56.744785+0000","last_undegraded":"2023-11-24T08:12:56.744785+0000","last_fullsized":"2023-11-24T08:12:56.744785+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:09:14.794128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.479793+0000","last_change":"2023-11-24T08:12:56.753341+0000","last_active":"2023-11-24T08:12:58.479793+0000","last_peered":"2023-11-24T08:12:58.479793+0000","last_clean":"2023-11-24T08:12:58.479793+0000","last_became_active":"2023-11-24T08:12:56.753014+0000","last_became_peered":"2023-11-24T08:12:56.753014+0000","last_unstale":"2023-11-24T08:12:58.479793+0000","last_undegraded":"2023-11-24T08:12:58.479793+0000","last_fullsized":"2023-11-24T08:12:58.479793+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:20:25.875254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"88'8","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.759104+0000","last_change":"2023-11-24T08:12:55.745951+0000","last_active":"2023-11-24T08:12:56.759104+0000","last_peered":"2023-11-24T08:12:56.759104+0000","last_clean":"2023-11-24T08:12:56.759104+0000","last_became_active":"2023-11-24T08:12:55.744671+0000","last_became_peered":"2023-11-24T08:12:55.744671+0000","last_unstale":"2023-11-24T08:12:56.759104+0000","last_undegraded":"2023-11-24T08:12:56.759104+0000","last_fullsized":"2023-11-24T08:12:56.759104+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:57:40.497190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482269+0000","last_change":"2023-11-24T08:12:56.751726+0000","last_active":"2023-11-24T08:12:58.482269+0000","last_peered":"2023-11-24T08:12:58.482269+0000","last_clean":"2023-11-24T08:12:58.482269+0000","last_became_active":"2023-11-24T08:12:56.749895+0000","last_became_peered":"2023-11-24T08:12:56.749895+0000","last_unstale":"2023-11-24T08:12:58.482269+0000","last_undegraded":"2023-11-24T08:12:58.482269+0000","last_fullsized":"2023-11-24T08:12:58.482269+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:41.469233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.750558+0000","last_change":"2023-11-24T08:12:55.736247+0000","last_active":"2023-11-24T08:12:56.750558+0000","last_peered":"2023-11-24T08:12:56.750558+0000","last_clean":"2023-11-24T08:12:56.750558+0000","last_became_active":"2023-11-24T08:12:55.735967+0000","last_became_peered":"2023-11-24T08:12:55.735967+0000","last_unstale":"2023-11-24T08:12:56.750558+0000","last_undegraded":"2023-11-24T08:12:56.750558+0000","last_fullsized":"2023-11-24T08:12:56.750558+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:19:53.885356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.747346+0000","last_change":"2023-11-24T08:12:56.740241+0000","last_active":"2023-11-24T08:12:57.747346+0000","last_peered":"2023-11-24T08:12:57.747346+0000","last_clean":"2023-11-24T08:12:57.747346+0000","last_became_active":"2023-11-24T08:12:56.739952+0000","last_became_peered":"2023-11-24T08:12:56.739952+0000","last_unstale":"2023-11-24T08:12:57.747346+0000","last_undegraded":"2023-11-24T08:12:57.747346+0000","last_fullsized":"2023-11-24T08:12:57.747346+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:48:50.122227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.756929+0000","last_change":"2023-11-24T08:12:55.739609+0000","last_active":"2023-11-24T08:12:56.756929+0000","last_peered":"2023-11-24T08:12:56.756929+0000","last_clean":"2023-11-24T08:12:56.756929+0000","last_became_active":"2023-11-24T08:12:55.739298+0000","last_became_peered":"2023-11-24T08:12:55.739298+0000","last_unstale":"2023-11-24T08:12:56.756929+0000","last_undegraded":"2023-11-24T08:12:56.756929+0000","last_fullsized":"2023-11-24T08:12:56.756929+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:44.347208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.745488+0000","last_change":"2023-11-24T08:12:56.755230+0000","last_active":"2023-11-24T08:12:57.745488+0000","last_peered":"2023-11-24T08:12:57.745488+0000","last_clean":"2023-11-24T08:12:57.745488+0000","last_became_active":"2023-11-24T08:12:56.754879+0000","last_became_peered":"2023-11-24T08:12:56.754879+0000","last_unstale":"2023-11-24T08:12:57.745488+0000","last_undegraded":"2023-11-24T08:12:57.745488+0000","last_fullsized":"2023-11-24T08:12:57.745488+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:05.187066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.738079+0000","last_change":"2023-11-24T08:12:55.742268+0000","last_active":"2023-11-24T08:12:56.738079+0000","last_peered":"2023-11-24T08:12:56.738079+0000","last_clean":"2023-11-24T08:12:56.738079+0000","last_became_active":"2023-11-24T08:12:55.741869+0000","last_became_peered":"2023-11-24T08:12:55.741869+0000","last_unstale":"2023-11-24T08:12:56.738079+0000","last_undegraded":"2023-11-24T08:12:56.738079+0000","last_fullsized":"2023-11-24T08:12:56.738079+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:35:22.291342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.740020+0000","last_change":"2023-11-24T08:12:56.749916+0000","last_active":"2023-11-24T08:12:57.740020+0000","last_peered":"2023-11-24T08:12:57.740020+0000","last_clean":"2023-11-24T08:12:57.740020+0000","last_became_active":"2023-11-24T08:12:56.749539+0000","last_became_peered":"2023-11-24T08:12:56.749539+0000","last_unstale":"2023-11-24T08:12:57.740020+0000","last_undegraded":"2023-11-24T08:12:57.740020+0000","last_fullsized":"2023-11-24T08:12:57.740020+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:15:33.798526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.744221+0000","last_change":"2023-11-24T08:12:55.737474+0000","last_active":"2023-11-24T08:12:56.744221+0000","last_peered":"2023-11-24T08:12:56.744221+0000","last_clean":"2023-11-24T08:12:56.744221+0000","last_became_active":"2023-11-24T08:12:55.737211+0000","last_became_peered":"2023-11-24T08:12:55.737211+0000","last_unstale":"2023-11-24T08:12:56.744221+0000","last_undegraded":"2023-11-24T08:12:56.744221+0000","last_fullsized":"2023-11-24T08:12:56.744221+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:45.772068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480973+0000","last_change":"2023-11-24T08:12:56.757269+0000","last_active":"2023-11-24T08:12:58.480973+0000","last_peered":"2023-11-24T08:12:58.480973+0000","last_clean":"2023-11-24T08:12:58.480973+0000","last_became_active":"2023-11-24T08:12:56.756977+0000","last_became_peered":"2023-11-24T08:12:56.756977+0000","last_unstale":"2023-11-24T08:12:58.480973+0000","last_undegraded":"2023-11-24T08:12:58.480973+0000","last_fullsized":"2023-11-24T08:12:58.480973+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T11:30:46.445335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.729630+0000","last_change":"2023-11-24T08:12:55.735145+0000","last_active":"2023-11-24T08:12:56.729630+0000","last_peered":"2023-11-24T08:12:56.729630+0000","last_clean":"2023-11-24T08:12:56.729630+0000","last_became_active":"2023-11-24T08:12:55.734900+0000","last_became_peered":"2023-11-24T08:12:55.734900+0000","last_unstale":"2023-11-24T08:12:56.729630+0000","last_undegraded":"2023-11-24T08:12:56.729630+0000","last_fullsized":"2023-11-24T08:12:56.729630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:39:08.631947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.739804+0000","last_change":"2023-11-24T08:12:56.754273+0000","last_active":"2023-11-24T08:12:57.739804+0000","last_peered":"2023-11-24T08:12:57.739804+0000","last_clean":"2023-11-24T08:12:57.739804+0000","last_became_active":"2023-11-24T08:12:56.753922+0000","last_became_peered":"2023-11-24T08:12:56.753922+0000","last_unstale":"2023-11-24T08:12:57.739804+0000","last_undegraded":"2023-11-24T08:12:57.739804+0000","last_fullsized":"2023-11-24T08:12:57.739804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T17:37:44.023949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.730687+0000","last_change":"2023-11-24T08:12:55.729965+0000","last_active":"2023-11-24T08:12:56.730687+0000","last_peered":"2023-11-24T08:12:56.730687+0000","last_clean":"2023-11-24T08:12:56.730687+0000","last_became_active":"2023-11-24T08:12:55.729659+0000","last_became_peered":"2023-11-24T08:12:55.729659+0000","last_unstale":"2023-11-24T08:12:56.730687+0000","last_undegraded":"2023-11-24T08:12:56.730687+0000","last_fullsized":"2023-11-24T08:12:56.730687+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:23:37.388060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480319+0000","last_change":"2023-11-24T08:12:56.753602+0000","last_active":"2023-11-24T08:12:58.480319+0000","last_peered":"2023-11-24T08:12:58.480319+0000","last_clean":"2023-11-24T08:12:58.480319+0000","last_became_active":"2023-11-24T08:12:56.753274+0000","last_became_peered":"2023-11-24T08:12:56.753274+0000","last_unstale":"2023-11-24T08:12:58.480319+0000","last_undegraded":"2023-11-24T08:12:58.480319+0000","last_fullsized":"2023-11-24T08:12:58.480319+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T19:06:24.938422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.716022+0000","last_change":"2023-11-24T08:12:55.734798+0000","last_active":"2023-11-24T08:12:56.716022+0000","last_peered":"2023-11-24T08:12:56.716022+0000","last_clean":"2023-11-24T08:12:56.716022+0000","last_became_active":"2023-11-24T08:12:55.734504+0000","last_became_peered":"2023-11-24T08:12:55.734504+0000","last_unstale":"2023-11-24T08:12:56.716022+0000","last_undegraded":"2023-11-24T08:12:56.716022+0000","last_fullsized":"2023-11-24T08:12:56.716022+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:04.175371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.745972+0000","last_change":"2023-11-24T08:12:56.755898+0000","last_active":"2023-11-24T08:12:57.745972+0000","last_peered":"2023-11-24T08:12:57.745972+0000","last_clean":"2023-11-24T08:12:57.745972+0000","last_became_active":"2023-11-24T08:12:56.752768+0000","last_became_peered":"2023-11-24T08:12:56.752768+0000","last_unstale":"2023-11-24T08:12:57.745972+0000","last_undegraded":"2023-11-24T08:12:57.745972+0000","last_fullsized":"2023-11-24T08:12:57.745972+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:44:46.783763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"88'2","reported_seq":14,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.755656+0000","last_change":"2023-11-24T08:12:55.736193+0000","last_active":"2023-11-24T08:12:56.755656+0000","last_peered":"2023-11-24T08:12:56.755656+0000","last_clean":"2023-11-24T08:12:56.755656+0000","last_became_active":"2023-11-24T08:12:55.735924+0000","last_became_peered":"2023-11-24T08:12:55.735924+0000","last_unstale":"2023-11-24T08:12:56.755656+0000","last_undegraded":"2023-11-24T08:12:56.755656+0000","last_fullsized":"2023-11-24T08:12:56.755656+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T12:52:28.922093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482383+0000","last_change":"2023-11-24T08:12:56.741432+0000","last_active":"2023-11-24T08:12:58.482383+0000","last_peered":"2023-11-24T08:12:58.482383+0000","last_clean":"2023-11-24T08:12:58.482383+0000","last_became_active":"2023-11-24T08:12:56.741118+0000","last_became_peered":"2023-11-24T08:12:56.741118+0000","last_unstale":"2023-11-24T08:12:58.482383+0000","last_undegraded":"2023-11-24T08:12:58.482383+0000","last_fullsized":"2023-11-24T08:12:58.482383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:28:24.141946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.736331+0000","last_change":"2023-11-24T08:12:55.738834+0000","last_active":"2023-11-24T08:12:56.736331+0000","last_peered":"2023-11-24T08:12:56.736331+0000","last_clean":"2023-11-24T08:12:56.736331+0000","last_became_active":"2023-11-24T08:12:55.737648+0000","last_became_peered":"2023-11-24T08:12:55.737648+0000","last_unstale":"2023-11-24T08:12:56.736331+0000","last_undegraded":"2023-11-24T08:12:56.736331+0000","last_fullsized":"2023-11-24T08:12:56.736331+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:15:32.786831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.748174+0000","last_change":"2023-11-24T08:12:56.740875+0000","last_active":"2023-11-24T08:12:57.748174+0000","last_peered":"2023-11-24T08:12:57.748174+0000","last_clean":"2023-11-24T08:12:57.748174+0000","last_became_active":"2023-11-24T08:12:56.737798+0000","last_became_peered":"2023-11-24T08:12:56.737798+0000","last_unstale":"2023-11-24T08:12:57.748174+0000","last_undegraded":"2023-11-24T08:12:57.748174+0000","last_fullsized":"2023-11-24T08:12:57.748174+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:40:47.828075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.724839+0000","last_change":"2023-11-24T08:12:55.739613+0000","last_active":"2023-11-24T08:12:56.724839+0000","last_peered":"2023-11-24T08:12:56.724839+0000","last_clean":"2023-11-24T08:12:56.724839+0000","last_became_active":"2023-11-24T08:12:55.739029+0000","last_became_peered":"2023-11-24T08:12:55.739029+0000","last_unstale":"2023-11-24T08:12:56.724839+0000","last_undegraded":"2023-11-24T08:12:56.724839+0000","last_fullsized":"2023-11-24T08:12:56.724839+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T16:22:29.130640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.745831+0000","last_change":"2023-11-24T08:12:56.757057+0000","last_active":"2023-11-24T08:12:57.745831+0000","last_peered":"2023-11-24T08:12:57.745831+0000","last_clean":"2023-11-24T08:12:57.745831+0000","last_became_active":"2023-11-24T08:12:56.756673+0000","last_became_peered":"2023-11-24T08:12:56.756673+0000","last_unstale":"2023-11-24T08:12:57.745831+0000","last_undegraded":"2023-11-24T08:12:57.745831+0000","last_fullsized":"2023-11-24T08:12:57.745831+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:39:50.007568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.725643+0000","last_change":"2023-11-24T08:12:55.736231+0000","last_active":"2023-11-24T08:12:56.725643+0000","last_peered":"2023-11-24T08:12:56.725643+0000","last_clean":"2023-11-24T08:12:56.725643+0000","last_became_active":"2023-11-24T08:12:55.735935+0000","last_became_peered":"2023-11-24T08:12:55.735935+0000","last_unstale":"2023-11-24T08:12:56.725643+0000","last_undegraded":"2023-11-24T08:12:56.725643+0000","last_fullsized":"2023-11-24T08:12:56.725643+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:21.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480683+0000","last_change":"2023-11-24T08:12:56.749436+0000","last_active":"2023-11-24T08:12:58.480683+0000","last_peered":"2023-11-24T08:12:58.480683+0000","last_clean":"2023-11-24T08:12:58.480683+0000","last_became_active":"2023-11-24T08:12:56.749027+0000","last_became_peered":"2023-11-24T08:12:56.749027+0000","last_unstale":"2023-11-24T08:12:58.480683+0000","last_undegraded":"2023-11-24T08:12:58.480683+0000","last_fullsized":"2023-11-24T08:12:58.480683+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T09:27:16.224088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.737875+0000","last_change":"2023-11-24T08:12:55.745403+0000","last_active":"2023-11-24T08:12:56.737875+0000","last_peered":"2023-11-24T08:12:56.737875+0000","last_clean":"2023-11-24T08:12:56.737875+0000","last_became_active":"2023-11-24T08:12:55.744901+0000","last_became_peered":"2023-11-24T08:12:55.744901+0000","last_unstale":"2023-11-24T08:12:56.737875+0000","last_undegraded":"2023-11-24T08:12:56.737875+0000","last_fullsized":"2023-11-24T08:12:56.737875+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:47:31.069722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.483343+0000","last_change":"2023-11-24T08:12:56.751785+0000","last_active":"2023-11-24T08:12:58.483343+0000","last_peered":"2023-11-24T08:12:58.483343+0000","last_clean":"2023-11-24T08:12:58.483343+0000","last_became_active":"2023-11-24T08:12:56.751466+0000","last_became_peered":"2023-11-24T08:12:56.751466+0000","last_unstale":"2023-11-24T08:12:58.483343+0000","last_undegraded":"2023-11-24T08:12:58.483343+0000","last_fullsized":"2023-11-24T08:12:58.483343+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:31:41.469233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.740654+0000","last_change":"2023-11-24T08:12:55.744601+0000","last_active":"2023-11-24T08:12:56.740654+0000","last_peered":"2023-11-24T08:12:56.740654+0000","last_clean":"2023-11-24T08:12:56.740654+0000","last_became_active":"2023-11-24T08:12:55.744306+0000","last_became_peered":"2023-11-24T08:12:55.744306+0000","last_unstale":"2023-11-24T08:12:56.740654+0000","last_undegraded":"2023-11-24T08:12:56.740654+0000","last_fullsized":"2023-11-24T08:12:56.740654+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:09:21.495024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480739+0000","last_change":"2023-11-24T08:12:56.751443+0000","last_active":"2023-11-24T08:12:58.480739+0000","last_peered":"2023-11-24T08:12:58.480739+0000","last_clean":"2023-11-24T08:12:58.480739+0000","last_became_active":"2023-11-24T08:12:56.750884+0000","last_became_peered":"2023-11-24T08:12:56.750884+0000","last_unstale":"2023-11-24T08:12:58.480739+0000","last_undegraded":"2023-11-24T08:12:58.480739+0000","last_fullsized":"2023-11-24T08:12:58.480739+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T18:06:45.358902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.726659+0000","last_change":"2023-11-24T08:12:55.735154+0000","last_active":"2023-11-24T08:12:56.726659+0000","last_peered":"2023-11-24T08:12:56.726659+0000","last_clean":"2023-11-24T08:12:56.726659+0000","last_became_active":"2023-11-24T08:12:55.734730+0000","last_became_peered":"2023-11-24T08:12:55.734730+0000","last_unstale":"2023-11-24T08:12:56.726659+0000","last_undegraded":"2023-11-24T08:12:56.726659+0000","last_fullsized":"2023-11-24T08:12:56.726659+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:59:36.915212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:57.745519+0000","last_change":"2023-11-24T08:12:56.741137+0000","last_active":"2023-11-24T08:12:57.745519+0000","last_peered":"2023-11-24T08:12:57.745519+0000","last_clean":"2023-11-24T08:12:57.745519+0000","last_became_active":"2023-11-24T08:12:56.734127+0000","last_became_peered":"2023-11-24T08:12:56.734127+0000","last_unstale":"2023-11-24T08:12:57.745519+0000","last_undegraded":"2023-11-24T08:12:57.745519+0000","last_fullsized":"2023-11-24T08:12:57.745519+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:05.187066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.736257+0000","last_change":"2023-11-24T08:12:55.744123+0000","last_active":"2023-11-24T08:12:56.736257+0000","last_peered":"2023-11-24T08:12:56.736257+0000","last_clean":"2023-11-24T08:12:56.736257+0000","last_became_active":"2023-11-24T08:12:55.743595+0000","last_became_peered":"2023-11-24T08:12:55.743595+0000","last_unstale":"2023-11-24T08:12:56.736257+0000","last_undegraded":"2023-11-24T08:12:56.736257+0000","last_fullsized":"2023-11-24T08:12:56.736257+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T14:54:54.103157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":11,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.482073+0000","last_change":"2023-11-24T08:12:56.753294+0000","last_active":"2023-11-24T08:12:58.482073+0000","last_peered":"2023-11-24T08:12:58.482073+0000","last_clean":"2023-11-24T08:12:58.482073+0000","last_became_active":"2023-11-24T08:12:56.752967+0000","last_became_peered":"2023-11-24T08:12:56.752967+0000","last_unstale":"2023-11-24T08:12:58.482073+0000","last_undegraded":"2023-11-24T08:12:58.482073+0000","last_fullsized":"2023-11-24T08:12:58.482073+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:55.692275+0000","last_clean_scrub_stamp":"2023-11-24T08:12:55.692275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:03:11.112483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"88'1","reported_seq":13,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.758601+0000","last_change":"2023-11-24T08:12:55.737285+0000","last_active":"2023-11-24T08:12:56.758601+0000","last_peered":"2023-11-24T08:12:56.758601+0000","last_clean":"2023-11-24T08:12:56.758601+0000","last_became_active":"2023-11-24T08:12:55.736961+0000","last_became_peered":"2023-11-24T08:12:55.736961+0000","last_unstale":"2023-11-24T08:12:56.758601+0000","last_undegraded":"2023-11-24T08:12:56.758601+0000","last_fullsized":"2023-11-24T08:12:56.758601+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:12:54.680581+0000","last_clean_scrub_stamp":"2023-11-24T08:12:54.680581+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T08:38:04.175371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"19'117","reported_seq":218,"reported_epoch":88,"state":"active+clean","last_fresh":"2023-11-24T08:12:56.729026+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:56.729026+0000","last_peered":"2023-11-24T08:12:56.729026+0000","last_clean":"2023-11-24T08:12:56.729026+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:56.729026+0000","last_undegraded":"2023-11-24T08:12:56.729026+0000","last_fullsized":"2023-11-24T08:12:56.729026+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542200,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1687552,"data_stored":1057347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.48},{"interface":"front","average":{"1min":1.196,"5min":1.073,"15min":1.0129999999999999},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8580000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.0900000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.75800000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.609},{"interface":"front","average":{"1min":1.196,"5min":1.181,"15min":1.1679999999999999},"min":{"1min":0.90500000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7150000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.2929999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.135,"15min":1.073},"min":{"1min":0.77300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.821,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.4059999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":0.89700000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.954,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.2170000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.234},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5860000000000001,"5min":3.387,"15min":3.387},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.276,"15min":1.23},"min":{"1min":0.98699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.865,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.367}]},{"osd":4,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.2210000000000001},"min":{"1min":0.93799999999999994,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9259999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.387,"5min":1.3939999999999999,"15min":1.377},"min":{"1min":0.97999999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.909,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.1479999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":0.91800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7909999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3959999999999999,"15min":1.3640000000000001},"min":{"1min":1.081,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.984,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.4410000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.542,"15min":1.4950000000000001},"min":{"1min":1.4510000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.073,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.569},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7909999999999999,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.321}]}]},{"osd":6,"up_from":14,"seq":60129542199,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":467603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4099999999999999,"5min":2.085,"15min":2.085},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.236,"5min":1.2110000000000001,"15min":1.236},"min":{"1min":0.745,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.179}]},{"osd":1,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.629,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.048,"5min":1.145,"15min":1.163},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.593,"5min":1.98,"15min":1.98},"last":1.286}]},{"osd":2,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4199999999999999,"5min":1.778,"15min":1.778},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6659999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.2270000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.335},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7250000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.548},{"interface":"front","average":{"1min":1.264,"5min":1.395,"15min":1.4330000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.9350000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3160000000000001,"15min":1.3600000000000001},"min":{"1min":0.71499999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.7709999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.49},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.581},"min":{"1min":0.84499999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1269999999999998,"5min":2.661,"15min":2.661},"last":2.1429999999999998}]},{"osd":5,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.6890000000000001},"min":{"1min":1.0900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0049999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.839},{"interface":"front","average":{"1min":1.419,"5min":1.538,"15min":1.6339999999999999},"min":{"1min":1.0580000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.752,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":2.0790000000000002}]},{"osd":7,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6930000000000001,"15min":1.7689999999999999},"min":{"1min":1.3460000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.2370000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.992},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.6759999999999999,"15min":1.7689999999999999},"min":{"1min":1.1439999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.748}]}]},{"osd":5,"up_from":14,"seq":60129542198,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1105920,"data_stored":468543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.337},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":1.0549999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.0069999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.1910000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.89300000000000002}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.153}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":1.405},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.431}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.248},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":1.4650000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.54},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.3819999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542199,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32244,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339596,"statfs":{"total":96636764160,"available":96603746304,"internally_reserved":0,"allocated":1089536,"data_stored":466909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0369999999999999,"15min":0.996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6180000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.034},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.74199999999999999,"5min":0.626,"15min":0.626},"max":{"1min":1.7909999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.171}]},{"osd":1,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.157,"15min":1.155},"min":{"1min":0.90300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2090000000000001,"15min":1.1699999999999999},"min":{"1min":0.81200000000000006,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6559999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.90100000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.88900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.746,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.79900000000000004,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.97899999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2869999999999999},"min":{"1min":0.748,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.399,"15min":1.415},"min":{"1min":0.96199999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.901,"5min":2.746,"15min":2.746},"last":1.0840000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2889999999999999,"15min":1.3149999999999999},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.919,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4690000000000001,"15min":1.4630000000000001},"min":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":0.94399999999999995}]},{"osd":6,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.411,"15min":1.4410000000000001},"min":{"1min":0.83999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.101,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.89200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.5600000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.526,"15min":1.516},"min":{"1min":1.0820000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.0059999999999998,"5min":2.548,"15min":2.548},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.5070000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.607}]}]},{"osd":3,"up_from":14,"seq":60129542199,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":468521,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97799999999999998,"15min":1.1220000000000001},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4139999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.88,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.56100000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":0.98299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.429,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0329999999999999,"15min":1.103},"min":{"1min":0.53300000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5740000000000001,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.76900000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.107,"15min":1.2030000000000001},"min":{"1min":0.57099999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.544,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.121,"5min":1.1140000000000001,"15min":1.145},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.5029999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.71199999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3149999999999999,"15min":1.4350000000000001},"min":{"1min":0.72399999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":0.92000000000000004},{"interface":"front","average":{"1min":1.202,"5min":1.2989999999999999,"15min":1.361},"min":{"1min":0.88,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6719999999999999,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.0329999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4470000000000001,"15min":1.548},"min":{"1min":0.92700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.829,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.149},{"interface":"front","average":{"1min":1.351,"5min":1.444,"15min":1.522},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7410000000000001,"5min":3.339,"15min":3.339},"last":0.83299999999999996}]},{"osd":6,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4750000000000001,"15min":1.637},"min":{"1min":0.89000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8440000000000001,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0840000000000001},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4850000000000001,"15min":1.5449999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0030000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.2390000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5449999999999999,"15min":1.655},"min":{"1min":1.077,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.105,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.214},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.6839999999999999,"15min":1.966},"min":{"1min":1.01,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0470000000000002,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.1879999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542199,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1093632,"data_stored":467491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.101},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.83099999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":1.01},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1819999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.89100000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.589},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.323}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.369}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":1.2749999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.6539999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.476}]}]},{"osd":1,"up_from":14,"seq":60129542199,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":467103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.476,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2809999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.56599999999999995},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0109999999999999},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.67300000000000004}]},{"osd":3,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1519999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6279999999999999,"5min":1.907,"15min":1.907},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.458,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.53700000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.167,"15min":1.165},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.91100000000000003},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.325},"min":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.0800000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.248,"15min":1.296},"min":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.661,"15min":2.661},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6819999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.0640000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.321,"15min":1.349},"min":{"1min":0.78900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.633,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3400000000000001,"15min":1.3340000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.1799999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3999999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.351,"15min":1.363},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.008,"15min":2.008},"last":1.163}]}]},{"osd":0,"up_from":14,"seq":60129542199,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1687552,"data_stored":1057311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.018},{"interface":"front","average":{"1min":1.006,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5069999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2130000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.48399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":0.82199999999999995}]},{"osd":3,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.05,"15min":1.0720000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4450000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.5329999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88800000000000001,"15min":0.95199999999999996},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.056}]},{"osd":4,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.591,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.135},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.29},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5880000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.357}]},{"osd":5,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1950000000000001,"15min":1.218},"min":{"1min":0.85799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.506,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.169},{"interface":"front","average":{"1min":1.252,"5min":1.2529999999999999,"15min":1.2669999999999999},"min":{"1min":0.79600000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.722,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2490000000000001,"15min":1.266},"min":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.512,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.2609999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2849999999999999,"15min":1.2869999999999999},"min":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9039999999999999,"5min":2.149,"15min":2.149},"last":1.395}]},{"osd":7,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.879,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7809999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.617},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6890000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.4259999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:01.653 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-24T08:13:01.653 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-24T08:13:01.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.913+0000 7f4fdea5d640 1 Processor -- start 2023-11-24T08:13:01.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.913+0000 7f4fdea5d640 1 -- start start 2023-11-24T08:13:01.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdea5d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f4fd806de10 0x7f4fd815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdea5d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4fd80c49f0 0x7f4fd80705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdea5d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd815ec90 con 0x7f4fd8072160 2023-11-24T08:13:01.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdea5d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd80cd530 con 0x7f4fd80c49f0 2023-11-24T08:13:01.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdea5d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd80c4ed0 con 0x7f4fd806de10 2023-11-24T08:13:01.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fd7fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f4fd8072160 0x7f4fd806d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37310/0 (socket says 172.21.15.106:37310) 2023-11-24T08:13:01.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fd7fff640 1 -- 172.21.15.106:0/3102514698 learned_addr learned my addr 172.21.15.106:0/3102514698 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:01.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdcffa640 1 --2- 172.21.15.106:0/3102514698 >> v2:172.21.15.106:3300/0 conn(0x7f4fd80c49f0 0x7f4fd80705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fcffff640 1 --2- 172.21.15.106:0/3102514698 >> v2:172.21.15.156:3300/0 conn(0x7f4fd806de10 0x7f4fd815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fd77fe640 1 -- 172.21.15.106:0/3102514698 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2910678692 0 0) 0x7f4fc8001110 con 0x7f4fd8072160 2023-11-24T08:13:01.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdcffa640 1 -- 172.21.15.106:0/3102514698 >> v2:172.21.15.156:3300/0 conn(0x7f4fd806de10 msgr2=0x7f4fd815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:01.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdcffa640 1 --2- 172.21.15.106:0/3102514698 >> v2:172.21.15.156:3300/0 conn(0x7f4fd806de10 0x7f4fd815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdcffa640 1 -- 172.21.15.106:0/3102514698 >> v1:172.21.15.106:6789/0 conn(0x7f4fd8072160 legacy=0x7f4fd806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.915+0000 7f4fdcffa640 1 -- 172.21.15.106:0/3102514698 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd80c51b0 con 0x7f4fd80c49f0 2023-11-24T08:13:01.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.917+0000 7f4fdcffa640 1 --2- 172.21.15.106:0/3102514698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80705c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80cef60 tx=0x7f4fd002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf8071acb553e2c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:01.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.917+0000 7f4fd77fe640 1 -- 172.21.15.106:0/3102514698 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd002cc80 con 0x7f4fd80c49f0 2023-11-24T08:13:01.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.917+0000 7f4fd77fe640 1 -- 172.21.15.106:0/3102514698 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fd002cde0 con 0x7f4fd80c49f0 2023-11-24T08:13:01.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.917+0000 7f4fd77fe640 1 -- 172.21.15.106:0/3102514698 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fd0034660 con 0x7f4fd80c49f0 2023-11-24T08:13:01.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 -- 172.21.15.106:0/3102514698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd80c49f0 msgr2=0x7f4fd80705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/3102514698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80705c0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f4fd80cef60 tx=0x7f4fd002bed0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 -- 172.21.15.106:0/3102514698 shutdown_connections 2023-11-24T08:13:01.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/3102514698 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80705c0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/3102514698 >> v2:172.21.15.156:3300/0 conn(0x7f4fd806de10 0x7f4fd815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 -- 172.21.15.106:0/3102514698 >> 172.21.15.106:0/3102514698 conn(0x7f4fd80b0c50 msgr2=0x7f4fd80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:01.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 -- 172.21.15.106:0/3102514698 shutdown_connections 2023-11-24T08:13:01.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.918+0000 7f4fdea5d640 1 -- 172.21.15.106:0/3102514698 wait complete. 2023-11-24T08:13:01.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.919+0000 7f4fdea5d640 1 Processor -- start 2023-11-24T08:13:01.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.920+0000 7f4fdea5d640 1 -- start start 2023-11-24T08:13:01.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.920+0000 7f4fdea5d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 0x7f4fd814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdea5d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fd8072160 0x7f4fd80b1d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdea5d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdea5d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd814e9d0 con 0x7f4fd806de10 2023-11-24T08:13:01.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdea5d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd80b2250 con 0x7f4fd80c49f0 2023-11-24T08:13:01.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdea5d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd809ac30 con 0x7f4fd8072160 2023-11-24T08:13:01.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 0x7f4fd814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 0x7f4fd814e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38688/0 (socket says 172.21.15.106:38688) 2023-11-24T08:13:01.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 -- 172.21.15.106:0/1568307833 learned_addr learned my addr 172.21.15.106:0/1568307833 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdcffa640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fcffff640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fd8072160 0x7f4fd80b1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 -- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fd8072160 msgr2=0x7f4fd80b1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fd8072160 0x7f4fd80b1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 -- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80c49f0 msgr2=0x7f4fd80aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fd7fff640 1 -- 172.21.15.106:0/1568307833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd0009a70 con 0x7f4fd806de10 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.921+0000 7f4fdcffa640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.922+0000 7f4fd7fff640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 0x7f4fd814e490 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800c7b0 tx=0x7f4fc800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:01.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.922+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80072d0 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.922+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc8007450 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.922+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc80047e0 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.922+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd809af10 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.922+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd809b4d0 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.923+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd809b6f0 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.924+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f4fc8004960 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.925+0000 7f4fd57fa640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4fb806b3e0 0x7f4fb806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.925+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fc8089090 con 0x7f4fd806de10 2023-11-24T08:13:01.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.925+0000 7f4fcffff640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4fb806b3e0 0x7f4fb806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:01.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.926+0000 7f4fcffff640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4fb806b3e0 0x7f4fb806d8a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4fd806c5a0 tx=0x7f4fc0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:01.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:01.930+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f4fc80538d0 con 0x7f4fd806de10 2023-11-24T08:13:02.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.187+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4fd815ec90 con 0x7f4fd806de10 2023-11-24T08:13:02.193 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.192+0000 7f4fd57fa640 1 -- 172.21.15.106:0/1568307833 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4fd815ec90 con 0x7f4fd806de10 2023-11-24T08:13:02.194 INFO:teuthology.orchestra.run.smithi106.stdout:[client.0] 2023-11-24T08:13:02.194 INFO:teuthology.orchestra.run.smithi106.stdout: key = AQDWWWBlnwRKEhAAAO4NguxJfOUpSmg5YT7uzQ== 2023-11-24T08:13:02.194 INFO:teuthology.orchestra.run.smithi106.stdout: caps mds = "allow" 2023-11-24T08:13:02.194 INFO:teuthology.orchestra.run.smithi106.stdout: caps mon = "allow r" 2023-11-24T08:13:02.195 INFO:teuthology.orchestra.run.smithi106.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2023-11-24T08:13:02.195 INFO:teuthology.orchestra.run.smithi106.stderr:updated caps for client.0 2023-11-24T08:13:02.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4fb806b3e0 msgr2=0x7f4fb806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4fb806b3e0 0x7f4fb806d8a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f4fd806c5a0 tx=0x7f4fc0008040 comp rx=0 tx=0).stop 2023-11-24T08:13:02.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 msgr2=0x7f4fd814e490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 0x7f4fd814e490 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800c7b0 tx=0x7f4fc800cc70 comp rx=0 tx=0).stop 2023-11-24T08:13:02.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 shutdown_connections 2023-11-24T08:13:02.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fd80c49f0 0x7f4fd80aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f4fb806b3e0 0x7f4fb806d8a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4fd8072160 0x7f4fd80b1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 --2- 172.21.15.106:0/1568307833 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4fd806de10 0x7f4fd814e490 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 >> 172.21.15.106:0/1568307833 conn(0x7f4fd80b0c50 msgr2=0x7f4fd80abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:02.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 shutdown_connections 2023-11-24T08:13:02.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.195+0000 7f4fdea5d640 1 -- 172.21.15.106:0/1568307833 wait complete. 2023-11-24T08:13:02.217 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:02.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.432+0000 7fb8d557a640 1 Processor -- start 2023-11-24T08:13:02.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.432+0000 7fb8d557a640 1 -- start start 2023-11-24T08:13:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.433+0000 7fb8d557a640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb8d0144b30 0x7fb8d015f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8d557a640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d015bf80 con 0x7fb8d00b1de0 2023-11-24T08:13:02.444 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8d557a640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d00b1560 con 0x7fb8d015c410 2023-11-24T08:13:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8d557a640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d00b1280 con 0x7fb8d0144b30 2023-11-24T08:13:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8d4d79640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb8d00b1de0 0x7fb8d00b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37318/0 (socket says 172.21.15.106:37318) 2023-11-24T08:13:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8d4d79640 1 -- 172.21.15.106:0/3453091806 learned_addr learned my addr 172.21.15.106:0/3453091806 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8cffff640 1 --2- 172.21.15.106:0/3453091806 >> v2:172.21.15.156:3300/0 conn(0x7fb8d0144b30 0x7fb8d015f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:02.445 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822003756 0 0) 0x7fb8d015bf80 con 0x7fb8d00b1de0 2023-11-24T08:13:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.434+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8bc003520 con 0x7fb8d00b1de0 2023-11-24T08:13:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1102745541 0 0) 0x7fb8bc003520 con 0x7fb8d00b1de0 2023-11-24T08:13:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 >> v2:172.21.15.156:3300/0 conn(0x7fb8d0144b30 msgr2=0x7fb8d015f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 --2- 172.21.15.106:0/3453091806 >> v2:172.21.15.156:3300/0 conn(0x7fb8d0144b30 0x7fb8d015f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.446 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 >> v1:172.21.15.156:6789/0 conn(0x7fb8d015c410 legacy=0x7fb8d00b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:02.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d015b930 con 0x7fb8d00b1de0 2023-11-24T08:13:02.447 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fb8c80019f0 con 0x7fb8d00b1de0 2023-11-24T08:13:02.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 >> v1:172.21.15.106:6789/0 conn(0x7fb8d00b1de0 legacy=0x7fb8d00b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc003970 0x7fb8bc003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0047c0 0x7fb8bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8bc005180 0x7fb8bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00b1280 con 0x7fb8bc0047c0 2023-11-24T08:13:02.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc00c0b0 con 0x7fb8bc003970 2023-11-24T08:13:02.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc00c270 con 0x7fb8bc005180 2023-11-24T08:13:02.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0047c0 0x7fb8bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:02.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8cffff640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc003970 0x7fb8bc003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:02.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8c7fff640 1 -- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8bc005180 msgr2=0x7fb8bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:02.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8bc005180 0x7fb8bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8c7fff640 1 -- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc003970 msgr2=0x7fb8bc003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:02.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.435+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc003970 0x7fb8bc003d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.436+0000 7fb8c7fff640 1 -- 172.21.15.106:0/3453091806 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8bc003520 con 0x7fb8bc0047c0 2023-11-24T08:13:02.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.436+0000 7fb8d4d79640 1 -- 172.21.15.106:0/3453091806 reap_dead start 2023-11-24T08:13:02.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.437+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0047c0 0x7fb8bc004c40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8003d80 tx=0x7fb8bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62f5cc13e27ca5e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:02.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.437+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b8032600 con 0x7fb8bc0047c0 2023-11-24T08:13:02.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.437+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b802e050 con 0x7fb8bc0047c0 2023-11-24T08:13:02.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.437+0000 7fb8cf7fe640 1 -- 172.21.15.106:0/3453091806 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b80329e0 con 0x7fb8bc0047c0 2023-11-24T08:13:02.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.437+0000 7fb8d557a640 1 -- 172.21.15.106:0/3453091806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0047c0 msgr2=0x7fb8bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.437+0000 7fb8d557a640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0047c0 0x7fb8bc004c40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8003d80 tx=0x7fb8bc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.438+0000 7fb8d557a640 1 -- 172.21.15.106:0/3453091806 shutdown_connections 2023-11-24T08:13:02.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.438+0000 7fb8d557a640 1 --2- 172.21.15.106:0/3453091806 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8bc0047c0 0x7fb8bc004c40 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.438+0000 7fb8d557a640 1 -- 172.21.15.106:0/3453091806 >> 172.21.15.106:0/3453091806 conn(0x7fb8d00cb830 msgr2=0x7fb8d00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:02.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.438+0000 7fb8d557a640 1 -- 172.21.15.106:0/3453091806 shutdown_connections 2023-11-24T08:13:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.438+0000 7fb8d557a640 1 -- 172.21.15.106:0/3453091806 wait complete. 2023-11-24T08:13:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.439+0000 7fb8d557a640 1 Processor -- start 2023-11-24T08:13:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 -- start start 2023-11-24T08:13:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc0047c0 0x7fb8d0154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 0x7fb8d01653a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.453 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8d006c5b0 0x7fb8d006ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d006cfa0 con 0x7fb8d0164f40 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0163ca0 con 0x7fb8bc0047c0 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.440+0000 7fb8d557a640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0164130 con 0x7fb8d006c5b0 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 0x7fb8d01653a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 0x7fb8d01653a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38710/0 (socket says 172.21.15.106:38710) 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 -- 172.21.15.106:0/1472198411 learned_addr learned my addr 172.21.15.106:0/1472198411 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:02.454 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8cffff640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc0047c0 0x7fb8d0154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 -- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8d006c5b0 msgr2=0x7fb8d006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8d006c5b0 0x7fb8d006ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 -- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc0047c0 msgr2=0x7fb8d0154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc0047c0 0x7fb8d0154f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 -- 172.21.15.106:0/1472198411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0155480 con 0x7fb8d0164f40 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.441+0000 7fb8c7fff640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 0x7fb8d01653a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000c00 tx=0x7fb8b80036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:02.455 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.442+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803c410 con 0x7fb8d0164f40 2023-11-24T08:13:02.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.442+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8b803c570 con 0x7fb8d0164f40 2023-11-24T08:13:02.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.442+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8b803c8e0 con 0x7fb8d0164f40 2023-11-24T08:13:02.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.442+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d0163090 con 0x7fb8d0164f40 2023-11-24T08:13:02.456 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.442+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0163650 con 0x7fb8d0164f40 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.443+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d004e840 con 0x7fb8d0164f40 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.444+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb8b8039050 con 0x7fb8d0164f40 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.445+0000 7fb8cd7fa640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb8b006b410 0x7fb8b006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.445+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8b80ab4e0 con 0x7fb8d0164f40 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.445+0000 7fb8cffff640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb8b006b410 0x7fb8b006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.446+0000 7fb8cffff640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb8b006b410 0x7fb8b006d8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0006fd0 tx=0x7fb8c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:02.457 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.449+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb8b8075e30 con 0x7fb8d0164f40 2023-11-24T08:13:02.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.674+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8d0163de0 con 0x7fb8d0164f40 2023-11-24T08:13:02.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.675+0000 7fb8cd7fa640 1 -- 172.21.15.106:0/1472198411 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fb8b8037090 con 0x7fb8d0164f40 2023-11-24T08:13:02.676 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:02.676 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":53,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:56.174099+0000","modified":"2023-11-24T08:12:59.244264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1896010072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1896010072},{"type":"v1","addr":"172.21.15.106:6837","nonce":1896010072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:13:02.677 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 53 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb8b006b410 msgr2=0x7fb8b006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb8b006b410 0x7fb8b006d8d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb8c0006fd0 tx=0x7fb8c0008040 comp rx=0 tx=0).stop 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 msgr2=0x7fb8d01653a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 0x7fb8d01653a0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8000c00 tx=0x7fb8b80036f0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 shutdown_connections 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb8b006b410 0x7fb8b006d8d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb8d006c5b0 0x7fb8d006ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.679 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb8d0164f40 0x7fb8d01653a0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 --2- 172.21.15.106:0/1472198411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8bc0047c0 0x7fb8d0154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:02.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.677+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 >> 172.21.15.106:0/1472198411 conn(0x7fb8d00cb830 msgr2=0x7fb8d00cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:02.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.678+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 shutdown_connections 2023-11-24T08:13:02.680 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:02.678+0000 7fb8d557a640 1 -- 172.21.15.106:0/1472198411 wait complete. 2023-11-24T08:13:02.687 INFO:tasks.ceph.mds.a.smithi106.stderr:2023-11-24T08:13:02.686+0000 7f698c883640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:13:03.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 53, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-24T08:12:56.174099+0000', 'modified': '2023-11-24T08:12:59.244264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7020}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7020': {'gid': 7020, 'name': 'a', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.106:6837/1896010072', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.106:6835', 'nonce': 1896010072}, {'type': 'v1', 'addr': '172.21.15.106:6837', 'nonce': 1896010072}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-24T08:13:03.102 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-24T08:13:03.102 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-24T08:13:03.329 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.328+0000 7f37a73fb640 1 Processor -- start 2023-11-24T08:13:03.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.328+0000 7f37a73fb640 1 -- start start 2023-11-24T08:13:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f37a73fb640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37a00cc0f0 0x7f37a00c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f37a73fb640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a015eea0 con 0x7f37a015b440 2023-11-24T08:13:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f37a73fb640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a00abf30 con 0x7f37a00c57e0 2023-11-24T08:13:03.331 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f37a73fb640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37a015b820 con 0x7f37a00cc0f0 2023-11-24T08:13:03.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f37a5197640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f37a015b440 0x7f37a00cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37328/0 (socket says 172.21.15.106:37328) 2023-11-24T08:13:03.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f37a5197640 1 -- 172.21.15.106:0/1762944695 learned_addr learned my addr 172.21.15.106:0/1762944695 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.330+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 958284137 0 0) 0x7f37a015eea0 con 0x7f37a015b440 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f37a4996640 1 --2- 172.21.15.106:0/1762944695 >> v2:172.21.15.156:3300/0 conn(0x7f37a00cc0f0 0x7f37a00c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f378c003520 con 0x7f37a015b440 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 345858955 0 0) 0x7f378c003520 con 0x7f37a015b440 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 >> v2:172.21.15.156:3300/0 conn(0x7f37a00cc0f0 msgr2=0x7f37a00c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 --2- 172.21.15.106:0/1762944695 >> v2:172.21.15.156:3300/0 conn(0x7f37a00cc0f0 0x7f37a00c52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 >> v1:172.21.15.156:6789/0 conn(0x7f37a00c57e0 legacy=0x7f37a006c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a015f0f0 con 0x7f37a015b440 2023-11-24T08:13:03.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f379c001670 con 0x7f37a015b440 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 >> v1:172.21.15.106:6789/0 conn(0x7f37a015b440 legacy=0x7f37a00cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378c003970 0x7f378c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378c0047c0 0x7f378c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c0052a0 0x7f378c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a015b820 con 0x7f378c003970 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f378c00c170 con 0x7f378c0047c0 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f378c00c330 con 0x7f378c0052a0 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.331+0000 7f37a5197640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378c003970 0x7f378c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a4996640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378c0047c0 0x7f378c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a5197640 1 -- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c0052a0 msgr2=0x7f378c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a5197640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c0052a0 0x7f378c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.335 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a5197640 1 -- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378c0047c0 msgr2=0x7f378c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a5197640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f378c0047c0 0x7f378c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a5197640 1 -- 172.21.15.106:0/1762944695 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c003520 con 0x7f378c003970 2023-11-24T08:13:03.336 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.332+0000 7f37a5998640 1 -- 172.21.15.106:0/1762944695 reap_dead start 2023-11-24T08:13:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.333+0000 7f37a5197640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378c003970 0x7f378c003d90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c210 tx=0x7f378c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d93602356ced4e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.333+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c005ad0 con 0x7f378c003970 2023-11-24T08:13:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.333+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c005c30 con 0x7f378c003970 2023-11-24T08:13:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.333+0000 7f3797fff640 1 -- 172.21.15.106:0/1762944695 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c02cab0 con 0x7f378c003970 2023-11-24T08:13:03.337 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 -- 172.21.15.106:0/1762944695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378c003970 msgr2=0x7f378c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378c003970 0x7f378c003d90 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c210 tx=0x7f378c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 -- 172.21.15.106:0/1762944695 shutdown_connections 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 --2- 172.21.15.106:0/1762944695 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f378c003970 0x7f378c003d90 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 -- 172.21.15.106:0/1762944695 >> 172.21.15.106:0/1762944695 conn(0x7f37a00b0c50 msgr2=0x7f37a00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 -- 172.21.15.106:0/1762944695 shutdown_connections 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.334+0000 7f37a73fb640 1 -- 172.21.15.106:0/1762944695 wait complete. 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.336+0000 7f37a73fb640 1 Processor -- start 2023-11-24T08:13:03.338 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.336+0000 7f37a73fb640 1 -- start start 2023-11-24T08:13:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a73fb640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c003970 0x7f37a014e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a73fb640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a0154730 0x7f37a01f25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.340 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a73fb640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 0x7f37a0153a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a73fb640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a01f2fb0 con 0x7f37a01f2b30 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a73fb640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a015ac90 con 0x7f37a0154730 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a73fb640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a015a680 con 0x7f378c003970 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a5998640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 0x7f37a0153a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a5998640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 0x7f37a0153a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38724/0 (socket says 172.21.15.106:38724) 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.337+0000 7f37a5998640 1 -- 172.21.15.106:0/2596862677 learned_addr learned my addr 172.21.15.106:0/2596862677 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a4996640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a0154730 0x7f37a01f25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.341 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5998640 1 -- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c003970 msgr2=0x7f37a014e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5197640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c003970 0x7f37a014e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5998640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c003970 0x7f37a014e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5998640 1 -- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a0154730 msgr2=0x7f37a01f25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5998640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a0154730 0x7f37a01f25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5998640 1 -- 172.21.15.106:0/2596862677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a01f3190 con 0x7f37a01f2b30 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a4996640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a0154730 0x7f37a01f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a5998640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 0x7f37a0153a20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f37a009d280 tx=0x7f379800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f3795ffb640 1 -- 172.21.15.106:0/2596862677 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3798005d40 con 0x7f37a01f2b30 2023-11-24T08:13:03.342 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f3795ffb640 1 -- 172.21.15.106:0/2596862677 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379800a610 con 0x7f37a01f2b30 2023-11-24T08:13:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f3795ffb640 1 -- 172.21.15.106:0/2596862677 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379800a9c0 con 0x7f37a01f2b30 2023-11-24T08:13:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a015a920 con 0x7f37a01f2b30 2023-11-24T08:13:03.343 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.338+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a0153f60 con 0x7f37a01f2b30 2023-11-24T08:13:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.341+0000 7f3795ffb640 1 -- 172.21.15.106:0/2596862677 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f379800ab20 con 0x7f37a01f2b30 2023-11-24T08:13:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.342+0000 7f3795ffb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f378406b410 0x7f378406d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.342+0000 7f3795ffb640 1 -- 172.21.15.106:0/2596862677 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3798088f60 con 0x7f37a01f2b30 2023-11-24T08:13:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.342+0000 7f37a5197640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f378406b410 0x7f378406d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.342+0000 7f37a9f8d740 1 Processor -- start 2023-11-24T08:13:03.344 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.343+0000 7f37a9f8d740 1 -- start start 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.343+0000 7f37a5197640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f378406b410 0x7f378406d8d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f379c034000 tx=0x7f379c009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.343+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 0x56441053c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.343+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564410536ba0 0x56441053b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x56441057f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441049a880 con 0x56441053bea0 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644104a9ae0 con 0x56441053b5e0 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644104a71d0 con 0x564410536ba0 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 0x56441053c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 0x56441053c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38728/0 (socket says 172.21.15.106:38728) 2023-11-24T08:13:03.346 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 -- 172.21.15.106:0/316680554 learned_addr learned my addr 172.21.15.106:0/316680554 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a4996640 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564410536ba0 0x56441053b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 -- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564410536ba0 msgr2=0x56441053b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564410536ba0 0x56441053b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5998640 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x56441057f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 -- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 msgr2=0x56441057f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x56441057f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a5197640 1 -- 172.21.15.106:0/316680554 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56441046a860 con 0x56441053bea0 2023-11-24T08:13:03.347 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.344+0000 7f37a4996640 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564410536ba0 0x56441053b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.345+0000 7f37a5197640 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 0x56441053c280 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c300 tx=0x7f379c007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2daf50116aa6cbb8 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.345+0000 7f3782ffd640 1 -- 172.21.15.106:0/316680554 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c03c040 con 0x56441053bea0 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.345+0000 7f3782ffd640 1 -- 172.21.15.106:0/316680554 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c004e20 con 0x56441053bea0 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.345+0000 7f3782ffd640 1 -- 172.21.15.106:0/316680554 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c001670 con 0x56441053bea0 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 -- 172.21.15.106:0/316680554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 msgr2=0x56441053c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 0x56441053c280 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c300 tx=0x7f379c007000 comp rx=0 tx=0).stop 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 -- 172.21.15.106:0/316680554 shutdown_connections 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x56441057f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x564410536ba0 0x56441053b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/316680554 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053bea0 0x56441053c280 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.349 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 -- 172.21.15.106:0/316680554 >> 172.21.15.106:0/316680554 conn(0x5644104fdc70 msgr2=0x564410581990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 -- 172.21.15.106:0/316680554 shutdown_connections 2023-11-24T08:13:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.346+0000 7f37a9f8d740 1 -- 172.21.15.106:0/316680554 wait complete. 2023-11-24T08:13:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.347+0000 7f37a9f8d740 1 -- start start 2023-11-24T08:13:03.350 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.347+0000 7f37a9f8d740 1 Processor -- start 2023-11-24T08:13:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x56441055a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x5644105858e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410588650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644104a2ac0 con 0x564410536ba0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56441043f2f0 con 0x56441053b5e0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x56441045cc70 con 0x56441053bea0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a9f8d740 10 client.0 _ll_register_callbacks cb 0x5644104613b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a5197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x56441055a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a5197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x56441055a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38734/0 (socket says 172.21.15.106:38734) 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.348+0000 7f37a5197640 1 -- 172.21.15.106:0/3755593211 learned_addr learned my addr 172.21.15.106:0/3755593211 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a4996640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x5644105858e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5998640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410588650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5197640 1 -- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 msgr2=0x564410588650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5197640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410588650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5197640 1 -- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 msgr2=0x5644105858e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5197640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x5644105858e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5197640 1 -- 172.21.15.106:0/3755593211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f378c003520 con 0x564410536ba0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f37a5197640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x56441055a6f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f379c007f30 tx=0x7f379c02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab961b4069589f57 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.349+0000 7f377bfff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f377bfff640 1 -- 172.21.15.106:0/3755593211 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c03c050 con 0x564410536ba0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f377bfff640 1 -- 172.21.15.106:0/3755593211 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f379c007640 con 0x564410536ba0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56441049baa0 con 0x564410536ba0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f377bfff640 1 -- 172.21.15.106:0/3755593211 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f379c053070 con 0x564410536ba0 2023-11-24T08:13:03.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f37a9f8d740 10 client.7170 fetch_fsmap learned FSMap version 53 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x5644104a1140 con 0x564410536ba0 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f377bfff640 1 -- 172.21.15.106:0/3755593211 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56441049baa0 con 0x564410536ba0 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f377bfff640 1 -- 172.21.15.106:0/3755593211 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f379c03f030 con 0x564410536ba0 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f37a9f8d740 10 client.7170 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.350+0000 7f37a9f8d740 10 client.7170 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f37a9f8d740 20 client.7170 populate_metadata read hostname 'smithi106' 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410583d80 0x564410584160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f37a9f8d740 4 client.7170 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 --> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56441049b7c0 con 0x564410583d80 2023-11-24T08:13:03.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f37a4996640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410583d80 0x564410584160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f37a4996640 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410583d80 0x564410584160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4e2d7ccde030efe server_cookie=de1744c91af0730c in_seq=0 out_seq=0 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.351+0000 7f377bfff640 10 client.7170 ms_handle_connect on v2:172.21.15.106:6835/1896010072 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.352+0000 7f377bfff640 1 -- 172.21.15.106:0/3755593211 <== mds.0 v2:172.21.15.106:6835/1896010072 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56441049b7c0 con 0x564410583d80 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.352+0000 7f377bfff640 10 client.7170 handle_command_reply: tid=0 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.352+0000 7f37a9f8d740 1 client.7170 shutdown 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.352+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 msgr2=0x56441055a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.352+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x56441055a6f0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f379c007f30 tx=0x7f379c02e070 comp rx=0 tx=0).stop 2023-11-24T08:13:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 shutdown_connections 2023-11-24T08:13:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410583d80 0x564410584160 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410588650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x5644105858e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f377bfff640 0 client.7170 ms_handle_reset on v2:172.21.15.106:6835/1896010072 2023-11-24T08:13:03.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/3755593211 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x56441055a6f0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 >> 172.21.15.106:0/3755593211 conn(0x5644104fdc70 msgr2=0x564410581990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 shutdown_connections 2023-11-24T08:13:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 1 -- 172.21.15.106:0/3755593211 wait complete. 2023-11-24T08:13:03.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.353+0000 7f37a9f8d740 20 client.7170 trim_cache size 0 max 16384 2023-11-24T08:13:03.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.370+0000 7f37a9f8d740 1 Processor -- start 2023-11-24T08:13:03.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.370+0000 7f37a9f8d740 1 -- start start 2023-11-24T08:13:03.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x564410586f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x564410587460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410562600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x564410205f20 con 0x564410536ba0 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564410456e30 con 0x56441053b5e0 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564410584e90 con 0x56441053bea0 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a5197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x564410586f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a5197640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x564410586f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38736/0 (socket says 172.21.15.106:38736) 2023-11-24T08:13:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a5197640 1 -- 172.21.15.106:0/4073893688 learned_addr learned my addr 172.21.15.106:0/4073893688 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a5197640 1 -- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 msgr2=0x564410562600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a5197640 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410562600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a5197640 1 -- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 msgr2=0x564410587460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.371+0000 7f37a4996640 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x564410587460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37a5197640 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x564410587460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37a5197640 1 -- 172.21.15.106:0/4073893688 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56441043eb80 con 0x564410536ba0 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37a4996640 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x564410587460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37a5197640 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x564410586f20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f379c005ec0 tx=0x7f379c053000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6456b1095da876c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37837fe640 1 -- 172.21.15.106:0/4073893688 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c03c070 con 0x564410536ba0 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37837fe640 1 -- 172.21.15.106:0/4073893688 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c035700 con 0x564410536ba0 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.372+0000 7f37837fe640 1 -- 172.21.15.106:0/4073893688 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379c035a70 con 0x564410536ba0 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 -- 172.21.15.106:0/4073893688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 msgr2=0x564410586f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x564410586f20 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f379c005ec0 tx=0x7f379c053000 comp rx=0 tx=0).stop 2023-11-24T08:13:03.378 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 -- 172.21.15.106:0/4073893688 shutdown_connections 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410562600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56441053b5e0 0x564410587460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/4073893688 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x564410536ba0 0x564410586f20 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 -- 172.21.15.106:0/4073893688 >> 172.21.15.106:0/4073893688 conn(0x5644104fdc70 msgr2=0x56441053c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 -- 172.21.15.106:0/4073893688 shutdown_connections 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 -- 172.21.15.106:0/4073893688 wait complete. 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.373+0000 7f37a9f8d740 1 -- start start 2023-11-24T08:13:03.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.374+0000 7f37a9f8d740 1 Processor -- start 2023-11-24T08:13:03.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564410536ba0 0x564410581220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 0x5644105eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410563390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x564410565a50 con 0x56441053b5e0 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644105ef210 con 0x564410536ba0 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x564410568660 con 0x56441053bea0 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a9f8d740 10 client.0 _ll_register_callbacks cb 0x56441053b530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 0x5644105eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 0x5644105eecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38742/0 (socket says 172.21.15.106:38742) 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 -- 172.21.15.106:0/2953804364 learned_addr learned my addr 172.21.15.106:0/2953804364 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:03.383 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 -- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 msgr2=0x564410563390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a5197640 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564410536ba0 0x564410581220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410563390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 -- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564410536ba0 msgr2=0x564410581220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564410536ba0 0x564410581220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.375+0000 7f37a4996640 1 -- 172.21.15.106:0/2953804364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644104a1140 con 0x56441053b5e0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a4996640 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 0x5644105eecd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3790000c00 tx=0x7f379000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71698f50ccdb9cfe server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f377bfff640 10 client.0 ms_handle_connect on v2:172.21.15.106:3300/0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f377bfff640 1 -- 172.21.15.106:0/2953804364 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379001b340 con 0x56441053b5e0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f377bfff640 1 -- 172.21.15.106:0/2953804364 <== mon.0 v2:172.21.15.106:3300/0 2 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f379001b4a0 con 0x56441053b5e0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f379c004e20 con 0x56441053b5e0 2023-11-24T08:13:03.384 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f377bfff640 1 -- 172.21.15.106:0/2953804364 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f379001b660 con 0x56441053b5e0 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 10 client.7176 fetch_fsmap learned FSMap version 53 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) v3 -- 0x564410464170 con 0x56441053b5e0 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f377bfff640 1 -- 172.21.15.106:0/2953804364 <== mon.0 v2:172.21.15.106:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379c004e20 con 0x56441053b5e0 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f377bfff640 1 -- 172.21.15.106:0/2953804364 <== mon.0 v2:172.21.15.106:3300/0 5 ==== fsmap(e 53) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3790003610 con 0x56441053b5e0 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 10 client.7176 fetch_fsmap finished waiting for FSMap version 53 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 10 client.7176 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 20 client.7176 populate_metadata read hostname 'smithi106' 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410536050 0x564410536430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 4 client.7176 mds_command: new command op to 7020 tid=0 multi_id=0 [{"prefix": "status"}] 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.376+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 --> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5644101a36c0 con 0x564410536050 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.377+0000 7f37a5197640 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410536050 0x564410536430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.377+0000 7f37a5197640 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410536050 0x564410536430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7794cf31b7360a1 server_cookie=ab61b5cd8a4972e3 in_seq=0 out_seq=0 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.377+0000 7f377bfff640 10 client.7176 ms_handle_connect on v2:172.21.15.106:6835/1896010072 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.377+0000 7f377bfff640 1 -- 172.21.15.106:0/2953804364 <== mds.0 v2:172.21.15.106:6835/1896010072 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5644101a36c0 con 0x564410536050 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f377bfff640 10 client.7176 handle_command_reply: tid=0 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 client.7176 shutdown 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 msgr2=0x5644105eecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 0x5644105eecd0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3790000c00 tx=0x7f379000eee0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 shutdown_connections 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:6835/1896010072,v1:172.21.15.106:6837/1896010072] conn(0x564410536050 0x564410536430 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x56441053bea0 0x564410563390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x56441053b5e0 0x5644105eecd0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f377bfff640 0 client.7176 ms_handle_reset on v2:172.21.15.106:6835/1896010072 2023-11-24T08:13:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 --2- 172.21.15.106:0/2953804364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564410536ba0 0x564410581220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 >> 172.21.15.106:0/2953804364 conn(0x5644104fdc70 msgr2=0x56441057e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 shutdown_connections 2023-11-24T08:13:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 1 -- 172.21.15.106:0/2953804364 wait complete. 2023-11-24T08:13:03.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.378+0000 7f37a9f8d740 20 client.7176 trim_cache size 0 max 16384 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "cluster_fsid": "d1a897c2-15ce-4272-ad66-4b3723970cb1", 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "whoami": 0, 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "id": 7020, 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "want_state": "up:active", 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "state": "up:active", 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "fs_name": "cephfs", 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "rank_uptime": 6.6661001110000004, 2023-11-24T08:13:03.388 INFO:teuthology.orchestra.run.smithi106.stdout: "mdsmap_epoch": 53, 2023-11-24T08:13:03.389 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch": 89, 2023-11-24T08:13:03.389 INFO:teuthology.orchestra.run.smithi106.stdout: "osdmap_epoch_barrier": 89, 2023-11-24T08:13:03.389 INFO:teuthology.orchestra.run.smithi106.stdout: "uptime": 12.061402315 2023-11-24T08:13:03.389 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f378406b410 msgr2=0x7f378406d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f378406b410 0x7f378406d8d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f379c034000 tx=0x7f379c009030 comp rx=0 tx=0).stop 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 msgr2=0x7f37a0153a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 0x7f37a0153a20 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f37a009d280 tx=0x7f379800cb90 comp rx=0 tx=0).stop 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 shutdown_connections 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37a01f2b30 0x7f37a0153a20 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37a0154730 0x7f37a01f25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f378406b410 0x7f378406d8d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 --2- 172.21.15.106:0/2596862677 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f378c003970 0x7f37a014e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 >> 172.21.15.106:0/2596862677 conn(0x7f37a00b0c50 msgr2=0x7f37a0071b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 shutdown_connections 2023-11-24T08:13:03.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.379+0000 7f37a73fb640 1 -- 172.21.15.106:0/2596862677 wait complete. 2023-11-24T08:13:03.399 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:13:03.611 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.610+0000 7f8b4b163640 1 Processor -- start 2023-11-24T08:13:03.612 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.610+0000 7f8b4b163640 1 -- start start 2023-11-24T08:13:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4b163640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f8b44144b30 0x7f8b4415f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.613 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4b163640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4415bf80 con 0x7f8b440b1de0 2023-11-24T08:13:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4b163640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b440b1560 con 0x7f8b4415c410 2023-11-24T08:13:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4b163640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b440b1280 con 0x7f8b44144b30 2023-11-24T08:13:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4a962640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f8b440b1de0 0x7f8b440b3c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:37342/0 (socket says 172.21.15.106:37342) 2023-11-24T08:13:03.614 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4a962640 1 -- 172.21.15.106:0/3425693156 learned_addr learned my addr 172.21.15.106:0/3425693156 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:03.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.612+0000 7f8b4a161640 1 --2- 172.21.15.106:0/3425693156 >> v2:172.21.15.156:3300/0 conn(0x7f8b44144b30 0x7f8b4415f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1262836741 0 0) 0x7f8b4415bf80 con 0x7f8b440b1de0 2023-11-24T08:13:03.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b30003520 con 0x7f8b440b1de0 2023-11-24T08:13:03.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1891486376 0 0) 0x7f8b30003520 con 0x7f8b440b1de0 2023-11-24T08:13:03.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 >> v2:172.21.15.156:3300/0 conn(0x7f8b44144b30 msgr2=0x7f8b4415f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 --2- 172.21.15.106:0/3425693156 >> v2:172.21.15.156:3300/0 conn(0x7f8b44144b30 0x7f8b4415f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 >> v1:172.21.15.156:6789/0 conn(0x7f8b4415c410 legacy=0x7f8b440b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b4415b930 con 0x7f8b440b1de0 2023-11-24T08:13:03.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f8b3c0019f0 con 0x7f8b440b1de0 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 >> v1:172.21.15.106:6789/0 conn(0x7f8b440b1de0 legacy=0x7f8b440b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30003970 0x7f8b30003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b300047c0 0x7f8b30004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b30005180 0x7f8b3000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440b1280 con 0x7f8b30005180 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3000c020 con 0x7f8b300047c0 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3000c1e0 con 0x7f8b30003970 2023-11-24T08:13:03.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.613+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b30005180 0x7f8b3000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b4a962640 1 -- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30003970 msgr2=0x7f8b30003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b49960640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b300047c0 0x7f8b30004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30003970 0x7f8b30003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b4a962640 1 -- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b300047c0 msgr2=0x7f8b30004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b300047c0 0x7f8b30004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b4a962640 1 -- 172.21.15.106:0/3425693156 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b30003520 con 0x7f8b30005180 2023-11-24T08:13:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.614+0000 7f8b49960640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b300047c0 0x7f8b30004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.615+0000 7f8b4a962640 1 -- 172.21.15.106:0/3425693156 reap_dead start 2023-11-24T08:13:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.615+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b30005180 0x7f8b3000b980 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00a830 tx=0x7f8b300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5acefe162c137c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.615+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0072d0 con 0x7f8b30005180 2023-11-24T08:13:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.615+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c02f070 con 0x7f8b30005180 2023-11-24T08:13:03.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.615+0000 7f8b4915f640 1 -- 172.21.15.106:0/3425693156 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0076b0 con 0x7f8b30005180 2023-11-24T08:13:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 -- 172.21.15.106:0/3425693156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b30005180 msgr2=0x7f8b3000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b30005180 0x7f8b3000b980 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00a830 tx=0x7f8b300057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 -- 172.21.15.106:0/3425693156 shutdown_connections 2023-11-24T08:13:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3425693156 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b30005180 0x7f8b3000b980 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 -- 172.21.15.106:0/3425693156 >> 172.21.15.106:0/3425693156 conn(0x7f8b440cb830 msgr2=0x7f8b440cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.623 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 -- 172.21.15.106:0/3425693156 shutdown_connections 2023-11-24T08:13:03.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.616+0000 7f8b4b163640 1 -- 172.21.15.106:0/3425693156 wait complete. 2023-11-24T08:13:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.618+0000 7f8b4b163640 1 Processor -- start 2023-11-24T08:13:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.618+0000 7f8b4b163640 1 -- start start 2023-11-24T08:13:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4b163640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30005180 0x7f8b440ca5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4b163640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b4415a400 0x7f8b4415a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4b163640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 0x7f8b440c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4b163640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4415adc0 con 0x7f8b440c81c0 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4b163640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440c49c0 con 0x7f8b4415a400 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4b163640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b440c3df0 con 0x7f8b30005180 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4a962640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 0x7f8b440c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4a962640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 0x7f8b440c8640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:38760/0 (socket says 172.21.15.106:38760) 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4a962640 1 -- 172.21.15.106:0/3611546424 learned_addr learned my addr 172.21.15.106:0/3611546424 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4a962640 1 -- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30005180 msgr2=0x7f8b440ca5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b4a161640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30005180 0x7f8b440ca5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.619+0000 7f8b49960640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b4415a400 0x7f8b4415a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30005180 0x7f8b440ca5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4a962640 1 -- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b4415a400 msgr2=0x7f8b4415a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b4415a400 0x7f8b4415a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4a962640 1 -- 172.21.15.106:0/3611546424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440c4090 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4a962640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 0x7f8b440c8640 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00a950 tx=0x7f8b3c001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c044070 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c0019c0 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b440c4370 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c007650 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.620+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b440caaf0 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.621+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b440c4b30 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.622+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f8b3c031030 con 0x7f8b440c81c0 2023-11-24T08:13:03.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.623+0000 7f8b3affd640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b2806b3e0 0x7f8b2806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:03.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.624+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b3c0ab650 con 0x7f8b440c81c0 2023-11-24T08:13:03.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.624+0000 7f8b4a161640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b2806b3e0 0x7f8b2806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:03.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.624+0000 7f8b4a161640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b2806b3e0 0x7f8b2806d8a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009970 tx=0x7f8b40008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:03.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.628+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b3c075f20 con 0x7f8b440c81c0 2023-11-24T08:13:03.860 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.859+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8b4404e840 con 0x7f8b440c81c0 2023-11-24T08:13:03.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.860+0000 7f8b3affd640 1 -- 172.21.15.106:0/3611546424 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8b3c079540 con 0x7f8b440c81c0 2023-11-24T08:13:03.861 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:13:03.864 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.862+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b2806b3e0 msgr2=0x7f8b2806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.862+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b2806b3e0 0x7f8b2806d8a0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f8b40009970 tx=0x7f8b40008040 comp rx=0 tx=0).stop 2023-11-24T08:13:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 msgr2=0x7f8b440c8640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 0x7f8b440c8640 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c00a950 tx=0x7f8b3c001d80 comp rx=0 tx=0).stop 2023-11-24T08:13:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 shutdown_connections 2023-11-24T08:13:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b440c81c0 0x7f8b440c8640 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b4415a400 0x7f8b4415a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b2806b3e0 0x7f8b2806d8a0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 --2- 172.21.15.106:0/3611546424 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b30005180 0x7f8b440ca5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 >> 172.21.15.106:0/3611546424 conn(0x7f8b440cb830 msgr2=0x7f8b4415dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 shutdown_connections 2023-11-24T08:13:03.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:03.863+0000 7f8b4b163640 1 -- 172.21.15.106:0/3611546424 wait complete. 2023-11-24T08:13:03.885 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:13:03.885 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:13:03.885 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi106.front.sepia.ceph.com 2023-11-24T08:13:03.885 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-24T08:13:03.885 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-24T08:13:03.886 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:13:03.886 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:13:03.886 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:13:03.886 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:13:03.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:03.886 DEBUG:teuthology.orchestra.run.smithi106:> ip addr 2023-11-24T08:13:03.940 INFO:teuthology.orchestra.run.smithi106.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:13:03.940 INFO:teuthology.orchestra.run.smithi106.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:13:03.940 INFO:teuthology.orchestra.run.smithi106.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:13:03.940 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.940 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 ::1/128 scope host 2023-11-24T08:13:03.940 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2a brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f0 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:d8:a1:2b brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout: altname enp7s0f1 2023-11-24T08:13:03.941 INFO:teuthology.orchestra.run.smithi106.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6c brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 0c:c4:7a:8f:ce:6d brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout: inet 172.21.15.106/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft 42132sec preferred_lft 42132sec 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::ec4:7aff:fe8f:ce6d/64 scope link 2023-11-24T08:13:03.942 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.943 INFO:teuthology.orchestra.run.smithi106.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:13:03.943 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:03.943 INFO:teuthology.orchestra.run.smithi106.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:13:03.943 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.943 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::a407:f7ff:fe84:63d0/64 scope link 2023-11-24T08:13:03.943 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.944 INFO:teuthology.orchestra.run.smithi106.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:13:03.944 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 92:79:d8:bb:c0:8c brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:13:03.944 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::9079:d8ff:febb:c08c/64 scope link 2023-11-24T08:13:03.944 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.944 INFO:teuthology.orchestra.run.smithi106.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:13:03.944 INFO:teuthology.orchestra.run.smithi106.stdout: link/ether 22:e3:f2:93:48:7e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:13:03.945 INFO:teuthology.orchestra.run.smithi106.stdout: inet6 fe80::20e3:f2ff:fe93:487e/64 scope link 2023-11-24T08:13:03.945 INFO:teuthology.orchestra.run.smithi106.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:03.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:03.946 DEBUG:teuthology.orchestra.run.smithi106:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:13:04.041 INFO:teuthology.orchestra.run.smithi106.stdout:1 2023-11-24T08:13:04.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.045 DEBUG:teuthology.orchestra.run.smithi106:> route 2023-11-24T08:13:04.102 INFO:teuthology.orchestra.run.smithi106.stdout:Kernel IP routing table 2023-11-24T08:13:04.102 INFO:teuthology.orchestra.run.smithi106.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:13:04.102 INFO:teuthology.orchestra.run.smithi106.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:13:04.102 INFO:teuthology.orchestra.run.smithi106.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:13:04.102 INFO:teuthology.orchestra.run.smithi106.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:13:04.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.103 DEBUG:teuthology.orchestra.run.smithi106:> (cd / && exec bash -c ' 2023-11-24T08:13:04.103 DEBUG:teuthology.orchestra.run.smithi106:> set -e 2023-11-24T08:13:04.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:13:04.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:13:04.103 DEBUG:teuthology.orchestra.run.smithi106:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:13:04.103 DEBUG:teuthology.orchestra.run.smithi106:> ') 2023-11-24T08:13:04.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.288 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:13:04.338 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:13:04.339 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:13:04.339 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:13:04.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.339 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link set brx.0 up 2023-11-24T08:13:04.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:13:04.420 DEBUG:teuthology.orchestra.run.smithi106:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-24T08:13:04.470 INFO:teuthology.orchestra.run.smithi106.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-24T08:13:04.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:13:04.471 DEBUG:teuthology.orchestra.run.smithi106:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:13:04.522 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-11-24T08:13:04.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.522 DEBUG:teuthology.orchestra.run.smithi106:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:13:04.664 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:13:04.665 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:13:04.702 INFO:teuthology.orchestra.run.smithi106.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:13:04.702 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:13:04.702 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:13:04.702 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: trying mount with new device syntax: 0@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:13:04.702 INFO:teuthology.orchestra.run.smithi106.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:13:04.704 INFO:teuthology.orchestra.run.smithi106.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2023-11-24T08:13:04.704 INFO:teuthology.orchestra.run.smithi106.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:13:04.704 INFO:teuthology.orchestra.run.smithi106.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:13:04.704 INFO:teuthology.orchestra.run.smithi106.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:13:04.704 INFO:teuthology.orchestra.run.smithi106.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:13:04.708 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:13:04.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.709 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-24T08:13:04.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:04.791 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:13:04.852 INFO:teuthology.orchestra.run.smithi106.stdout:client5506 2023-11-24T08:13:04.852 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:13:04.852 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:04.903 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-24T08:13:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.158+0000 7f5875b0c640 1 Processor -- start 2023-11-24T08:13:05.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.158+0000 7f5875b0c640 1 -- start start 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.159+0000 7f5875b0c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5870159ce0 0x7f587006de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f5875b0c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58700b21d0 0x7f58700cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f5875b0c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58700b3a40 con 0x7f58700cbf80 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f5875b0c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700cc3d0 con 0x7f58700b21d0 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f5875b0c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58700cdfa0 con 0x7f5870159ce0 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58700b21d0 0x7f58700cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f58700b21d0 0x7f58700cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56980/0 (socket says 172.21.15.106:56980) 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 -- 172.21.15.106:0/3305128107 learned_addr learned my addr 172.21.15.106:0/3305128107 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586f7fe640 1 --2- 172.21.15.106:0/3305128107 >> v2:172.21.15.156:3300/0 conn(0x7f5870159ce0 0x7f587006de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:05.162 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 -- 172.21.15.106:0/3305128107 >> v2:172.21.15.156:3300/0 conn(0x7f5870159ce0 msgr2=0x7f587006de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 --2- 172.21.15.106:0/3305128107 >> v2:172.21.15.156:3300/0 conn(0x7f5870159ce0 0x7f587006de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 -- 172.21.15.106:0/3305128107 >> v1:172.21.15.156:6789/0 conn(0x7f58700cbf80 legacy=0x7f587015d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.160+0000 7f586effd640 1 -- 172.21.15.106:0/3305128107 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58700cd9e0 con 0x7f58700b21d0 2023-11-24T08:13:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.162+0000 7f586effd640 1 --2- 172.21.15.106:0/3305128107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f58700cba40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f587006e770 tx=0x7f586402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b34780636976988 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:05.163 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.162+0000 7f586e7fc640 1 -- 172.21.15.106:0/3305128107 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864031870 con 0x7f58700b21d0 2023-11-24T08:13:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.162+0000 7f586e7fc640 1 -- 172.21.15.106:0/3305128107 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58640319d0 con 0x7f58700b21d0 2023-11-24T08:13:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.162+0000 7f586e7fc640 1 -- 172.21.15.106:0/3305128107 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864031d60 con 0x7f58700b21d0 2023-11-24T08:13:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.162+0000 7f5875b0c640 1 -- 172.21.15.106:0/3305128107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 msgr2=0x7f58700cba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:05.164 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.162+0000 7f5875b0c640 1 --2- 172.21.15.106:0/3305128107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f58700cba40 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f587006e770 tx=0x7f586402b910 comp rx=0 tx=0).stop 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.163+0000 7f5875b0c640 1 -- 172.21.15.106:0/3305128107 shutdown_connections 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.163+0000 7f5875b0c640 1 --2- 172.21.15.106:0/3305128107 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f58700cba40 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.163+0000 7f5875b0c640 1 --2- 172.21.15.106:0/3305128107 >> v2:172.21.15.156:3300/0 conn(0x7f5870159ce0 0x7f587006de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.163+0000 7f5875b0c640 1 -- 172.21.15.106:0/3305128107 >> 172.21.15.106:0/3305128107 conn(0x7f587006a920 msgr2=0x7f587006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.163+0000 7f5875b0c640 1 -- 172.21.15.106:0/3305128107 shutdown_connections 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.163+0000 7f5875b0c640 1 -- 172.21.15.106:0/3305128107 wait complete. 2023-11-24T08:13:05.165 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.164+0000 7f5875b0c640 1 Processor -- start 2023-11-24T08:13:05.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.165+0000 7f5875b0c640 1 -- start start 2023-11-24T08:13:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.165+0000 7f5875b0c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f587015a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.165+0000 7f5875b0c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58700cbf80 0x7f587014c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:05.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f5875b0c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870159ce0 0x7f5870152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f5875b0c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587014e030 con 0x7f58700b21d0 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f5875b0c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587014e310 con 0x7f5870159ce0 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f5875b0c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587014e5f0 con 0x7f58700cbf80 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f587015a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f587015a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:56994/0 (socket says 172.21.15.106:56994) 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 -- 172.21.15.106:0/254114728 learned_addr learned my addr 172.21.15.106:0/254114728 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 -- 172.21.15.106:0/254114728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58700cbf80 msgr2=0x7f587014c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58700cbf80 0x7f587014c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 -- 172.21.15.106:0/254114728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870159ce0 msgr2=0x7f5870152f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870159ce0 0x7f5870152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 -- 172.21.15.106:0/254114728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640094d0 con 0x7f58700b21d0 2023-11-24T08:13:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.166+0000 7f586f7fe640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f587015a0c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f586000c7f0 tx=0x7f586000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.167+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860004220 con 0x7f58700b21d0 2023-11-24T08:13:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.167+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600043a0 con 0x7f58700b21d0 2023-11-24T08:13:05.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.167+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587014e8d0 con 0x7f58700b21d0 2023-11-24T08:13:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.167+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5860007860 con 0x7f58700b21d0 2023-11-24T08:13:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.167+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587014c790 con 0x7f58700b21d0 2023-11-24T08:13:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.168+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870067470 con 0x7f58700b21d0 2023-11-24T08:13:05.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.169+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f5860007bd0 con 0x7f58700b21d0 2023-11-24T08:13:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.170+0000 7f5857fff640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f585c06b3e0 0x7f585c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.170+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f586008a200 con 0x7f58700b21d0 2023-11-24T08:13:05.171 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.170+0000 7f586effd640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f585c06b3e0 0x7f585c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:05.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.171+0000 7f586effd640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f585c06b3e0 0x7f585c06d8a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5864036000 tx=0x7f5864030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:05.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.174+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f58600548f0 con 0x7f58700b21d0 2023-11-24T08:13:05.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.404+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f587014e170 con 0x7f58700b21d0 2023-11-24T08:13:05.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.405+0000 7f5857fff640 1 -- 172.21.15.106:0/254114728 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v53) v1 ==== 53+0+73 (secure 0 0 0) 0x7f587014e170 con 0x7f58700b21d0 2023-11-24T08:13:05.406 INFO:teuthology.orchestra.run.smithi106.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-24T08:13:05.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f585c06b3e0 msgr2=0x7f585c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:05.411 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f585c06b3e0 0x7f585c06d8a0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f5864036000 tx=0x7f5864030000 comp rx=0 tx=0).stop 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 msgr2=0x7f587015a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f587015a0c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f586000c7f0 tx=0x7f586000ccb0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 shutdown_connections 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5870159ce0 0x7f5870152f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f585c06b3e0 0x7f585c06d8a0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f58700cbf80 0x7f587014c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.412 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 --2- 172.21.15.106:0/254114728 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f58700b21d0 0x7f587015a0c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:05.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 >> 172.21.15.106:0/254114728 conn(0x7f587006a920 msgr2=0x7f58700ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:05.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.407+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 shutdown_connections 2023-11-24T08:13:05.413 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:05.408+0000 7f5875b0c640 1 -- 172.21.15.106:0/254114728 wait complete. 2023-11-24T08:13:05.426 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi156.front.sepia.ceph.com 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-24T08:13:05.427 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:13:05.427 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:05.428 DEBUG:teuthology.orchestra.run.smithi156:> ip addr 2023-11-24T08:13:05.491 INFO:teuthology.orchestra.run.smithi156.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-24T08:13:05.491 INFO:teuthology.orchestra.run.smithi156.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-24T08:13:05.491 INFO:teuthology.orchestra.run.smithi156.stdout: inet 127.0.0.1/8 scope host lo 2023-11-24T08:13:05.491 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.491 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 ::1/128 scope host 2023-11-24T08:13:05.492 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.492 INFO:teuthology.orchestra.run.smithi156.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:13:05.492 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c0 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:05.492 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f0 2023-11-24T08:13:05.492 INFO:teuthology.orchestra.run.smithi156.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:13:05.492 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:da:6a:c1 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout: altname enp7s0f1 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fc brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 0c:c4:7a:88:6d:fd brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout: inet 172.21.15.156/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft 42146sec preferred_lft 42146sec 2023-11-24T08:13:05.493 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::ec4:7aff:fe88:6dfd/64 scope link 2023-11-24T08:13:05.494 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.494 INFO:teuthology.orchestra.run.smithi156.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-24T08:13:05.494 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff 2023-11-24T08:13:05.494 INFO:teuthology.orchestra.run.smithi156.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-24T08:13:05.494 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.495 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::1831:a6ff:fe65:552d/64 scope link 2023-11-24T08:13:05.495 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.495 INFO:teuthology.orchestra.run.smithi156.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:13:05.495 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether 6a:3b:9f:67:71:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:13:05.496 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::683b:9fff:fe67:7152/64 scope link 2023-11-24T08:13:05.496 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.496 INFO:teuthology.orchestra.run.smithi156.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-24T08:13:05.497 INFO:teuthology.orchestra.run.smithi156.stdout: link/ether a2:a3:a7:89:e5:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:13:05.497 INFO:teuthology.orchestra.run.smithi156.stdout: inet6 fe80::a0a3:a7ff:fe89:e507/64 scope link 2023-11-24T08:13:05.497 INFO:teuthology.orchestra.run.smithi156.stdout: valid_lft forever preferred_lft forever 2023-11-24T08:13:05.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:05.498 DEBUG:teuthology.orchestra.run.smithi156:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-24T08:13:05.590 INFO:teuthology.orchestra.run.smithi156.stdout:1 2023-11-24T08:13:05.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:05.595 DEBUG:teuthology.orchestra.run.smithi156:> route 2023-11-24T08:13:05.650 INFO:teuthology.orchestra.run.smithi156.stdout:Kernel IP routing table 2023-11-24T08:13:05.650 INFO:teuthology.orchestra.run.smithi156.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-24T08:13:05.650 INFO:teuthology.orchestra.run.smithi156.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-24T08:13:05.650 INFO:teuthology.orchestra.run.smithi156.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-24T08:13:05.650 INFO:teuthology.orchestra.run.smithi156.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-24T08:13:05.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:05.651 DEBUG:teuthology.orchestra.run.smithi156:> (cd / && exec bash -c ' 2023-11-24T08:13:05.651 DEBUG:teuthology.orchestra.run.smithi156:> set -e 2023-11-24T08:13:05.651 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-24T08:13:05.652 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-24T08:13:05.652 DEBUG:teuthology.orchestra.run.smithi156:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-24T08:13:05.652 DEBUG:teuthology.orchestra.run.smithi156:> ') 2023-11-24T08:13:05.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:05.841 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:13:05.894 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:13:05.894 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:13:05.894 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:13:05.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:05.895 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link set brx.0 up 2023-11-24T08:13:05.987 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:13:05.987 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-24T08:13:06.039 INFO:teuthology.orchestra.run.smithi156.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-24T08:13:06.039 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-24T08:13:06.040 DEBUG:teuthology.orchestra.run.smithi156:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:13:06.086 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-11-24T08:13:06.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:06.086 DEBUG:teuthology.orchestra.run.smithi156:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:13:06.276 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:13:06.276 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:13:06.313 INFO:teuthology.orchestra.run.smithi156.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-11-24T08:13:06.313 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-11-24T08:13:06.313 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: resolved to: "172.21.15.106:3300,172.21.15.156:3300,172.21.15.156:3301" 2023-11-24T08:13:06.313 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: trying mount with new device syntax: 1@d1a897c2-15ce-4272-ad66-4b3723970cb1.cephfs=/ 2023-11-24T08:13:06.313 INFO:teuthology.orchestra.run.smithi156.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.106:3300/172.21.15.156:3300/172.21.15.156:3301" will pass to kernel 2023-11-24T08:13:06.316 INFO:teuthology.orchestra.run.smithi156.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2023-11-24T08:13:06.316 INFO:teuthology.orchestra.run.smithi156.stdout: You just mounted a file system that supports labels which does not 2023-11-24T08:13:06.316 INFO:teuthology.orchestra.run.smithi156.stdout: contain labels, onto an SELinux box. It is likely that confined 2023-11-24T08:13:06.316 INFO:teuthology.orchestra.run.smithi156.stdout: applications will generate AVC messages and not be allowed access to 2023-11-24T08:13:06.316 INFO:teuthology.orchestra.run.smithi156.stdout: this file system. For more details see restorecon(8) and mount(8). 2023-11-24T08:13:06.320 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-24T08:13:06.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:06.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-24T08:13:06.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:06.410 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-11-24T08:13:06.467 INFO:teuthology.orchestra.run.smithi156.stdout:client7191 2023-11-24T08:13:06.467 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:13:06.467 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:06.520 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-11-24T08:13:06.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.747+0000 7fb4769a9640 1 Processor -- start 2023-11-24T08:13:06.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.747+0000 7fb4769a9640 1 -- start start 2023-11-24T08:13:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb4769a9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb47015c7f0 0x7fb47006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:06.750 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb4769a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4700cece0 con 0x7fb4700cf450 2023-11-24T08:13:06.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb4769a9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb47006d040 con 0x7fb4700ce6c0 2023-11-24T08:13:06.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb4769a9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4700ceaf0 con 0x7fb47015c7f0 2023-11-24T08:13:06.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb46f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb4700cf450 0x7fb4700cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48568/0 (socket says 172.21.15.106:48568) 2023-11-24T08:13:06.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb46f7fe640 1 -- 172.21.15.106:0/3707611013 learned_addr learned my addr 172.21.15.106:0/3707611013 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:06.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.749+0000 7fb46ffff640 1 --2- 172.21.15.106:0/3707611013 >> v2:172.21.15.106:3300/0 conn(0x7fb47015c7f0 0x7fb47006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:06.752 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.750+0000 7fb46effd640 1 -- 172.21.15.106:0/3707611013 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146854154 0 0) 0x7fb4700cece0 con 0x7fb4700cf450 2023-11-24T08:13:06.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.750+0000 7fb46effd640 1 -- 172.21.15.106:0/3707611013 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb458003520 con 0x7fb4700cf450 2023-11-24T08:13:06.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.750+0000 7fb46ffff640 1 -- 172.21.15.106:0/3707611013 >> v1:172.21.15.156:6789/0 conn(0x7fb4700ce6c0 legacy=0x7fb4700714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:06.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.750+0000 7fb46ffff640 1 -- 172.21.15.106:0/3707611013 >> v1:172.21.15.106:6789/0 conn(0x7fb4700cf450 legacy=0x7fb4700cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:06.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.750+0000 7fb46ffff640 1 -- 172.21.15.106:0/3707611013 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47006d320 con 0x7fb47015c7f0 2023-11-24T08:13:06.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.751+0000 7fb46ffff640 1 --2- 172.21.15.106:0/3707611013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb47015c7f0 0x7fb47006db20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb4600097a0 tx=0x7fb46002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d986214a83d897e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:06.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.751+0000 7fb46effd640 1 -- 172.21.15.106:0/3707611013 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460031830 con 0x7fb47015c7f0 2023-11-24T08:13:06.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.751+0000 7fb46effd640 1 -- 172.21.15.106:0/3707611013 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb460031990 con 0x7fb47015c7f0 2023-11-24T08:13:06.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.751+0000 7fb46effd640 1 -- 172.21.15.106:0/3707611013 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb460031d20 con 0x7fb47015c7f0 2023-11-24T08:13:06.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 -- 172.21.15.106:0/3707611013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb47015c7f0 msgr2=0x7fb47006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:06.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 --2- 172.21.15.106:0/3707611013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb47015c7f0 0x7fb47006db20 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fb4600097a0 tx=0x7fb46002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:06.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 -- 172.21.15.106:0/3707611013 shutdown_connections 2023-11-24T08:13:06.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 --2- 172.21.15.106:0/3707611013 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb47015c7f0 0x7fb47006db20 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:06.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 -- 172.21.15.106:0/3707611013 >> 172.21.15.106:0/3707611013 conn(0x7fb47006a920 msgr2=0x7fb47006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:06.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 -- 172.21.15.106:0/3707611013 shutdown_connections 2023-11-24T08:13:06.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.752+0000 7fb4769a9640 1 -- 172.21.15.106:0/3707611013 wait complete. 2023-11-24T08:13:06.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.754+0000 7fb4769a9640 1 Processor -- start 2023-11-24T08:13:06.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.754+0000 7fb4769a9640 1 -- start start 2023-11-24T08:13:06.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb4769a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4700ce6c0 0x7fb4701f6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:06.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb4769a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 0x7fb4701f48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:06.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb4769a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb47015c7f0 0x7fb4701f3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:06.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb4769a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701f4700 con 0x7fb4700cf450 2023-11-24T08:13:06.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb4769a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701f4140 con 0x7fb47015c7f0 2023-11-24T08:13:06.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb4769a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701f4420 con 0x7fb4700ce6c0 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb46f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 0x7fb4701f48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb46f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 0x7fb4701f48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57006/0 (socket says 172.21.15.106:57006) 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb46f7fe640 1 -- 172.21.15.106:0/1016283800 learned_addr learned my addr 172.21.15.106:0/1016283800 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.755+0000 7fb474f46640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb47015c7f0 0x7fb4701f3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46ffff640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4700ce6c0 0x7fb4701f6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46f7fe640 1 -- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4700ce6c0 msgr2=0x7fb4701f6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46f7fe640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4700ce6c0 0x7fb4701f6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46f7fe640 1 -- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb47015c7f0 msgr2=0x7fb4701f3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46f7fe640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb47015c7f0 0x7fb4701f3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46f7fe640 1 -- 172.21.15.106:0/1016283800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb460009400 con 0x7fb4700cf450 2023-11-24T08:13:06.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb474f46640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb47015c7f0 0x7fb4701f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46f7fe640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 0x7fb4701f48d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb46400c500 tx=0x7fb46400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb464005ce0 con 0x7fb4700cf450 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb464005e40 con 0x7fb4700cf450 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb464007330 con 0x7fb4700cf450 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4700c8490 con 0x7fb4700cf450 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.756+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47015ac90 con 0x7fb4700cf450 2023-11-24T08:13:06.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.757+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb47015aeb0 con 0x7fb4700cf450 2023-11-24T08:13:06.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.758+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fb464007550 con 0x7fb4700cf450 2023-11-24T08:13:06.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.760+0000 7fb46cff9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb45c06b3e0 0x7fb45c06d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:06.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.760+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb464089690 con 0x7fb4700cf450 2023-11-24T08:13:06.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.760+0000 7fb46ffff640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb45c06b3e0 0x7fb45c06d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:06.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.761+0000 7fb46ffff640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb45c06b3e0 0x7fb45c06d8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb460036000 tx=0x7fb46003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:06.766 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.764+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb47015aeb0 con 0x7fb4700cf450 2023-11-24T08:13:06.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:06.994+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb4700c89b0 con 0x7fb4700cf450 2023-11-24T08:13:07.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.374+0000 7fb46cff9640 1 -- 172.21.15.106:0/1016283800 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 136+0+0 (secure 0 0 0) 0x7fb4700c89b0 con 0x7fb4700cf450 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb45c06b3e0 msgr2=0x7fb45c06d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb45c06b3e0 0x7fb45c06d8a0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb460036000 tx=0x7fb46003c040 comp rx=0 tx=0).stop 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 msgr2=0x7fb4701f48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 0x7fb4701f48d0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fb46400c500 tx=0x7fb46400c9c0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 shutdown_connections 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb47015c7f0 0x7fb4701f3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb4700cf450 0x7fb4701f48d0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb45c06b3e0 0x7fb45c06d8a0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 --2- 172.21.15.106:0/1016283800 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb4700ce6c0 0x7fb4701f6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 >> 172.21.15.106:0/1016283800 conn(0x7fb47006a920 msgr2=0x7fb4700af860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:07.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 shutdown_connections 2023-11-24T08:13:07.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.377+0000 7fb4769a9640 1 -- 172.21.15.106:0/1016283800 wait complete. 2023-11-24T08:13:07.400 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-24T08:13:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.614+0000 7f10b10d1640 1 Processor -- start 2023-11-24T08:13:07.615 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.614+0000 7f10b10d1640 1 -- start start 2023-11-24T08:13:07.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.615+0000 7f10b10d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac15b1d0 0x7f10ac071f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:07.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.615+0000 7f10b10d1640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10ac070550 0x7f10ac0b0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:07.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.615+0000 7f10b10d1640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac0ca0e0 con 0x7f10ac0b1080 2023-11-24T08:13:07.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.615+0000 7f10b10d1640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac15fde0 con 0x7f10ac070550 2023-11-24T08:13:07.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.615+0000 7f10b10d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b1510 con 0x7f10ac15b1d0 2023-11-24T08:13:07.618 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10ac070550 0x7f10ac0b0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:07.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f10ac070550 0x7f10ac0b0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57008/0 (socket says 172.21.15.106:57008) 2023-11-24T08:13:07.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 -- 172.21.15.106:0/131520901 learned_addr learned my addr 172.21.15.106:0/131520901 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:07.619 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 -- 172.21.15.106:0/131520901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac15b1d0 msgr2=0x7f10ac071f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:07.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 --2- 172.21.15.106:0/131520901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac15b1d0 0x7f10ac071f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 -- 172.21.15.106:0/131520901 >> v1:172.21.15.156:6789/0 conn(0x7f10ac0b1080 legacy=0x7f10ac06c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:07.620 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.616+0000 7f10a3fff640 1 -- 172.21.15.106:0/131520901 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac072800 con 0x7f10ac070550 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.618+0000 7f10a3fff640 1 --2- 172.21.15.106:0/131520901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac070550 0x7f10ac0b0b40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1094009870 tx=0x7f109402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97bc18791ce930c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.618+0000 7f10ab7fe640 1 -- 172.21.15.106:0/131520901 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094031640 con 0x7f10ac070550 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.618+0000 7f10ab7fe640 1 -- 172.21.15.106:0/131520901 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10940317a0 con 0x7f10ac070550 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.618+0000 7f10ab7fe640 1 -- 172.21.15.106:0/131520901 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094031b30 con 0x7f10ac070550 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.618+0000 7f10b10d1640 1 -- 172.21.15.106:0/131520901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac070550 msgr2=0x7f10ac0b0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.618+0000 7f10b10d1640 1 --2- 172.21.15.106:0/131520901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac070550 0x7f10ac0b0b40 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f1094009870 tx=0x7f109402b8f0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.621 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.619+0000 7f10b10d1640 1 -- 172.21.15.106:0/131520901 shutdown_connections 2023-11-24T08:13:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.619+0000 7f10b10d1640 1 --2- 172.21.15.106:0/131520901 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac070550 0x7f10ac0b0b40 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.619+0000 7f10b10d1640 1 --2- 172.21.15.106:0/131520901 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac15b1d0 0x7f10ac071f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.619+0000 7f10b10d1640 1 -- 172.21.15.106:0/131520901 >> 172.21.15.106:0/131520901 conn(0x7f10ac0cbb40 msgr2=0x7f10ac0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.619+0000 7f10b10d1640 1 -- 172.21.15.106:0/131520901 shutdown_connections 2023-11-24T08:13:07.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.619+0000 7f10b10d1640 1 -- 172.21.15.106:0/131520901 wait complete. 2023-11-24T08:13:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.620+0000 7f10b10d1640 1 Processor -- start 2023-11-24T08:13:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 -- start start 2023-11-24T08:13:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10ac070550 0x7f10ac0cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac0b1080 0x7f10ac09a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 0x7f10ac0b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:07.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0cdbe0 con 0x7f10ac15b1d0 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0ce670 con 0x7f10ac070550 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.621+0000 7f10b10d1640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac09a7e0 con 0x7f10ac0b1080 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 0x7f10ac0b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 0x7f10ac0b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57010/0 (socket says 172.21.15.106:57010) 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 -- 172.21.15.106:0/471228454 learned_addr learned my addr 172.21.15.106:0/471228454 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10abfff640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10ac070550 0x7f10ac0cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 -- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac0b1080 msgr2=0x7f10ac09a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10a3fff640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac0b1080 0x7f10ac09a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac0b1080 0x7f10ac09a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 -- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10ac070550 msgr2=0x7f10ac0cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10ac070550 0x7f10ac0cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10b08d0640 1 -- 172.21.15.106:0/471228454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10940094d0 con 0x7f10ac15b1d0 2023-11-24T08:13:07.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.622+0000 7f10abfff640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10ac070550 0x7f10ac0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:07.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.623+0000 7f10b08d0640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 0x7f10ac0b3ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f10a400c780 tx=0x7f10a400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:07.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.623+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4004b60 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.623+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a400ce90 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.623+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400a850 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.623+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0cddd0 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.623+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0ce390 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.624+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac0b1cc0 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.625+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f10a400a9b0 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.626+0000 7f10a97fa640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f109006b410 0x7f109006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.626+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10a408a150 con 0x7f10ac15b1d0 2023-11-24T08:13:07.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.626+0000 7f10abfff640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f109006b410 0x7f109006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:07.629 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.627+0000 7f10abfff640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f109006b410 0x7f109006d8d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0cee70 tx=0x7f109c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:07.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.630+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f10a40bce90 con 0x7f10ac15b1d0 2023-11-24T08:13:07.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.842+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f10ac0c9d70 con 0x7f109006b410 2023-11-24T08:13:07.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.844+0000 7f10a97fa640 1 -- 172.21.15.106:0/471228454 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f109c00a460 con 0x7f109006b410 2023-11-24T08:13:07.846 INFO:teuthology.orchestra.run.smithi106.stdout:[ 2023-11-24T08:13:07.846 INFO:teuthology.orchestra.run.smithi106.stdout: { 2023-11-24T08:13:07.846 INFO:teuthology.orchestra.run.smithi106.stdout: "name": "cephfs" 2023-11-24T08:13:07.846 INFO:teuthology.orchestra.run.smithi106.stdout: } 2023-11-24T08:13:07.846 INFO:teuthology.orchestra.run.smithi106.stdout:] 2023-11-24T08:13:07.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f109006b410 msgr2=0x7f109006d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f109006b410 0x7f109006d8d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f10ac0cee70 tx=0x7f109c008040 comp rx=0 tx=0).stop 2023-11-24T08:13:07.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 msgr2=0x7f10ac0b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:07.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 0x7f10ac0b3ac0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f10a400c780 tx=0x7f10a400cc40 comp rx=0 tx=0).stop 2023-11-24T08:13:07.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 shutdown_connections 2023-11-24T08:13:07.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f10ac15b1d0 0x7f10ac0b3ac0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f109006b410 0x7f109006d8d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f10ac0b1080 0x7f10ac09a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 --2- 172.21.15.106:0/471228454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f10ac070550 0x7f10ac0cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.847+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 >> 172.21.15.106:0/471228454 conn(0x7f10ac0cbb40 msgr2=0x7f10ac0adac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.848+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 shutdown_connections 2023-11-24T08:13:07.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:07.848+0000 7f10b10d1640 1 -- 172.21.15.106:0/471228454 wait complete. 2023-11-24T08:13:07.866 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-24T08:13:08.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.080+0000 7f1a6ac23640 1 Processor -- start 2023-11-24T08:13:08.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.080+0000 7f1a6ac23640 1 -- start start 2023-11-24T08:13:08.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a6ac23640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1a640cc0f0 0x7f1a640c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a6ac23640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1a640c57e0 0x7f1a6406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a6ac23640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6415eea0 con 0x7f1a6415b440 2023-11-24T08:13:08.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a6ac23640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6415b820 con 0x7f1a640cc0f0 2023-11-24T08:13:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a6ac23640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a640abf30 con 0x7f1a640c57e0 2023-11-24T08:13:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a689bf640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f1a6415b440 0x7f1a640cb9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48572/0 (socket says 172.21.15.106:48572) 2023-11-24T08:13:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a689bf640 1 -- 172.21.15.106:0/2238291229 learned_addr learned my addr 172.21.15.106:0/2238291229 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a63fff640 1 --2- 172.21.15.106:0/2238291229 >> v2:172.21.15.106:3300/0 conn(0x7f1a640cc0f0 0x7f1a640c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.084 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.082+0000 7f1a691c0640 1 --2- 172.21.15.106:0/2238291229 >> v2:172.21.15.156:3300/0 conn(0x7f1a640c57e0 0x7f1a6406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a637fe640 1 -- 172.21.15.106:0/2238291229 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443363974 0 0) 0x7f1a6415eea0 con 0x7f1a6415b440 2023-11-24T08:13:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a637fe640 1 -- 172.21.15.106:0/2238291229 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a54003520 con 0x7f1a6415b440 2023-11-24T08:13:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a63fff640 1 -- 172.21.15.106:0/2238291229 >> v2:172.21.15.156:3300/0 conn(0x7f1a640c57e0 msgr2=0x7f1a6406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a63fff640 1 --2- 172.21.15.106:0/2238291229 >> v2:172.21.15.156:3300/0 conn(0x7f1a640c57e0 0x7f1a6406c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.085 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a63fff640 1 -- 172.21.15.106:0/2238291229 >> v1:172.21.15.106:6789/0 conn(0x7f1a6415b440 legacy=0x7f1a640cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.086 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a63fff640 1 -- 172.21.15.106:0/2238291229 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6415f0f0 con 0x7f1a640cc0f0 2023-11-24T08:13:08.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.083+0000 7f1a691c0640 1 --2- 172.21.15.106:0/2238291229 >> v2:172.21.15.156:3300/0 conn(0x7f1a640c57e0 0x7f1a6406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.084+0000 7f1a63fff640 1 --2- 172.21.15.106:0/2238291229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c52a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f1a640c95e0 tx=0x7f1a5002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7840a1fd55954f5f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.084+0000 7f1a637fe640 1 -- 172.21.15.106:0/2238291229 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5002e070 con 0x7f1a640cc0f0 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.084+0000 7f1a637fe640 1 -- 172.21.15.106:0/2238291229 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a500315a0 con 0x7f1a640cc0f0 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.084+0000 7f1a637fe640 1 -- 172.21.15.106:0/2238291229 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50031910 con 0x7f1a640cc0f0 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 -- 172.21.15.106:0/2238291229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 msgr2=0x7f1a640c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/2238291229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c52a0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f1a640c95e0 tx=0x7f1a5002b840 comp rx=0 tx=0).stop 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 -- 172.21.15.106:0/2238291229 shutdown_connections 2023-11-24T08:13:08.090 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/2238291229 >> v2:172.21.15.156:3300/0 conn(0x7f1a640c57e0 0x7f1a6406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/2238291229 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c52a0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 -- 172.21.15.106:0/2238291229 >> 172.21.15.106:0/2238291229 conn(0x7f1a640b0c50 msgr2=0x7f1a640b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:08.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 -- 172.21.15.106:0/2238291229 shutdown_connections 2023-11-24T08:13:08.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.085+0000 7f1a6ac23640 1 -- 172.21.15.106:0/2238291229 wait complete. 2023-11-24T08:13:08.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.087+0000 7f1a6ac23640 1 Processor -- start 2023-11-24T08:13:08.091 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.087+0000 7f1a6ac23640 1 -- start start 2023-11-24T08:13:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.088+0000 7f1a6ac23640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a640c57e0 0x7f1a64150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.088+0000 7f1a6ac23640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.088+0000 7f1a6ac23640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6415b440 0x7f1a640c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.088+0000 7f1a6ac23640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6409b4d0 con 0x7f1a640cc0f0 2023-11-24T08:13:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.088+0000 7f1a6ac23640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6409b740 con 0x7f1a6415b440 2023-11-24T08:13:08.095 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.088+0000 7f1a6ac23640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6409ba20 con 0x7f1a640c57e0 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57030/0 (socket says 172.21.15.106:57030) 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 -- 172.21.15.106:0/1452530872 learned_addr learned my addr 172.21.15.106:0/1452530872 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 -- 172.21.15.106:0/1452530872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a640c57e0 msgr2=0x7f1a64150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a640c57e0 0x7f1a64150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 -- 172.21.15.106:0/1452530872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6415b440 msgr2=0x7f1a640c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6415b440 0x7f1a640c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 -- 172.21.15.106:0/1452530872 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50009400 con 0x7f1a640cc0f0 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.089+0000 7f1a63fff640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c68c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1a50034040 tx=0x7f1a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.096 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.090+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50003810 con 0x7f1a640cc0f0 2023-11-24T08:13:08.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.090+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a50040070 con 0x7f1a640cc0f0 2023-11-24T08:13:08.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.090+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a50003b70 con 0x7f1a640cc0f0 2023-11-24T08:13:08.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.090+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6409bd00 con 0x7f1a640cc0f0 2023-11-24T08:13:08.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.090+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a64151150 con 0x7f1a640cc0f0 2023-11-24T08:13:08.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.091+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6409c220 con 0x7f1a640cc0f0 2023-11-24T08:13:08.097 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.092+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f1a50003cd0 con 0x7f1a640cc0f0 2023-11-24T08:13:08.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.093+0000 7f1a617fa640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1a4806b3e0 0x7f1a4806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.093+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1a50042070 con 0x7f1a640cc0f0 2023-11-24T08:13:08.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.093+0000 7f1a689bf640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1a4806b3e0 0x7f1a4806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.094+0000 7f1a689bf640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1a4806b3e0 0x7f1a4806d8a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000f60 tx=0x7f1a580078f0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.098 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.097+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1a6409c220 con 0x7f1a640cc0f0 2023-11-24T08:13:08.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.285+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1a6409c220 con 0x7f1a640cc0f0 2023-11-24T08:13:08.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.287+0000 7f1a617fa640 1 -- 172.21.15.106:0/1452530872 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1a6409c220 con 0x7f1a640cc0f0 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1a4806b3e0 msgr2=0x7f1a4806d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1a4806b3e0 0x7f1a4806d8a0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f1a58000f60 tx=0x7f1a580078f0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 msgr2=0x7f1a640c68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c68c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f1a50034040 tx=0x7f1a500046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 shutdown_connections 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1a6415b440 0x7f1a640c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1a640cc0f0 0x7f1a640c68c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.291 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1a4806b3e0 0x7f1a4806d8a0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 --2- 172.21.15.106:0/1452530872 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1a640c57e0 0x7f1a64150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 >> 172.21.15.106:0/1452530872 conn(0x7f1a640b0c50 msgr2=0x7f1a640719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:08.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.289+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 shutdown_connections 2023-11-24T08:13:08.292 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.290+0000 7f1a6ac23640 1 -- 172.21.15.106:0/1452530872 wait complete. 2023-11-24T08:13:08.310 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs subvol_grp6338 2023-11-24T08:13:08.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.520+0000 7f63bcee5640 1 Processor -- start 2023-11-24T08:13:08.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.520+0000 7f63bcee5640 1 -- start start 2023-11-24T08:13:08.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63bcee5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 0x7f63b815e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.523 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63bcee5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b815ec90 con 0x7f63b806de10 2023-11-24T08:13:08.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63bcee5640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63b80cd530 con 0x7f63b8072160 2023-11-24T08:13:08.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63bcee5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80c4ed0 con 0x7f63b80c49f0 2023-11-24T08:13:08.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63b5d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f63b806de10 0x7f63b80705c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48588/0 (socket says 172.21.15.106:48588) 2023-11-24T08:13:08.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63b5d9b640 1 -- 172.21.15.106:0/882467240 learned_addr learned my addr 172.21.15.106:0/882467240 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:08.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.522+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233950893 0 0) 0x7f63ac001110 con 0x7f63b806de10 2023-11-24T08:13:08.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b6d9d640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 0x7f63b815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a0005730 con 0x7f63b806de10 2023-11-24T08:13:08.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 637397964 0 0) 0x7f63a0005730 con 0x7f63b806de10 2023-11-24T08:13:08.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 msgr2=0x7f63b815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:08.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 0x7f63b815e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 >> v1:172.21.15.156:6789/0 conn(0x7f63b8072160 legacy=0x7f63b806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b80c51b0 con 0x7f63b806de10 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f63ac001800 con 0x7f63b806de10 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 >> v1:172.21.15.106:6789/0 conn(0x7f63b806de10 legacy=0x7f63b80705c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 0x7f63a0005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a0006180 0x7f63a00069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0006f10 0x7f63a0007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b80c4ed0 con 0x7f63a0006f10 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a000dcd0 con 0x7f63a0006180 2023-11-24T08:13:08.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.523+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a000de90 con 0x7f63b80c49f0 2023-11-24T08:13:08.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b6d9d640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0006f10 0x7f63a0007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b5d9b640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a0006180 0x7f63a00069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b659c640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 0x7f63a0005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b6d9d640 1 -- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 msgr2=0x7f63a0005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:08.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b6d9d640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63b80c49f0 0x7f63a0005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b6d9d640 1 -- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a0006180 msgr2=0x7f63a00069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:08.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b6d9d640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63a0006180 0x7f63a00069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.524+0000 7f63b6d9d640 1 -- 172.21.15.106:0/882467240 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a0005730 con 0x7f63a0006f10 2023-11-24T08:13:08.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.525+0000 7f63b6d9d640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0006f10 0x7f63a0007390 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f63b806ead0 tx=0x7f63a0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0764a76ea4edb89 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.525+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b002cb90 con 0x7f63a0006f10 2023-11-24T08:13:08.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.525+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b000f040 con 0x7f63a0006f10 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.525+0000 7f63b559a640 1 -- 172.21.15.106:0/882467240 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0034440 con 0x7f63a0006f10 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63bcee5640 1 -- 172.21.15.106:0/882467240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0006f10 msgr2=0x7f63a0007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63bcee5640 1 --2- 172.21.15.106:0/882467240 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63a0006f10 0x7f63a0007390 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f63b806ead0 tx=0x7f63a0007a00 comp rx=0 tx=0).stop 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63b6d9d640 1 -- 172.21.15.106:0/882467240 reap_dead start 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63bcee5640 1 -- 172.21.15.106:0/882467240 shutdown_connections 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63bcee5640 1 -- 172.21.15.106:0/882467240 >> 172.21.15.106:0/882467240 conn(0x7f63b80b0c50 msgr2=0x7f63b80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63bcee5640 1 -- 172.21.15.106:0/882467240 shutdown_connections 2023-11-24T08:13:08.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.526+0000 7f63bcee5640 1 -- 172.21.15.106:0/882467240 wait complete. 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.528+0000 7f63bcee5640 1 Processor -- start 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.528+0000 7f63bcee5640 1 -- start start 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63bcee5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63a0006f10 0x7f63b81541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63bcee5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 0x7f63b80b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63bcee5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b80b4920 0x7f63b80b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63bcee5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8154700 con 0x7f63b80999d0 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63bcee5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b815c0f0 con 0x7f63b80b4920 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63bcee5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63b815b520 con 0x7f63a0006f10 2023-11-24T08:13:08.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63b5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 0x7f63b80b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63b5d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 0x7f63b80b43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57042/0 (socket says 172.21.15.106:57042) 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.529+0000 7f63b5d9b640 1 -- 172.21.15.106:0/1065167446 learned_addr learned my addr 172.21.15.106:0/1065167446 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b659c640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63a0006f10 0x7f63b81541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b5d9b640 1 -- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63a0006f10 msgr2=0x7f63b81541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b5d9b640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63a0006f10 0x7f63b81541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b5d9b640 1 -- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b80b4920 msgr2=0x7f63b80b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b5d9b640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b80b4920 0x7f63b80b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b5d9b640 1 -- 172.21.15.106:0/1065167446 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b815b7c0 con 0x7f63b80999d0 2023-11-24T08:13:08.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63b5d9b640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 0x7f63b80b43e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00e7e0 tx=0x7f63ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00c850 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63ac00c9b0 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63ac00cd60 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b815baa0 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.530+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b80c9220 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.532+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b806ef60 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.532+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7f63ac01d460 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.533+0000 7f63a6ffd640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f639806b410 0x7f639806d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.533+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63ac089e10 con 0x7f63b80999d0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.534+0000 7f63b659c640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f639806b410 0x7f639806d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.534+0000 7f63b659c640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f639806b410 0x7f639806d8d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f63a8007fd0 tx=0x7f63a8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.539 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.538+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f63ac0546a0 con 0x7f63b80999d0 2023-11-24T08:13:08.749 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.748+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "subvol_grp6338", "target": ["mon-mgr", ""]}) v1 -- 0x7f63b80c5a50 con 0x7f639806b410 2023-11-24T08:13:08.757 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:13:08.755+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:13:08.757 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:13:08.755+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:13:08.757 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:13:08.755+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:13:08.757 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:13:08.755+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:13:08.757 INFO:tasks.ceph.mgr.y.smithi106.stderr:2023-11-24T08:13:08.755+0000 7f67aa1bf640 -1 client.0 error registering admin socket command: (17) File exists 2023-11-24T08:13:08.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.776+0000 7f63a6ffd640 1 -- 172.21.15.106:0/1065167446 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f63a800d050 con 0x7f639806b410 2023-11-24T08:13:08.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f639806b410 msgr2=0x7f639806d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f639806b410 0x7f639806d8d0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f63a8007fd0 tx=0x7f63a8009040 comp rx=0 tx=0).stop 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 msgr2=0x7f63b80b43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 0x7f63b80b43e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f63ac00e7e0 tx=0x7f63ac00eca0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 shutdown_connections 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63b80b4920 0x7f63b80b23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f63b80999d0 0x7f63b80b43e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f639806b410 0x7f639806d8d0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 --2- 172.21.15.106:0/1065167446 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f63a0006f10 0x7f63b81541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 >> 172.21.15.106:0/1065167446 conn(0x7f63b80b0c50 msgr2=0x7f63b8154970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 shutdown_connections 2023-11-24T08:13:08.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.779+0000 7f63bcee5640 1 -- 172.21.15.106:0/1065167446 wait complete. 2023-11-24T08:13:08.797 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol7599 --group_name subvol_grp6338 2023-11-24T08:13:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.989+0000 7efcbfed0640 1 Processor -- start 2023-11-24T08:13:08.991 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.989+0000 7efcbfed0640 1 -- start start 2023-11-24T08:13:08.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbfed0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcb815c7f0 0x7efcb806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbfed0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cf450 0x7efcb80cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbfed0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7efcb80cdfb0 0x7efcb80ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbfed0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb8156010 con 0x7efcb815c7f0 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbfed0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb806ce30 con 0x7efcb80cdfb0 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbfed0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80cfe10 con 0x7efcb80cf450 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbdc6c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcb815c7f0 0x7efcb806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbdc6c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7efcb815c7f0 0x7efcb806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57072/0 (socket says 172.21.15.106:57072) 2023-11-24T08:13:08.993 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.991+0000 7efcbdc6c640 1 -- 172.21.15.106:0/2954179034 learned_addr learned my addr 172.21.15.106:0/2954179034 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:08.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.992+0000 7efcbd46b640 1 --2- 172.21.15.106:0/2954179034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cf450 0x7efcb80cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:08.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.992+0000 7efcbdc6c640 1 -- 172.21.15.106:0/2954179034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cf450 msgr2=0x7efcb80cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:08.994 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.992+0000 7efcbdc6c640 1 --2- 172.21.15.106:0/2954179034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cf450 0x7efcb80cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.992+0000 7efcbdc6c640 1 -- 172.21.15.106:0/2954179034 >> v2:172.21.15.156:3300/0 conn(0x7efcb80cdfb0 msgr2=0x7efcb80ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.992+0000 7efcbdc6c640 1 --2- 172.21.15.106:0/2954179034 >> v2:172.21.15.156:3300/0 conn(0x7efcb80cdfb0 0x7efcb80ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.992+0000 7efcbdc6c640 1 -- 172.21.15.106:0/2954179034 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80cece0 con 0x7efcb815c7f0 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.993+0000 7efcbdc6c640 1 --2- 172.21.15.106:0/2954179034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb815c7f0 0x7efcb806db20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7efcb80aebc0 tx=0x7efca802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d807b14463d309 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.993+0000 7efcbcc6a640 1 -- 172.21.15.106:0/2954179034 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8031870 con 0x7efcb815c7f0 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.993+0000 7efcbcc6a640 1 -- 172.21.15.106:0/2954179034 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca80319d0 con 0x7efcb815c7f0 2023-11-24T08:13:08.995 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.993+0000 7efcbcc6a640 1 -- 172.21.15.106:0/2954179034 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca8031d60 con 0x7efcb815c7f0 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 -- 172.21.15.106:0/2954179034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb815c7f0 msgr2=0x7efcb806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 --2- 172.21.15.106:0/2954179034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb815c7f0 0x7efcb806db20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7efcb80aebc0 tx=0x7efca802b910 comp rx=0 tx=0).stop 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 -- 172.21.15.106:0/2954179034 shutdown_connections 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 --2- 172.21.15.106:0/2954179034 >> v2:172.21.15.156:3300/0 conn(0x7efcb80cdfb0 0x7efcb80ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 --2- 172.21.15.106:0/2954179034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cf450 0x7efcb80cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 --2- 172.21.15.106:0/2954179034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb815c7f0 0x7efcb806db20 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 -- 172.21.15.106:0/2954179034 >> 172.21.15.106:0/2954179034 conn(0x7efcb806a920 msgr2=0x7efcb806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:08.996 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 -- 172.21.15.106:0/2954179034 shutdown_connections 2023-11-24T08:13:08.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.994+0000 7efcbfed0640 1 -- 172.21.15.106:0/2954179034 wait complete. 2023-11-24T08:13:08.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.995+0000 7efcbfed0640 1 Processor -- start 2023-11-24T08:13:08.997 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 -- start start 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cdfb0 0x7efcb809cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 0x7efcb814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c7f0 0x7efcb814f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb81566f0 con 0x7efcb80cf450 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb814f050 con 0x7efcb815c7f0 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.996+0000 7efcbfed0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb814ea90 con 0x7efcb80cdfb0 2023-11-24T08:13:08.999 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 0x7efcb814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 0x7efcb814f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57082/0 (socket says 172.21.15.106:57082) 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 -- 172.21.15.106:0/1016408705 learned_addr learned my addr 172.21.15.106:0/1016408705 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 -- 172.21.15.106:0/1016408705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cdfb0 msgr2=0x7efcb809cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cdfb0 0x7efcb809cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 -- 172.21.15.106:0/1016408705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c7f0 msgr2=0x7efcb814f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c7f0 0x7efcb814f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 -- 172.21.15.106:0/1016408705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca80094d0 con 0x7efcb80cf450 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcbd46b640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 0x7efcb814f3f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c7f0 tx=0x7efcb400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.997+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4004220 con 0x7efcb80cf450 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.998+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb40043a0 con 0x7efcb80cf450 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.998+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb4007860 con 0x7efcb80cf450 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.998+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb814ed70 con 0x7efcb80cf450 2023-11-24T08:13:09.000 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.998+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb809d010 con 0x7efcb80cf450 2023-11-24T08:13:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:08.999+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb8156170 con 0x7efcb80cf450 2023-11-24T08:13:09.001 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.000+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7efcb4007bd0 con 0x7efcb80cf450 2023-11-24T08:13:09.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.000+0000 7efcae7fc640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efca406b3e0 0x7efca406d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.001+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efcb408a150 con 0x7efcb80cf450 2023-11-24T08:13:09.002 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.001+0000 7efcbdc6c640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efca406b3e0 0x7efca406d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.003 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.001+0000 7efcbdc6c640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efca406b3e0 0x7efca406d8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efca8036000 tx=0x7efca8030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:09.006 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.005+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7efcb40bce90 con 0x7efcb80cf450 2023-11-24T08:13:09.213 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.211+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol7599", "group_name": "subvol_grp6338", "target": ["mon-mgr", ""]}) v1 -- 0x7efcb8061ac0 con 0x7efca406b3e0 2023-11-24T08:13:09.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.261+0000 7efcae7fc640 1 -- 172.21.15.106:0/1016408705 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7efcb8061ac0 con 0x7efca406b3e0 2023-11-24T08:13:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efca406b3e0 msgr2=0x7efca406d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efca406b3e0 0x7efca406d8a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7efca8036000 tx=0x7efca8030000 comp rx=0 tx=0).stop 2023-11-24T08:13:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 msgr2=0x7efcb814f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 0x7efcb814f3f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7efcb400c7f0 tx=0x7efcb400ccb0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 shutdown_connections 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcb815c7f0 0x7efcb814f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.264+0000 7efcbfed0640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7efcb80cf450 0x7efcb814f3f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.265+0000 7efcbfed0640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7efca406b3e0 0x7efca406d8a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.265+0000 7efcbfed0640 1 --2- 172.21.15.106:0/1016408705 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7efcb80cdfb0 0x7efcb809cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.265+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 >> 172.21.15.106:0/1016408705 conn(0x7efcb806a920 msgr2=0x7efcb80cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.265+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 shutdown_connections 2023-11-24T08:13:09.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.265+0000 7efcbfed0640 1 -- 172.21.15.106:0/1016408705 wait complete. 2023-11-24T08:13:09.284 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-24T08:13:09.285 DEBUG:teuthology.orchestra.run.smithi106:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:13:09.399 DEBUG:teuthology.orchestra.run.smithi156:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-24T08:13:09.519 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-24T08:13:09.519 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:13:09.519 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:09.580 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:13:09.580 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:09.635 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:09.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.853+0000 7fe18a7fc640 1 Processor -- start 2023-11-24T08:13:09.855 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.853+0000 7fe18a7fc640 1 -- start start 2023-11-24T08:13:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe18a7fc640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe18415c7f0 0x7fe18406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe18a7fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1840cf450 0x7fe1840cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe18a7fc640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1840cece0 con 0x7fe1840cdfb0 2023-11-24T08:13:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe18a7fc640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1840cfe40 con 0x7fe18415c7f0 2023-11-24T08:13:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe18a7fc640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1840ceaf0 con 0x7fe1840cf450 2023-11-24T08:13:09.858 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe183fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe18415c7f0 0x7fe18406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe188d99640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe1840cdfb0 0x7fe1840ce3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48602/0 (socket says 172.21.15.106:48602) 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe188d99640 1 -- 172.21.15.106:0/842325382 learned_addr learned my addr 172.21.15.106:0/842325382 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe1837fe640 1 --2- 172.21.15.106:0/842325382 >> v2:172.21.15.156:3300/0 conn(0x7fe1840cf450 0x7fe1840cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe183fff640 1 -- 172.21.15.106:0/842325382 >> v2:172.21.15.156:3300/0 conn(0x7fe1840cf450 msgr2=0x7fe1840cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.856+0000 7fe183fff640 1 --2- 172.21.15.106:0/842325382 >> v2:172.21.15.156:3300/0 conn(0x7fe1840cf450 0x7fe1840cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.856+0000 7fe183fff640 1 -- 172.21.15.106:0/842325382 >> v1:172.21.15.106:6789/0 conn(0x7fe1840cdfb0 legacy=0x7fe1840ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.855+0000 7fe182ffd640 1 -- 172.21.15.106:0/842325382 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2530072754 0 0) 0x7fe1840cece0 con 0x7fe1840cdfb0 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.856+0000 7fe183fff640 1 -- 172.21.15.106:0/842325382 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18406d040 con 0x7fe18415c7f0 2023-11-24T08:13:09.859 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.856+0000 7fe1837fe640 1 --2- 172.21.15.106:0/842325382 >> v2:172.21.15.156:3300/0 conn(0x7fe1840cf450 0x7fe1840cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.857+0000 7fe183fff640 1 --2- 172.21.15.106:0/842325382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe18415c7f0 0x7fe18406db20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe184070410 tx=0x7fe17402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6485b12ff852773b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.857+0000 7fe182ffd640 1 -- 172.21.15.106:0/842325382 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174031830 con 0x7fe18415c7f0 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.857+0000 7fe182ffd640 1 -- 172.21.15.106:0/842325382 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe174031990 con 0x7fe18415c7f0 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.857+0000 7fe182ffd640 1 -- 172.21.15.106:0/842325382 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe174031d20 con 0x7fe18415c7f0 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.857+0000 7fe18a7fc640 1 -- 172.21.15.106:0/842325382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe18415c7f0 msgr2=0x7fe18406db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.857+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/842325382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe18415c7f0 0x7fe18406db20 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fe184070410 tx=0x7fe17402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.858+0000 7fe18a7fc640 1 -- 172.21.15.106:0/842325382 shutdown_connections 2023-11-24T08:13:09.861 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.858+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/842325382 >> v2:172.21.15.156:3300/0 conn(0x7fe1840cf450 0x7fe1840cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.858+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/842325382 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe18415c7f0 0x7fe18406db20 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.858+0000 7fe18a7fc640 1 -- 172.21.15.106:0/842325382 >> 172.21.15.106:0/842325382 conn(0x7fe18406a920 msgr2=0x7fe18406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:09.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.858+0000 7fe18a7fc640 1 -- 172.21.15.106:0/842325382 shutdown_connections 2023-11-24T08:13:09.862 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.858+0000 7fe18a7fc640 1 -- 172.21.15.106:0/842325382 wait complete. 2023-11-24T08:13:09.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.860+0000 7fe18a7fc640 1 Processor -- start 2023-11-24T08:13:09.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.860+0000 7fe18a7fc640 1 -- start start 2023-11-24T08:13:09.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.861+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1840cdfb0 0x7fe184151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.865 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.861+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 0x7fe18415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.861+0000 7fe18a7fc640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe18415c7f0 0x7fe18415f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.861+0000 7fe18a7fc640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184151ca0 con 0x7fe1840cf450 2023-11-24T08:13:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.861+0000 7fe18a7fc640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe184151a80 con 0x7fe1840cdfb0 2023-11-24T08:13:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.861+0000 7fe18a7fc640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1841f6940 con 0x7fe18415c7f0 2023-11-24T08:13:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 0x7fe18415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.866 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 0x7fe18415eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57098/0 (socket says 172.21.15.106:57098) 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 -- 172.21.15.106:0/3943414034 learned_addr learned my addr 172.21.15.106:0/3943414034 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe183fff640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1840cdfb0 0x7fe184151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe188d99640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe18415c7f0 0x7fe18415f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 -- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe18415c7f0 msgr2=0x7fe18415f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe18415c7f0 0x7fe18415f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 -- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1840cdfb0 msgr2=0x7fe184151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1840cdfb0 0x7fe184151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:09.867 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.862+0000 7fe1837fe640 1 -- 172.21.15.106:0/3943414034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe174009400 con 0x7fe1840cf450 2023-11-24T08:13:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.863+0000 7fe1837fe640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 0x7fe18415eec0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c7f0 tx=0x7fe17800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.863+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe178007a30 con 0x7fe1840cf450 2023-11-24T08:13:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.863+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe178007b90 con 0x7fe1840cf450 2023-11-24T08:13:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.863+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1841f6c20 con 0x7fe1840cf450 2023-11-24T08:13:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.863+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17800ce40 con 0x7fe1840cf450 2023-11-24T08:13:09.868 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.863+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1841f71e0 con 0x7fe1840cf450 2023-11-24T08:13:09.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.864+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1841f7400 con 0x7fe1840cf450 2023-11-24T08:13:09.869 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.865+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe1780113f0 con 0x7fe1840cf450 2023-11-24T08:13:09.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.866+0000 7fe180ff9640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe17006b3e0 0x7fe17006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:09.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.867+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe17808a290 con 0x7fe1840cf450 2023-11-24T08:13:09.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.867+0000 7fe183fff640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe17006b3e0 0x7fe17006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:09.870 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.867+0000 7fe183fff640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe17006b3e0 0x7fe17006d8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe174036000 tx=0x7fe17403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:09.872 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:09.871+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe178054a10 con 0x7fe1840cf450 2023-11-24T08:13:10.064 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.062+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe184067470 con 0x7fe1840cf450 2023-11-24T08:13:10.066 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.065+0000 7fe180ff9640 1 -- 172.21.15.106:0/3943414034 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16608 (secure 0 0 0) 0x7fe178058030 con 0x7fe1840cf450 2023-11-24T08:13:10.067 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:10.067 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:57.727441+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.067+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe17006b3e0 msgr2=0x7fe17006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.067+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe17006b3e0 0x7fe17006d8a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fe174036000 tx=0x7fe17403c040 comp rx=0 tx=0).stop 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.067+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 msgr2=0x7fe18415eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.067+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 0x7fe18415eec0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fe17800c7f0 tx=0x7fe17800cbf0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 shutdown_connections 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe17006b3e0 0x7fe17006d8a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe18415c7f0 0x7fe18415f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1840cf450 0x7fe18415eec0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 --2- 172.21.15.106:0/3943414034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1840cdfb0 0x7fe184151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 >> 172.21.15.106:0/3943414034 conn(0x7fe18406a920 msgr2=0x7fe1840afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:10.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 shutdown_connections 2023-11-24T08:13:10.071 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.068+0000 7fe18a7fc640 1 -- 172.21.15.106:0/3943414034 wait complete. 2023-11-24T08:13:10.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:12:53.987858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:12:55.193239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:13:10.090 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:13:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.303+0000 7fe1d6fb0640 1 Processor -- start 2023-11-24T08:13:10.306 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.303+0000 7fe1d6fb0640 1 -- start start 2023-11-24T08:13:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d6fb0640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe1d006de10 0x7fe1d015e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d6fb0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1d00c49f0 0x7fe1d00705c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d6fb0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d015ec90 con 0x7fe1d0072160 2023-11-24T08:13:10.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d6fb0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d00cd530 con 0x7fe1d00c49f0 2023-11-24T08:13:10.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d6fb0640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d00c4ed0 con 0x7fe1d006de10 2023-11-24T08:13:10.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d4d4c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe1d0072160 0x7fe1d006d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48604/0 (socket says 172.21.15.106:48604) 2023-11-24T08:13:10.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d554d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1d00c49f0 0x7fe1d00705c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1d4d4c640 1 -- 172.21.15.106:0/4074865460 learned_addr learned my addr 172.21.15.106:0/4074865460 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.305+0000 7fe1cffff640 1 --2- 172.21.15.106:0/4074865460 >> v2:172.21.15.156:3300/0 conn(0x7fe1d006de10 0x7fe1d015e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.306+0000 7fe1d554d640 1 -- 172.21.15.106:0/4074865460 >> v2:172.21.15.156:3300/0 conn(0x7fe1d006de10 msgr2=0x7fe1d015e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.306+0000 7fe1d554d640 1 --2- 172.21.15.106:0/4074865460 >> v2:172.21.15.156:3300/0 conn(0x7fe1d006de10 0x7fe1d015e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.306+0000 7fe1d554d640 1 -- 172.21.15.106:0/4074865460 >> v1:172.21.15.106:6789/0 conn(0x7fe1d0072160 legacy=0x7fe1d006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.306+0000 7fe1d554d640 1 -- 172.21.15.106:0/4074865460 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d00c51b0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.306+0000 7fe1cf7fe640 1 -- 172.21.15.106:0/4074865460 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566519363 0 0) 0x7fe1c0001490 con 0x7fe1d0072160 2023-11-24T08:13:10.310 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.307+0000 7fe1d554d640 1 --2- 172.21.15.106:0/4074865460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d00705c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00cef60 tx=0x7fe1c802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40eecefb78f38ec1 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.307+0000 7fe1cf7fe640 1 -- 172.21.15.106:0/4074865460 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8031830 con 0x7fe1d00c49f0 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.307+0000 7fe1cf7fe640 1 -- 172.21.15.106:0/4074865460 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c8031990 con 0x7fe1d00c49f0 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.307+0000 7fe1cf7fe640 1 -- 172.21.15.106:0/4074865460 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c8031d20 con 0x7fe1d00c49f0 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/4074865460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 msgr2=0x7fe1d00705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/4074865460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d00705c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fe1d00cef60 tx=0x7fe1c802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/4074865460 shutdown_connections 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/4074865460 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d00705c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/4074865460 >> v2:172.21.15.156:3300/0 conn(0x7fe1d006de10 0x7fe1d015e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/4074865460 >> 172.21.15.106:0/4074865460 conn(0x7fe1d00b0c50 msgr2=0x7fe1d00b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/4074865460 shutdown_connections 2023-11-24T08:13:10.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.308+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/4074865460 wait complete. 2023-11-24T08:13:10.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.310+0000 7fe1d6fb0640 1 Processor -- start 2023-11-24T08:13:10.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.310+0000 7fe1d6fb0640 1 -- start start 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d6fb0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d006de10 0x7fe1d0159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d6fb0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1d0072160 0x7fe1d00c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d6fb0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d009a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d6fb0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00b1d10 con 0x7fe1d00c49f0 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d6fb0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d00999d0 con 0x7fe1d006de10 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d6fb0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d014a6a0 con 0x7fe1d0072160 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d554d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d009a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d554d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d009a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57126/0 (socket says 172.21.15.106:57126) 2023-11-24T08:13:10.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d554d640 1 -- 172.21.15.106:0/1679957124 learned_addr learned my addr 172.21.15.106:0/1679957124 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1d4d4c640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d006de10 0x7fe1d0159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.311+0000 7fe1cffff640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1d0072160 0x7fe1d00c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d554d640 1 -- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1d0072160 msgr2=0x7fe1d00c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d554d640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1d0072160 0x7fe1d00c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d554d640 1 -- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d006de10 msgr2=0x7fe1d0159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d554d640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d006de10 0x7fe1d0159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d554d640 1 -- 172.21.15.106:0/1679957124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c8009400 con 0x7fe1d00c49f0 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d554d640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d009a700 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8034040 tx=0x7fe1c80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c80317d0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c803a9e0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d00c8f80 con 0x7fe1d00c49f0 2023-11-24T08:13:10.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c803ad90 con 0x7fe1d00c49f0 2023-11-24T08:13:10.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.312+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d0099f00 con 0x7fe1d00c49f0 2023-11-24T08:13:10.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.314+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d004e8b0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.314+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 12) v1 ==== 89741+0+0 (secure 0 0 0) 0x7fe1c80443f0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.316+0000 7fe1cd7fa640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1b006b3e0 0x7fe1b006d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.318 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.316+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1c80ac3f0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.316+0000 7fe1d4d4c640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1b006b3e0 0x7fe1b006d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.317+0000 7fe1d4d4c640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1b006b3e0 0x7fe1b006d8a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00012e0 tx=0x7fe1c0007d30 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:10.322 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.320+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe1c8076c00 con 0x7fe1d00c49f0 2023-11-24T08:13:10.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.403+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe1c8076fd0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.515+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1d015dab0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.516+0000 7fe1cd7fa640 1 -- 172.21.15.106:0/1679957124 <== mon.0 v2:172.21.15.106:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1c808fff0 con 0x7fe1d00c49f0 2023-11-24T08:13:10.519 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:13:10.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.518+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1b006b3e0 msgr2=0x7fe1b006d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1b006b3e0 0x7fe1b006d8a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fe1c00012e0 tx=0x7fe1c0007d30 comp rx=0 tx=0).stop 2023-11-24T08:13:10.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 msgr2=0x7fe1d009a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.521 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d009a700 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fe1c8034040 tx=0x7fe1c80046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 shutdown_connections 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1d00c49f0 0x7fe1d009a700 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe1b006b3e0 0x7fe1b006d8a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1d0072160 0x7fe1d00c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 --2- 172.21.15.106:0/1679957124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1d006de10 0x7fe1d0159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 >> 172.21.15.106:0/1679957124 conn(0x7fe1d00b0c50 msgr2=0x7fe1d00abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 shutdown_connections 2023-11-24T08:13:10.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.519+0000 7fe1d6fb0640 1 -- 172.21.15.106:0/1679957124 wait complete. 2023-11-24T08:13:10.540 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:13:10.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.748+0000 7f428b189640 1 Processor -- start 2023-11-24T08:13:10.751 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.748+0000 7f428b189640 1 -- start start 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.750+0000 7f428b189640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4284140520 0x7f42840c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.750+0000 7f428b189640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4284161b10 0x7f428406ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.750+0000 7f428b189640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4284161fa0 con 0x7f42840c65f0 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.750+0000 7f428b189640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42840c6a70 con 0x7f4284161b10 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.750+0000 7f428b189640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f428415de00 con 0x7f4284140520 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.750+0000 7f4289726640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4284161b10 0x7f428406ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4289726640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f4284161b10 0x7f428406ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57138/0 (socket says 172.21.15.106:57138) 2023-11-24T08:13:10.753 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4289726640 1 -- 172.21.15.106:0/1397774434 learned_addr learned my addr 172.21.15.106:0/1397774434 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:10.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4288f25640 1 --2- 172.21.15.106:0/1397774434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4284140520 0x7f42840c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4289726640 1 -- 172.21.15.106:0/1397774434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4284140520 msgr2=0x7f42840c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:10.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4289726640 1 --2- 172.21.15.106:0/1397774434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4284140520 0x7f42840c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4289726640 1 -- 172.21.15.106:0/1397774434 >> v1:172.21.15.156:6789/0 conn(0x7f42840c65f0 legacy=0x7f4284161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:10.754 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.751+0000 7f4289726640 1 -- 172.21.15.106:0/1397774434 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f428415e0e0 con 0x7f4284161b10 2023-11-24T08:13:10.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.752+0000 7f4289726640 1 --2- 172.21.15.106:0/1397774434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428406ff70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f427c009e50 tx=0x7f427c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac7c0a34d516d070 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:10.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.753+0000 7f427b7fe640 1 -- 172.21.15.106:0/1397774434 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c02ca70 con 0x7f4284161b10 2023-11-24T08:13:10.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.753+0000 7f427b7fe640 1 -- 172.21.15.106:0/1397774434 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c02cbd0 con 0x7f4284161b10 2023-11-24T08:13:10.755 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.753+0000 7f427b7fe640 1 -- 172.21.15.106:0/1397774434 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0343f0 con 0x7f4284161b10 2023-11-24T08:13:10.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.753+0000 7f428b189640 1 -- 172.21.15.106:0/1397774434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 msgr2=0x7f428406ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.753+0000 7f428b189640 1 --2- 172.21.15.106:0/1397774434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428406ff70 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f427c009e50 tx=0x7f427c02bed0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.753+0000 7f428b189640 1 -- 172.21.15.106:0/1397774434 shutdown_connections 2023-11-24T08:13:10.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.754+0000 7f428b189640 1 --2- 172.21.15.106:0/1397774434 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428406ff70 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.754+0000 7f428b189640 1 --2- 172.21.15.106:0/1397774434 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f4284140520 0x7f42840c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.756 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.754+0000 7f428b189640 1 -- 172.21.15.106:0/1397774434 >> 172.21.15.106:0/1397774434 conn(0x7f42840b2e40 msgr2=0x7f42840b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:10.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.754+0000 7f428b189640 1 -- 172.21.15.106:0/1397774434 shutdown_connections 2023-11-24T08:13:10.757 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.754+0000 7f428b189640 1 -- 172.21.15.106:0/1397774434 wait complete. 2023-11-24T08:13:10.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.755+0000 7f428b189640 1 Processor -- start 2023-11-24T08:13:10.758 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.756+0000 7f428b189640 1 -- start start 2023-11-24T08:13:10.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.756+0000 7f428b189640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42840c65f0 0x7f428406cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f428b189640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4284140520 0x7f42841f6790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.759 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f428b189640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428415bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f428b189640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42840cce10 con 0x7f4284161b10 2023-11-24T08:13:10.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f428b189640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f428406bf30 con 0x7f4284140520 2023-11-24T08:13:10.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f428b189640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42841581f0 con 0x7f42840c65f0 2023-11-24T08:13:10.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428415bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.760 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428415bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57142/0 (socket says 172.21.15.106:57142) 2023-11-24T08:13:10.761 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 -- 172.21.15.106:0/53511587 learned_addr learned my addr 172.21.15.106:0/53511587 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4288f25640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42840c65f0 0x7f428406cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f427bfff640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4284140520 0x7f42841f6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 -- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42840c65f0 msgr2=0x7f428406cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42840c65f0 0x7f428406cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 -- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4284140520 msgr2=0x7f42841f6790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4284140520 0x7f42841f6790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.757+0000 7f4289726640 1 -- 172.21.15.106:0/53511587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c009b30 con 0x7f4284161b10 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.758+0000 7f4289726640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428415bbb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f427c009e50 tx=0x7f427c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.758+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c009510 con 0x7f4284161b10 2023-11-24T08:13:10.762 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.758+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c03e450 con 0x7f4284161b10 2023-11-24T08:13:10.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.758+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c03e800 con 0x7f4284161b10 2023-11-24T08:13:10.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.758+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42841584d0 con 0x7f4284161b10 2023-11-24T08:13:10.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.758+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4284158a90 con 0x7f4284161b10 2023-11-24T08:13:10.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.759+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f428404e8b0 con 0x7f4284161b10 2023-11-24T08:13:10.763 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.760+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f427c03e960 con 0x7f4284161b10 2023-11-24T08:13:10.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.762+0000 7f42797fa640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427006b470 0x7f427006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:10.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.762+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f427c0acf20 con 0x7f4284161b10 2023-11-24T08:13:10.764 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.762+0000 7f4288f25640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427006b470 0x7f427006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:10.765 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.763+0000 7f4288f25640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427006b470 0x7f427006d930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4280006fd0 tx=0x7f4280008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:10.769 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.766+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f427c077690 con 0x7f4284161b10 2023-11-24T08:13:10.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.956+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f42840701c0 con 0x7f4284161b10 2023-11-24T08:13:10.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.957+0000 7f42797fa640 1 -- 172.21.15.106:0/53511587 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f427c07acb0 con 0x7f4284161b10 2023-11-24T08:13:10.959 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:13:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.959+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427006b470 msgr2=0x7f427006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.959+0000 7f428b189640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427006b470 0x7f427006d930 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4280006fd0 tx=0x7f4280008040 comp rx=0 tx=0).stop 2023-11-24T08:13:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.959+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 msgr2=0x7f428415bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:10.962 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.959+0000 7f428b189640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428415bbb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f427c009e50 tx=0x7f427c004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 shutdown_connections 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f4284161b10 0x7f428415bbb0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4284140520 0x7f42841f6790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f427006b470 0x7f427006d930 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 --2- 172.21.15.106:0/53511587 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f42840c65f0 0x7f428406cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 >> 172.21.15.106:0/53511587 conn(0x7f42840b2e40 msgr2=0x7f42840709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 shutdown_connections 2023-11-24T08:13:10.963 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:10.960+0000 7f428b189640 1 -- 172.21.15.106:0/53511587 wait complete. 2023-11-24T08:13:10.981 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:13:11.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.191+0000 7f37dd8d3640 1 Processor -- start 2023-11-24T08:13:11.194 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.192+0000 7f37dd8d3640 1 -- start start 2023-11-24T08:13:11.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.193+0000 7f37dd8d3640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37d80d4010 0x7f37d80cead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.193+0000 7f37dd8d3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d80cfc80 0x7f37d80d0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.193+0000 7f37dd8d3640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d80d05a0 con 0x7f37d815d580 2023-11-24T08:13:11.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.193+0000 7f37dd8d3640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d80cf300 con 0x7f37d80d4010 2023-11-24T08:13:11.196 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.193+0000 7f37dd8d3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80d1090 con 0x7f37d80cfc80 2023-11-24T08:13:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.194+0000 7f37d67fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37d80d4010 0x7f37d80cead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.194+0000 7f37d67fc640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f37d80d4010 0x7f37d80cead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:37498/0 (socket says 172.21.15.106:37498) 2023-11-24T08:13:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.194+0000 7f37d67fc640 1 -- 172.21.15.106:0/4049098487 learned_addr learned my addr 172.21.15.106:0/4049098487 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.194+0000 7f37d77fe640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d80cfc80 0x7f37d80d0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.197 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.195+0000 7f37d67fc640 1 -- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d80cfc80 msgr2=0x7f37d80d0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.195+0000 7f37d67fc640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d80cfc80 0x7f37d80d0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.195+0000 7f37d67fc640 1 -- 172.21.15.106:0/4049098487 >> v1:172.21.15.156:6789/0 conn(0x7f37d815d580 legacy=0x7f37d80d1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.195+0000 7f37d67fc640 1 -- 172.21.15.106:0/4049098487 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80d1370 con 0x7f37d80d4010 2023-11-24T08:13:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.195+0000 7f37d77fe640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d80cfc80 0x7f37d80d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-24T08:13:11.198 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.196+0000 7f37d67fc640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80cead0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009870 tx=0x7f37c002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6634d04f91de9690 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37d5ffb640 1 -- 172.21.15.106:0/4049098487 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0030840 con 0x7f37d80d4010 2023-11-24T08:13:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37d5ffb640 1 -- 172.21.15.106:0/4049098487 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c00309a0 con 0x7f37d80d4010 2023-11-24T08:13:11.199 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37d5ffb640 1 -- 172.21.15.106:0/4049098487 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c0030d30 con 0x7f37d80d4010 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37dd8d3640 1 -- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 msgr2=0x7f37d80cead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80cead0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009870 tx=0x7f37c002b980 comp rx=0 tx=0).stop 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37dd8d3640 1 -- 172.21.15.106:0/4049098487 shutdown_connections 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d80cfc80 0x7f37d80d0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/4049098487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80cead0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.197+0000 7f37dd8d3640 1 -- 172.21.15.106:0/4049098487 >> 172.21.15.106:0/4049098487 conn(0x7f37d809c770 msgr2=0x7f37d80d3420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.198+0000 7f37dd8d3640 1 -- 172.21.15.106:0/4049098487 shutdown_connections 2023-11-24T08:13:11.200 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.198+0000 7f37dd8d3640 1 -- 172.21.15.106:0/4049098487 wait complete. 2023-11-24T08:13:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.199+0000 7f37dd8d3640 1 Processor -- start 2023-11-24T08:13:11.202 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.199+0000 7f37dd8d3640 1 -- start start 2023-11-24T08:13:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.200+0000 7f37dd8d3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 0x7f37d814f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.206 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.200+0000 7f37dd8d3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.200+0000 7f37dd8d3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d815d580 0x7f37d80c9910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.200+0000 7f37dd8d3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80c70b0 con 0x7f37d80cfc80 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.200+0000 7f37dd8d3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d809e600 con 0x7f37d80d4010 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37dd8d3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80cb3e0 con 0x7f37d815d580 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 0x7f37d814f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 0x7f37d814f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57154/0 (socket says 172.21.15.106:57154) 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 -- 172.21.15.106:0/3440785475 learned_addr learned my addr 172.21.15.106:0/3440785475 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d67fc640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d77fe640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d815d580 0x7f37d80c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 -- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d815d580 msgr2=0x7f37d80c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d815d580 0x7f37d80c9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.207 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 -- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 msgr2=0x7f37d80c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d6ffd640 1 -- 172.21.15.106:0/3440785475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0009400 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.201+0000 7f37d67fc640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.202+0000 7f37d6ffd640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 0x7f37d814f750 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c7c0 tx=0x7f37c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.202+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c8003b90 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.202+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c8003cf0 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.202+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c00094d0 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.202+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c800aa90 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.202+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d80cb9a0 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.203+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d809e770 con 0x7f37d80cfc80 2023-11-24T08:13:11.208 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.204+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f37c800abf0 con 0x7f37d80cfc80 2023-11-24T08:13:11.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.205+0000 7f37cffff640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f37b806b4a0 0x7f37b806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.206+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37c808a000 con 0x7f37d80cfc80 2023-11-24T08:13:11.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.206+0000 7f37d67fc640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f37b806b4a0 0x7f37b806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.209 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.206+0000 7f37d67fc640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f37b806b4a0 0x7f37b806d960 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37c0036000 tx=0x7f37c003b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:11.212 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.210+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f37c8054730 con 0x7f37d80cfc80 2023-11-24T08:13:11.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.398+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37d804e8b0 con 0x7f37d80cfc80 2023-11-24T08:13:11.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.400+0000 7f37cffff640 1 -- 172.21.15.106:0/3440785475 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37c8057d50 con 0x7f37d80cfc80 2023-11-24T08:13:11.402 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f37b806b4a0 msgr2=0x7f37b806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f37b806b4a0 0x7f37b806d960 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f37c0036000 tx=0x7f37c003b040 comp rx=0 tx=0).stop 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 msgr2=0x7f37d814f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 0x7f37d814f750 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f37c800c7c0 tx=0x7f37c800cc80 comp rx=0 tx=0).stop 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 shutdown_connections 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f37b806b4a0 0x7f37b806d960 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f37d815d580 0x7f37d80c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37d80d4010 0x7f37d80c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 --2- 172.21.15.106:0/3440785475 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f37d80cfc80 0x7f37d814f750 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 >> 172.21.15.106:0/3440785475 conn(0x7f37d809c770 msgr2=0x7f37d80b2ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 shutdown_connections 2023-11-24T08:13:11.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.402+0000 7f37dd8d3640 1 -- 172.21.15.106:0/3440785475 wait complete. 2023-11-24T08:13:11.421 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:13:11.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.619+0000 7f20bf59e640 1 Processor -- start 2023-11-24T08:13:11.622 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.619+0000 7f20bf59e640 1 -- start start 2023-11-24T08:13:11.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20bf59e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20c0159920 0x7f20c00ae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20bf59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f20c0159d00 0x7f20c006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20bf59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c006dc40 0x7f20c00c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.624 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20bf59e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c006d270 con 0x7f20c0159920 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20bf59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20c006c660 con 0x7f20c0159d00 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20bf59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c006c940 con 0x7f20c006dc40 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20be59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20c0159920 0x7f20c00ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20be59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f20c0159920 0x7f20c00ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57156/0 (socket says 172.21.15.106:57156) 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.621+0000 7f20be59c640 1 -- 172.21.15.106:0/489225845 learned_addr learned my addr 172.21.15.106:0/489225845 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.622+0000 7f20be59c640 1 -- 172.21.15.106:0/489225845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c006dc40 msgr2=0x7f20c00c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.622+0000 7f20be59c640 1 --2- 172.21.15.106:0/489225845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c006dc40 0x7f20c00c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.622+0000 7f20be59c640 1 -- 172.21.15.106:0/489225845 >> v2:172.21.15.156:3300/0 conn(0x7f20c0159d00 msgr2=0x7f20c006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.622+0000 7f20be59c640 1 --2- 172.21.15.106:0/489225845 >> v2:172.21.15.156:3300/0 conn(0x7f20c0159d00 0x7f20c006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.625 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.622+0000 7f20be59c640 1 -- 172.21.15.106:0/489225845 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c006cc20 con 0x7f20c0159920 2023-11-24T08:13:11.626 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.623+0000 7f20be59c640 1 --2- 172.21.15.106:0/489225845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c00ae310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f20c00c6120 tx=0x7f20b002de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725bf0e99adcdd8b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.623+0000 7f20bdd9b640 1 -- 172.21.15.106:0/489225845 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b002ebc0 con 0x7f20c0159920 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.623+0000 7f20bdd9b640 1 -- 172.21.15.106:0/489225845 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20b0032070 con 0x7f20c0159920 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.623+0000 7f20bdd9b640 1 -- 172.21.15.106:0/489225845 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20b0038400 con 0x7f20c0159920 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 -- 172.21.15.106:0/489225845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 msgr2=0x7f20c00ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 --2- 172.21.15.106:0/489225845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c00ae310 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f20c00c6120 tx=0x7f20b002de10 comp rx=0 tx=0).stop 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 -- 172.21.15.106:0/489225845 shutdown_connections 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 --2- 172.21.15.106:0/489225845 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c006dc40 0x7f20c00c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 --2- 172.21.15.106:0/489225845 >> v2:172.21.15.156:3300/0 conn(0x7f20c0159d00 0x7f20c006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 --2- 172.21.15.106:0/489225845 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c00ae310 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 -- 172.21.15.106:0/489225845 >> 172.21.15.106:0/489225845 conn(0x7f20c00cbb40 msgr2=0x7f20c00cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 -- 172.21.15.106:0/489225845 shutdown_connections 2023-11-24T08:13:11.627 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.624+0000 7f20bf59e640 1 -- 172.21.15.106:0/489225845 wait complete. 2023-11-24T08:13:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.626+0000 7f20bf59e640 1 Processor -- start 2023-11-24T08:13:11.628 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.626+0000 7f20bf59e640 1 -- start start 2023-11-24T08:13:11.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bf59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c006dc40 0x7f20c00c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bf59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c009a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bf59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c0159d00 0x7f20c009ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bf59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00acbd0 con 0x7f20c0159920 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bf59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00aceb0 con 0x7f20c006dc40 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bf59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20c00ad190 con 0x7f20c0159d00 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c009a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c009a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57160/0 (socket says 172.21.15.106:57160) 2023-11-24T08:13:11.631 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 -- 172.21.15.106:0/3037601021 learned_addr learned my addr 172.21.15.106:0/3037601021 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20be59c640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c006dc40 0x7f20c00c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20bed9d640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c0159d00 0x7f20c009ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 -- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c0159d00 msgr2=0x7f20c009ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c0159d00 0x7f20c009ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 -- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c006dc40 msgr2=0x7f20c00c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c006dc40 0x7f20c00c4ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.627+0000 7f20b7fff640 1 -- 172.21.15.106:0/3037601021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b000b760 con 0x7f20c0159920 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.628+0000 7f20b7fff640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c009a710 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f20a800b410 tx=0x7f20a800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.628+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8011830 con 0x7f20c0159920 2023-11-24T08:13:11.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.628+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a80119b0 con 0x7f20c0159920 2023-11-24T08:13:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.628+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8011d60 con 0x7f20c0159920 2023-11-24T08:13:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.628+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20c00ad470 con 0x7f20c0159920 2023-11-24T08:13:11.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.628+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20c00ac860 con 0x7f20c0159920 2023-11-24T08:13:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.629+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20c015a110 con 0x7f20c0159920 2023-11-24T08:13:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.630+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f20a801a400 con 0x7f20c0159920 2023-11-24T08:13:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.631+0000 7f20b6ffd640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20a006b470 0x7f20a006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.631+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20a808a310 con 0x7f20c0159920 2023-11-24T08:13:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.631+0000 7f20be59c640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20a006b470 0x7f20a006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:11.634 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.632+0000 7f20be59c640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20a006b470 0x7f20a006d930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f20b0039000 tx=0x7f20b0034000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:11.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.635+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f20a8054a40 con 0x7f20c0159920 2023-11-24T08:13:11.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.845+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20c004e840 con 0x7f20c0159920 2023-11-24T08:13:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.846+0000 7f20b6ffd640 1 -- 172.21.15.106:0/3037601021 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7f20a8058060 con 0x7f20c0159920 2023-11-24T08:13:11.849 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20a006b470 msgr2=0x7f20a006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20a006b470 0x7f20a006d930 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f20b0039000 tx=0x7f20b0034000 comp rx=0 tx=0).stop 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 msgr2=0x7f20c009a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c009a710 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f20a800b410 tx=0x7f20a800b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 shutdown_connections 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f20a006b470 0x7f20a006d930 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f20c0159d00 0x7f20c009ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f20c0159920 0x7f20c009a710 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 --2- 172.21.15.106:0/3037601021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20c006dc40 0x7f20c00c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.849+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 >> 172.21.15.106:0/3037601021 conn(0x7f20c00cbb40 msgr2=0x7f20c014f680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.850+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 shutdown_connections 2023-11-24T08:13:11.853 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:11.850+0000 7f20bf59e640 1 -- 172.21.15.106:0/3037601021 wait complete. 2023-11-24T08:13:11.868 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-24T08:13:11.869 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:11.869 DEBUG:teuthology.orchestra.run.smithi106:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-24T08:13:11.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:13:11.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:11.970 DEBUG:teuthology.orchestra.run.smithi106:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-24T08:13:12.018 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:13:12.018 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:12.079 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:13:12.079 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:12.131 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:12.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.348+0000 7f32bad4c640 1 Processor -- start 2023-11-24T08:13:12.351 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.348+0000 7f32bad4c640 1 -- start start 2023-11-24T08:13:12.353 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.350+0000 7f32bad4c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b415e390 0x7f32b4161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.353 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.350+0000 7f32bad4c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b40ce0b0 con 0x7f32b40cfb60 2023-11-24T08:13:12.353 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.350+0000 7f32bad4c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b4161960 con 0x7f32b4157c70 2023-11-24T08:13:12.353 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.350+0000 7f32bad4c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b415e8d0 con 0x7f32b415e390 2023-11-24T08:13:12.353 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.351+0000 7f32b3fff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f32b40cfb60 0x7f32b415dc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48606/0 (socket says 172.21.15.106:48606) 2023-11-24T08:13:12.353 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.351+0000 7f32b3fff640 1 -- 172.21.15.106:0/1128779896 learned_addr learned my addr 172.21.15.106:0/1128779896 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:12.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.351+0000 7f32b92e9640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b415e390 0x7f32b4161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.351+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897081743 0 0) 0x7f32b40ce0b0 con 0x7f32b40cfb60 2023-11-24T08:13:12.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.351+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32a4003520 con 0x7f32b40cfb60 2023-11-24T08:13:12.355 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.351+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3252587764 0 0) 0x7f32a4003520 con 0x7f32b40cfb60 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b415e390 msgr2=0x7f32b4161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b415e390 0x7f32b4161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 >> v1:172.21.15.156:6789/0 conn(0x7f32b4157c70 legacy=0x7f32b40cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b415ebb0 con 0x7f32b40cfb60 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f32a0001670 con 0x7f32b40cfb60 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 >> v1:172.21.15.106:6789/0 conn(0x7f32b40cfb60 legacy=0x7f32b415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32a40039a0 0x7f32a4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a40051d0 0x7f32a400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b415e8d0 con 0x7f32a40047c0 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a400c0d0 con 0x7f32a40051d0 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a400c290 con 0x7f32a40039a0 2023-11-24T08:13:12.356 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b92e9640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b415e390 0x7f32b4161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b3fff640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b8ae8640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32a40039a0 0x7f32a4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b3fff640 1 -- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32a40039a0 msgr2=0x7f32a4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b3fff640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32a40039a0 0x7f32a4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b3fff640 1 -- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a40051d0 msgr2=0x7f32a400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b3fff640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32a40051d0 0x7f32a400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.357 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.352+0000 7f32b3fff640 1 -- 172.21.15.106:0/1128779896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a4003520 con 0x7f32a40047c0 2023-11-24T08:13:12.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.353+0000 7f32b3fff640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c240 tx=0x7f32a40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c39d460406f245 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:12.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a0005ad0 con 0x7f32a40047c0 2023-11-24T08:13:12.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a0005c30 con 0x7f32a40047c0 2023-11-24T08:13:12.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32b37fe640 1 -- 172.21.15.106:0/1128779896 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a002caf0 con 0x7f32a40047c0 2023-11-24T08:13:12.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32bad4c640 1 -- 172.21.15.106:0/1128779896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 msgr2=0x7f32a4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.358 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1128779896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32a4004c40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f32a000c240 tx=0x7f32a40057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32b92e9640 1 -- 172.21.15.106:0/1128779896 reap_dead start 2023-11-24T08:13:12.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32bad4c640 1 -- 172.21.15.106:0/1128779896 shutdown_connections 2023-11-24T08:13:12.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.354+0000 7f32bad4c640 1 -- 172.21.15.106:0/1128779896 >> 172.21.15.106:0/1128779896 conn(0x7f32b406a920 msgr2=0x7f32b406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:12.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.355+0000 7f32bad4c640 1 -- 172.21.15.106:0/1128779896 shutdown_connections 2023-11-24T08:13:12.359 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.355+0000 7f32bad4c640 1 -- 172.21.15.106:0/1128779896 wait complete. 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.356+0000 7f32bad4c640 1 Processor -- start 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.356+0000 7f32bad4c640 1 -- start start 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.357+0000 7f32bad4c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32b40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.357+0000 7f32bad4c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b414b810 0x7f32b409b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.357+0000 7f32bad4c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b409bc30 0x7f32b40c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.357+0000 7f32bad4c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b414b630 con 0x7f32a40047c0 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.357+0000 7f32bad4c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b409c0b0 con 0x7f32b409bc30 2023-11-24T08:13:12.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.357+0000 7f32bad4c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b414b1a0 con 0x7f32b414b810 2023-11-24T08:13:12.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32b40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32b40caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57188/0 (socket says 172.21.15.106:57188) 2023-11-24T08:13:12.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 -- 172.21.15.106:0/1439407327 learned_addr learned my addr 172.21.15.106:0/1439407327 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 -- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b414b810 msgr2=0x7f32b409b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b414b810 0x7f32b409b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 -- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b409bc30 msgr2=0x7f32b40c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b92e9640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b409bc30 0x7f32b40c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b409bc30 0x7f32b40c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.358+0000 7f32b8ae8640 1 -- 172.21.15.106:0/1439407327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b414a590 con 0x7f32a40047c0 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.359+0000 7f32b8ae8640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32b40caeb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f32a800e550 tx=0x7f32a800ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.359+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a80052e0 con 0x7f32a40047c0 2023-11-24T08:13:12.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.359+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a8005440 con 0x7f32a40047c0 2023-11-24T08:13:12.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.359+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b414a870 con 0x7f32a40047c0 2023-11-24T08:13:12.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.359+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a8003810 con 0x7f32a40047c0 2023-11-24T08:13:12.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.359+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b414ae30 con 0x7f32a40047c0 2023-11-24T08:13:12.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.360+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b4067470 con 0x7f32a40047c0 2023-11-24T08:13:12.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.361+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f32a8003970 con 0x7f32a40047c0 2023-11-24T08:13:12.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.362+0000 7f32b17fa640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f329806b470 0x7f329806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.362+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f32a8089e70 con 0x7f32a40047c0 2023-11-24T08:13:12.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.362+0000 7f32b3fff640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f329806b470 0x7f329806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.365 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.363+0000 7f32b3fff640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f329806b470 0x7f329806d930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f32a0034000 tx=0x7f32a0007000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:12.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.367+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32a8054550 con 0x7f32a40047c0 2023-11-24T08:13:12.561 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.558+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32b40cf3f0 con 0x7f32a40047c0 2023-11-24T08:13:12.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.561+0000 7f32b17fa640 1 -- 172.21.15.106:0/1439407327 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16608 (secure 0 0 0) 0x7f32b40cf3f0 con 0x7f32a40047c0 2023-11-24T08:13:12.564 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:12.564 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:57.727441+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f329806b470 msgr2=0x7f329806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f329806b470 0x7f329806d930 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f32a0034000 tx=0x7f32a0007000 comp rx=0 tx=0).stop 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 msgr2=0x7f32b40caeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32b40caeb0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f32a800e550 tx=0x7f32a800ea10 comp rx=0 tx=0).stop 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 shutdown_connections 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32b409bc30 0x7f32b40c8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f329806b470 0x7f329806d930 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32b414b810 0x7f32b409b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 --2- 172.21.15.106:0/1439407327 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32a40047c0 0x7f32b40caeb0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 >> 172.21.15.106:0/1439407327 conn(0x7f32b406a920 msgr2=0x7f32b4070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 shutdown_connections 2023-11-24T08:13:12.567 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.564+0000 7f32bad4c640 1 -- 172.21.15.106:0/1439407327 wait complete. 2023-11-24T08:13:12.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:12:53.987858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:12:55.193239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:13:12.586 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:13:12.801 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.799+0000 7f705c290640 1 Processor -- start 2023-11-24T08:13:12.802 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.799+0000 7f705c290640 1 -- start start 2023-11-24T08:13:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.800+0000 7f705c290640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7054146eb0 0x7f7054161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.800+0000 7f705c290640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70540cd2e0 0x7f7054070e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.800+0000 7f705c290640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7054161a10 con 0x7f705406e160 2023-11-24T08:13:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.800+0000 7f705c290640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7054161cf0 con 0x7f70540cd2e0 2023-11-24T08:13:12.803 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.800+0000 7f705c290640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7054161fd0 con 0x7f7054146eb0 2023-11-24T08:13:12.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a82d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f70540cd2e0 0x7f7054070e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705982b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f705406e160 0x7f70540ccbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48612/0 (socket says 172.21.15.106:48612) 2023-11-24T08:13:12.804 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705982b640 1 -- 172.21.15.106:0/3238552683 learned_addr learned my addr 172.21.15.106:0/3238552683 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a02c640 1 --2- 172.21.15.106:0/3238552683 >> v2:172.21.15.156:3300/0 conn(0x7f7054146eb0 0x7f7054161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a82d640 1 -- 172.21.15.106:0/3238552683 >> v2:172.21.15.156:3300/0 conn(0x7f7054146eb0 msgr2=0x7f7054161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a82d640 1 --2- 172.21.15.106:0/3238552683 >> v2:172.21.15.156:3300/0 conn(0x7f7054146eb0 0x7f7054161150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a82d640 1 -- 172.21.15.106:0/3238552683 >> v1:172.21.15.106:6789/0 conn(0x7f705406e160 legacy=0x7f70540ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a82d640 1 -- 172.21.15.106:0/3238552683 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7054070660 con 0x7f70540cd2e0 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.801+0000 7f705a02c640 1 --2- 172.21.15.106:0/3238552683 >> v2:172.21.15.156:3300/0 conn(0x7f7054146eb0 0x7f7054161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:12.805 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.802+0000 7f705a82d640 1 --2- 172.21.15.106:0/3238552683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70540cd2e0 0x7f7054070e40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f705406ea50 tx=0x7f704c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d8ffd229a631f server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:12.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.803+0000 7f705902a640 1 -- 172.21.15.106:0/3238552683 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c02cc70 con 0x7f70540cd2e0 2023-11-24T08:13:12.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.803+0000 7f705902a640 1 -- 172.21.15.106:0/3238552683 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c02cdd0 con 0x7f70540cd2e0 2023-11-24T08:13:12.806 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.803+0000 7f705902a640 1 -- 172.21.15.106:0/3238552683 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c034660 con 0x7f70540cd2e0 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.803+0000 7f705c290640 1 -- 172.21.15.106:0/3238552683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70540cd2e0 msgr2=0x7f7054070e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.803+0000 7f705c290640 1 --2- 172.21.15.106:0/3238552683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70540cd2e0 0x7f7054070e40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f705406ea50 tx=0x7f704c02bec0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.803+0000 7f705c290640 1 -- 172.21.15.106:0/3238552683 shutdown_connections 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.804+0000 7f705c290640 1 --2- 172.21.15.106:0/3238552683 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f70540cd2e0 0x7f7054070e40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.804+0000 7f705c290640 1 --2- 172.21.15.106:0/3238552683 >> v2:172.21.15.156:3300/0 conn(0x7f7054146eb0 0x7f7054161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.804+0000 7f705c290640 1 -- 172.21.15.106:0/3238552683 >> 172.21.15.106:0/3238552683 conn(0x7f70540b0c50 msgr2=0x7f70540b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.804+0000 7f705c290640 1 -- 172.21.15.106:0/3238552683 shutdown_connections 2023-11-24T08:13:12.807 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.804+0000 7f705c290640 1 -- 172.21.15.106:0/3238552683 wait complete. 2023-11-24T08:13:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.805+0000 7f705c290640 1 Processor -- start 2023-11-24T08:13:12.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.805+0000 7f705c290640 1 -- start start 2023-11-24T08:13:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.806+0000 7f705c290640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f705406e160 0x7f70541f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.806+0000 7f705c290640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70540cd2e0 0x7f70541f0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.806+0000 7f705c290640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 0x7f705415fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.806+0000 7f705c290640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70541f4120 con 0x7f7054146eb0 2023-11-24T08:13:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.806+0000 7f705c290640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7054146350 con 0x7f705406e160 2023-11-24T08:13:12.809 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.806+0000 7f705c290640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f705415f0d0 con 0x7f70540cd2e0 2023-11-24T08:13:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 0x7f705415fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 0x7f705415fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57204/0 (socket says 172.21.15.106:57204) 2023-11-24T08:13:12.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 -- 172.21.15.106:0/417829565 learned_addr learned my addr 172.21.15.106:0/417829565 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a02c640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f705406e160 0x7f70541f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 -- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70540cd2e0 msgr2=0x7f70541f0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70540cd2e0 0x7f70541f0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 -- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f705406e160 msgr2=0x7f70541f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f705406e160 0x7f70541f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.807+0000 7f705a82d640 1 -- 172.21.15.106:0/417829565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f704c009a70 con 0x7f7054146eb0 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.808+0000 7f705a82d640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 0x7f705415fc70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f704c009e80 tx=0x7f704c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.808+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c040650 con 0x7f7054146eb0 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.808+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f704c0407d0 con 0x7f7054146eb0 2023-11-24T08:13:12.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.808+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f704c040b80 con 0x7f7054146eb0 2023-11-24T08:13:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.808+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f705415e550 con 0x7f7054146eb0 2023-11-24T08:13:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.808+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f705415eb10 con 0x7f7054146eb0 2023-11-24T08:13:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.809+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70540adf40 con 0x7f7054146eb0 2023-11-24T08:13:12.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.810+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f704c040ce0 con 0x7f7054146eb0 2023-11-24T08:13:12.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.811+0000 7f704affd640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f703806b470 0x7f703806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:12.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.811+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f704c0acff0 con 0x7f7054146eb0 2023-11-24T08:13:12.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.811+0000 7f705a02c640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f703806b470 0x7f703806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:12.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.812+0000 7f705a02c640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f703806b470 0x7f703806d930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f70541467b0 tx=0x7f7050008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:12.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:12.815+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f704c077720 con 0x7f7054146eb0 2023-11-24T08:13:13.007 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.004+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f705404e8b0 con 0x7f7054146eb0 2023-11-24T08:13:13.008 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.006+0000 7f704affd640 1 -- 172.21.15.106:0/417829565 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f704c044020 con 0x7f7054146eb0 2023-11-24T08:13:13.009 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f703806b470 msgr2=0x7f703806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f703806b470 0x7f703806d930 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f70541467b0 tx=0x7f7050008040 comp rx=0 tx=0).stop 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 msgr2=0x7f705415fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 0x7f705415fc70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f704c009e80 tx=0x7f704c004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 shutdown_connections 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7054146eb0 0x7f705415fc70 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f703806b470 0x7f703806d930 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f70540cd2e0 0x7f70541f0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.011 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 --2- 172.21.15.106:0/417829565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f705406e160 0x7f70541f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.008+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 >> 172.21.15.106:0/417829565 conn(0x7f70540b0c50 msgr2=0x7f70540aeb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:13.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.009+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 shutdown_connections 2023-11-24T08:13:13.012 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.009+0000 7f705c290640 1 -- 172.21.15.106:0/417829565 wait complete. 2023-11-24T08:13:13.030 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:13:13.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.251+0000 7fdfdbd2c640 1 Processor -- start 2023-11-24T08:13:13.254 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.251+0000 7fdfdbd2c640 1 -- start start 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfdbd2c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdfd4072160 0x7fdfd415e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfdbd2c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfdbd2c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfd415ec90 con 0x7fdfd406dc40 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfdbd2c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfd406e0f0 con 0x7fdfd4072160 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfdbd2c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40cd530 con 0x7fdfd40c49f0 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfd9ac8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fdfd4072160 0x7fdfd415e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfda2c9640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdfd406dc40 0x7fdfd40cd750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:48614/0 (socket says 172.21.15.106:48614) 2023-11-24T08:13:13.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.253+0000 7fdfda2c9640 1 -- 172.21.15.106:0/1625078002 learned_addr learned my addr 172.21.15.106:0/1625078002 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.254+0000 7fdfd92c7640 1 --2- 172.21.15.106:0/1625078002 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.254+0000 7fdfd9ac8640 1 -- 172.21.15.106:0/1625078002 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 msgr2=0x7fdfd406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.254+0000 7fdfd9ac8640 1 --2- 172.21.15.106:0/1625078002 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.254+0000 7fdfd9ac8640 1 -- 172.21.15.106:0/1625078002 >> v1:172.21.15.106:6789/0 conn(0x7fdfd406dc40 legacy=0x7fdfd40cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.257 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.254+0000 7fdfd9ac8640 1 -- 172.21.15.106:0/1625078002 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd40c4ed0 con 0x7fdfd4072160 2023-11-24T08:13:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.255+0000 7fdfd9ac8640 1 --2- 172.21.15.106:0/1625078002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4072160 0x7fdfd415e750 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00097a0 tx=0x7fdfd002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ec6216d2862ce80 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.255+0000 7fdfd8ac6640 1 -- 172.21.15.106:0/1625078002 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd002d040 con 0x7fdfd4072160 2023-11-24T08:13:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.255+0000 7fdfd8ac6640 1 -- 172.21.15.106:0/1625078002 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd0031810 con 0x7fdfd4072160 2023-11-24T08:13:13.258 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.255+0000 7fdfd8ac6640 1 -- 172.21.15.106:0/1625078002 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd0031ba0 con 0x7fdfd4072160 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1625078002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4072160 msgr2=0x7fdfd415e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1625078002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4072160 0x7fdfd415e750 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fdfd00097a0 tx=0x7fdfd002b8b0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1625078002 shutdown_connections 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1625078002 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1625078002 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4072160 0x7fdfd415e750 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1625078002 >> 172.21.15.106:0/1625078002 conn(0x7fdfd40b0c50 msgr2=0x7fdfd40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1625078002 shutdown_connections 2023-11-24T08:13:13.259 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.256+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1625078002 wait complete. 2023-11-24T08:13:13.260 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.258+0000 7fdfdbd2c640 1 Processor -- start 2023-11-24T08:13:13.261 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.259+0000 7fdfdbd2c640 1 -- start start 2023-11-24T08:13:13.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfdbd2c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfd406dc40 0x7fdfd40ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfdbd2c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfdbd2c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 0x7fdfd4145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.263 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfdbd2c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd4145770 con 0x7fdfd4145a10 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfdbd2c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40aeab0 con 0x7fdfd406dc40 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfdbd2c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfd40aed90 con 0x7fdfd40c49f0 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfda2c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 0x7fdfd4145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfda2c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 0x7fdfd4145ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57224/0 (socket says 172.21.15.106:57224) 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfda2c9640 1 -- 172.21.15.106:0/1010179885 learned_addr learned my addr 172.21.15.106:0/1010179885 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfd9ac8640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfd406dc40 0x7fdfd40ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.260+0000 7fdfd92c7640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfda2c9640 1 -- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 msgr2=0x7fdfd415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfda2c9640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfda2c9640 1 -- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfd406dc40 msgr2=0x7fdfd40ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfda2c9640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfd406dc40 0x7fdfd40ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfda2c9640 1 -- 172.21.15.106:0/1010179885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd0009400 con 0x7fdfd4145a10 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfd92c7640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:13.264 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfda2c9640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 0x7fdfd4145ec0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00cbc0 tx=0x7fdfcc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:13.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00ad40 con 0x7fdfd4145a10 2023-11-24T08:13:13.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfcc012400 con 0x7fdfd4145a10 2023-11-24T08:13:13.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfcc00aea0 con 0x7fdfd4145a10 2023-11-24T08:13:13.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfd40af070 con 0x7fdfd4145a10 2023-11-24T08:13:13.265 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.261+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfd40af630 con 0x7fdfd4145a10 2023-11-24T08:13:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.263+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfd40af820 con 0x7fdfd4145a10 2023-11-24T08:13:13.266 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.264+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fdfcc007070 con 0x7fdfd4145a10 2023-11-24T08:13:13.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.265+0000 7fdfca7fc640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdfb806b470 0x7fdfb806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.265+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdfcc089f50 con 0x7fdfd4145a10 2023-11-24T08:13:13.267 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.265+0000 7fdfd9ac8640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdfb806b470 0x7fdfb806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.268 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.266+0000 7fdfd9ac8640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdfb806b470 0x7fdfb806d930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0036000 tx=0x7fdfd003c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:13.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.271+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdfcc0546a0 con 0x7fdfd4145a10 2023-11-24T08:13:13.468 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.466+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfd415ec90 con 0x7fdfd4145a10 2023-11-24T08:13:13.469 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.466+0000 7fdfca7fc640 1 -- 172.21.15.106:0/1010179885 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfd415ec90 con 0x7fdfd4145a10 2023-11-24T08:13:13.469 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:13:13.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdfb806b470 msgr2=0x7fdfb806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdfb806b470 0x7fdfb806d930 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fdfd0036000 tx=0x7fdfd003c040 comp rx=0 tx=0).stop 2023-11-24T08:13:13.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 msgr2=0x7fdfd4145ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.471 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 0x7fdfd4145ec0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc00cbc0 tx=0x7fdfcc00a430 comp rx=0 tx=0).stop 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 shutdown_connections 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdfd4145a10 0x7fdfd4145ec0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fdfb806b470 0x7fdfb806d930 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdfd40c49f0 0x7fdfd415a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 --2- 172.21.15.106:0/1010179885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdfd406dc40 0x7fdfd40ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 >> 172.21.15.106:0/1010179885 conn(0x7fdfd40b0c50 msgr2=0x7fdfd4070e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 shutdown_connections 2023-11-24T08:13:13.472 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.469+0000 7fdfdbd2c640 1 -- 172.21.15.106:0/1010179885 wait complete. 2023-11-24T08:13:13.490 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:13:13.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.695+0000 7fcfdbd78640 1 Processor -- start 2023-11-24T08:13:13.698 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.695+0000 7fcfdbd78640 1 -- start start 2023-11-24T08:13:13.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.697+0000 7fcfdbd78640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fcfd4146eb0 0x7fcfd4161150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.697+0000 7fcfdbd78640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcfd406e160 0x7fcfd40ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.697+0000 7fcfdbd78640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 0x7fcfd406c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.700 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.697+0000 7fcfdbd78640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd40cd580 con 0x7fcfd406e160 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.697+0000 7fcfdbd78640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd4161a10 con 0x7fcfd4146eb0 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.697+0000 7fcfdbd78640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd4161cf0 con 0x7fcfd40cd110 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcfd406e160 0x7fcfd40ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcfd406e160 0x7fcfd40ccbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57228/0 (socket says 172.21.15.106:57228) 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 -- 172.21.15.106:0/3336252047 learned_addr learned my addr 172.21.15.106:0/3336252047 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 -- 172.21.15.106:0/3336252047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 msgr2=0x7fcfd406c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 --2- 172.21.15.106:0/3336252047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 0x7fcfd406c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 -- 172.21.15.106:0/3336252047 >> v2:172.21.15.156:3300/0 conn(0x7fcfd4146eb0 msgr2=0x7fcfd4161150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 --2- 172.21.15.106:0/3336252047 >> v2:172.21.15.156:3300/0 conn(0x7fcfd4146eb0 0x7fcfd4161150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.701 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.698+0000 7fcfd9313640 1 -- 172.21.15.106:0/3336252047 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd4161fd0 con 0x7fcfd406e160 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.699+0000 7fcfd9313640 1 --2- 172.21.15.106:0/3336252047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd406e160 0x7fcfd40ccbd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcfd406ea50 tx=0x7fcfc402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd5c0f2cd8731fb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.699+0000 7fcfd8b12640 1 -- 172.21.15.106:0/3336252047 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4031870 con 0x7fcfd406e160 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.699+0000 7fcfd8b12640 1 -- 172.21.15.106:0/3336252047 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc40319d0 con 0x7fcfd406e160 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.699+0000 7fcfd8b12640 1 -- 172.21.15.106:0/3336252047 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4031d60 con 0x7fcfd406e160 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3336252047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd406e160 msgr2=0x7fcfd40ccbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3336252047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd406e160 0x7fcfd40ccbd0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fcfd406ea50 tx=0x7fcfc402b910 comp rx=0 tx=0).stop 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3336252047 shutdown_connections 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3336252047 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 0x7fcfd406c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3336252047 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd406e160 0x7fcfd40ccbd0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3336252047 >> v2:172.21.15.156:3300/0 conn(0x7fcfd4146eb0 0x7fcfd4161150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.703 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3336252047 >> 172.21.15.106:0/3336252047 conn(0x7fcfd40b0c50 msgr2=0x7fcfd40b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:13.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3336252047 shutdown_connections 2023-11-24T08:13:13.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.700+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3336252047 wait complete. 2023-11-24T08:13:13.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.702+0000 7fcfdbd78640 1 Processor -- start 2023-11-24T08:13:13.704 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.702+0000 7fcfdbd78640 1 -- start start 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfdbd78640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd406e160 0x7fcfd409af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfdbd78640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 0x7fcfd41e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfdbd78640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 0x7fcfd40ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfdbd78640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd409b840 con 0x7fcfd4146eb0 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfdbd78640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd409bb20 con 0x7fcfd406e160 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfdbd78640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd409be00 con 0x7fcfd40cd110 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfda315640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 0x7fcfd40ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.707 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfda315640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 0x7fcfd40ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:57234/0 (socket says 172.21.15.106:57234) 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.703+0000 7fcfda315640 1 -- 172.21.15.106:0/3978066485 learned_addr learned my addr 172.21.15.106:0/3978066485 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfda315640 1 -- 172.21.15.106:0/3978066485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 msgr2=0x7fcfd41e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfda315640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 0x7fcfd41e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfda315640 1 -- 172.21.15.106:0/3978066485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd406e160 msgr2=0x7fcfd409af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfda315640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd406e160 0x7fcfd409af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfda315640 1 -- 172.21.15.106:0/3978066485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc40094d0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfda315640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 0x7fcfd40ad400 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b970 tx=0x7fcfcc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00cb40 con 0x7fcfd4146eb0 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00ccc0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc014680 con 0x7fcfd4146eb0 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.704+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd409c0e0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.708 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.705+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd41e9170 con 0x7fcfd4146eb0 2023-11-24T08:13:13.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.706+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd404e8b0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.707+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcfcc01d3b0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.707+0000 7fcfca7fc640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcfb806b470 0x7fcfb806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:13.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.708+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcfcc08a160 con 0x7fcfd4146eb0 2023-11-24T08:13:13.710 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.708+0000 7fcfd9b14640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcfb806b470 0x7fcfb806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:13.711 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.708+0000 7fcfd9b14640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcfb806b470 0x7fcfb806d930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009710 tx=0x7fcfd0006c90 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:13.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.712+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcfcc0bce90 con 0x7fcfd4146eb0 2023-11-24T08:13:13.904 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.901+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcfd409c2d0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.905 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.902+0000 7fcfca7fc640 1 -- 172.21.15.106:0/3978066485 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcfcc0190a0 con 0x7fcfd4146eb0 2023-11-24T08:13:13.905 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:13:13.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.904+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcfb806b470 msgr2=0x7fcfb806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.904+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcfb806b470 0x7fcfb806d930 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fcfd0009710 tx=0x7fcfd0006c90 comp rx=0 tx=0).stop 2023-11-24T08:13:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 msgr2=0x7fcfd40ad400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 0x7fcfd40ad400 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00b970 tx=0x7fcfcc00be30 comp rx=0 tx=0).stop 2023-11-24T08:13:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 shutdown_connections 2023-11-24T08:13:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcfd4146eb0 0x7fcfd40ad400 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcfb806b470 0x7fcfb806d930 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcfd40cd110 0x7fcfd41e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 --2- 172.21.15.106:0/3978066485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfd406e160 0x7fcfd409af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 >> 172.21.15.106:0/3978066485 conn(0x7fcfd40b0c50 msgr2=0x7fcfd4145780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 shutdown_connections 2023-11-24T08:13:13.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:13.905+0000 7fcfdbd78640 1 -- 172.21.15.106:0/3978066485 wait complete. 2023-11-24T08:13:13.925 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-24T08:13:14.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.170+0000 7fccab59e640 1 Processor -- start 2023-11-24T08:13:14.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.170+0000 7fccab59e640 1 -- start start 2023-11-24T08:13:14.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccab59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 0x7fccac070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccab59e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fccac0b15e0 0x7fccac15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccab59e640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac0b1a20 con 0x7fccac06d640 2023-11-24T08:13:14.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccab59e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac0cd920 con 0x7fccac0b15e0 2023-11-24T08:13:14.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccab59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15e2e0 con 0x7fccac15a1c0 2023-11-24T08:13:14.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccaad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fccac06d640 0x7fccac0ce050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50656/0 (socket says 172.21.15.106:50656) 2023-11-24T08:13:14.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccaad9d640 1 -- 172.21.15.106:0/3146026061 learned_addr learned my addr 172.21.15.106:0/3146026061 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fccaa59c640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 0x7fccac070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fcca3fff640 1 --2- 172.21.15.106:0/3146026061 >> v2:172.21.15.156:3300/0 conn(0x7fccac0b15e0 0x7fccac15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.172+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996077278 0 0) 0x7fcca4001490 con 0x7fccac06d640 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc98003520 con 0x7fccac06d640 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 169023408 0 0) 0x7fcc98003520 con 0x7fccac06d640 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 msgr2=0x7fccac070ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 0x7fccac070ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 >> v2:172.21.15.156:3300/0 conn(0x7fccac0b15e0 msgr2=0x7fccac15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:14.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 --2- 172.21.15.106:0/3146026061 >> v2:172.21.15.156:3300/0 conn(0x7fccac0b15e0 0x7fccac15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac15e580 con 0x7fccac06d640 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fcca4001b80 con 0x7fccac06d640 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 >> v1:172.21.15.106:6789/0 conn(0x7fccac06d640 legacy=0x7fccac0ce050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc98003970 0x7fcc98003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc980047c0 0x7fcc98004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 0x7fcc98005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0cd920 con 0x7fcc98003970 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15e2e0 con 0x7fcc980047c0 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9800b980 con 0x7fccac15a1c0 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.173+0000 7fccaa59c640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc98003970 0x7fcc98003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.174+0000 7fccaa59c640 1 -- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 msgr2=0x7fcc98005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:14.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.174+0000 7fccaa59c640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac15a1c0 0x7fcc98005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.174+0000 7fccaa59c640 1 -- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc980047c0 msgr2=0x7fcc98004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:14.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.174+0000 7fcca3fff640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc980047c0 0x7fcc98004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.174+0000 7fccaa59c640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc980047c0 0x7fcc98004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.174+0000 7fccaa59c640 1 -- 172.21.15.106:0/3146026061 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98003520 con 0x7fcc98003970 2023-11-24T08:13:14.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.175+0000 7fccaa59c640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc98003970 0x7fcc98003d90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009ac0 tx=0x7fcc980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e41d4fbaf121a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:14.180 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.175+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c02c710 con 0x7fcc98003970 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.175+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c02c870 con 0x7fcc98003970 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.175+0000 7fcca9d9b640 1 -- 172.21.15.106:0/3146026061 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c02cc00 con 0x7fcc98003970 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccab59e640 1 -- 172.21.15.106:0/3146026061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc98003970 msgr2=0x7fcc98003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccab59e640 1 --2- 172.21.15.106:0/3146026061 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcc98003970 0x7fcc98003d90 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009ac0 tx=0x7fcc980057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccaad9d640 1 -- 172.21.15.106:0/3146026061 reap_dead start 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccab59e640 1 -- 172.21.15.106:0/3146026061 shutdown_connections 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccab59e640 1 -- 172.21.15.106:0/3146026061 >> 172.21.15.106:0/3146026061 conn(0x7fccac0cbb40 msgr2=0x7fccac0cbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccab59e640 1 -- 172.21.15.106:0/3146026061 shutdown_connections 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.176+0000 7fccab59e640 1 -- 172.21.15.106:0/3146026061 wait complete. 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.178+0000 7fccab59e640 1 Processor -- start 2023-11-24T08:13:14.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.178+0000 7fccab59e640 1 -- start start 2023-11-24T08:13:14.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fccab59e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc98003970 0x7fccac14bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fccab59e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 0x7fccac0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fccab59e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac0c2a20 0x7fccac0c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fccab59e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0c3410 con 0x7fccac150a70 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fccab59e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0af3a0 con 0x7fcc98003970 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fccab59e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0af680 con 0x7fccac0c2a20 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 0x7fccac0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 0x7fccac0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45518/0 (socket says 172.21.15.106:45518) 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 -- 172.21.15.106:0/357854537 learned_addr learned my addr 172.21.15.106:0/357854537 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 -- 172.21.15.106:0/357854537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac0c2a20 msgr2=0x7fccac0c2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac0c2a20 0x7fccac0c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 -- 172.21.15.106:0/357854537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc98003970 msgr2=0x7fccac14bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc98003970 0x7fccac14bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 -- 172.21.15.106:0/357854537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac0af960 con 0x7fccac150a70 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.179+0000 7fcca3fff640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 0x7fccac0aea30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcc940108e0 tx=0x7fcc94010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:14.184 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.180+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9400eb60 con 0x7fccac150a70 2023-11-24T08:13:14.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.180+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9400ecc0 con 0x7fccac150a70 2023-11-24T08:13:14.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.180+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac0afc40 con 0x7fccac150a70 2023-11-24T08:13:14.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.180+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9401c400 con 0x7fccac150a70 2023-11-24T08:13:14.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.180+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac14c130 con 0x7fccac150a70 2023-11-24T08:13:14.185 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.181+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac04e840 con 0x7fccac150a70 2023-11-24T08:13:14.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.182+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcc9401c560 con 0x7fccac150a70 2023-11-24T08:13:14.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.183+0000 7fcca2ffd640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc8c06b4c0 0x7fcc8c06d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.183+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcc9408a670 con 0x7fccac150a70 2023-11-24T08:13:14.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.183+0000 7fccaa59c640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc8c06b4c0 0x7fcc8c06d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.184+0000 7fccaa59c640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc8c06b4c0 0x7fcc8c06d980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c036000 tx=0x7fcc9c02e000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:14.189 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.187+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fcc94054f50 con 0x7fccac150a70 2023-11-24T08:13:14.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.388+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fccac0b0440 con 0x7fccac150a70 2023-11-24T08:13:14.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.389+0000 7fcca2ffd640 1 -- 172.21.15.106:0/357854537 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v89) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcc94058570 con 0x7fccac150a70 2023-11-24T08:13:14.391 INFO:teuthology.orchestra.run.smithi106.stderr:listed 0 entries 2023-11-24T08:13:14.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.391+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc8c06b4c0 msgr2=0x7fcc8c06d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.391+0000 7fccab59e640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc8c06b4c0 0x7fcc8c06d980 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c036000 tx=0x7fcc9c02e000 comp rx=0 tx=0).stop 2023-11-24T08:13:14.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.391+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 msgr2=0x7fccac0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.391+0000 7fccab59e640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 0x7fccac0aea30 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fcc940108e0 tx=0x7fcc94010da0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 shutdown_connections 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcc8c06b4c0 0x7fcc8c06d980 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fccac0c2a20 0x7fccac0c2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fccac150a70 0x7fccac0aea30 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 --2- 172.21.15.106:0/357854537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc98003970 0x7fccac14bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 >> 172.21.15.106:0/357854537 conn(0x7fccac0cbb40 msgr2=0x7fccac153dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 shutdown_connections 2023-11-24T08:13:14.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.392+0000 7fccab59e640 1 -- 172.21.15.106:0/357854537 wait complete. 2023-11-24T08:13:14.413 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-24T08:13:14.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:14.413 DEBUG:teuthology.orchestra.run.smithi156:> sudo umount /home/ubuntu/cephtest/mnt.1 2023-11-24T08:13:14.521 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:13:14.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:14.521 DEBUG:teuthology.orchestra.run.smithi156:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-24T08:13:14.571 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:13:14.571 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:14.630 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:13:14.630 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:13:14.630 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:14.689 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:13:14.689 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:14.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.906+0000 7f8b68390640 1 Processor -- start 2023-11-24T08:13:14.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.906+0000 7f8b68390640 1 -- start start 2023-11-24T08:13:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.907+0000 7f8b68390640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cf450 0x7f8b600cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.907+0000 7f8b68390640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b600cdfb0 0x7f8b600ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.907+0000 7f8b68390640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b600cece0 con 0x7f8b6015c7f0 2023-11-24T08:13:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.907+0000 7f8b68390640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b60156010 con 0x7f8b600cdfb0 2023-11-24T08:13:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.907+0000 7f8b68390640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6006ce30 con 0x7f8b600cf450 2023-11-24T08:13:14.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b600cdfb0 0x7f8b600ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f8b600cdfb0 0x7f8b600ce440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45524/0 (socket says 172.21.15.106:45524) 2023-11-24T08:13:14.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 -- 172.21.15.106:0/3927427479 learned_addr learned my addr 172.21.15.106:0/3927427479 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:14.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 -- 172.21.15.106:0/3927427479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cf450 msgr2=0x7f8b600cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:14.911 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 --2- 172.21.15.106:0/3927427479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cf450 0x7f8b600cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 -- 172.21.15.106:0/3927427479 >> v1:172.21.15.106:6789/0 conn(0x7f8b6015c7f0 legacy=0x7f8b6006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6692d640 1 -- 172.21.15.106:0/3927427479 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6006d040 con 0x7f8b600cdfb0 2023-11-24T08:13:14.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.908+0000 7f8b6512a640 1 -- 172.21.15.106:0/3927427479 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2909598305 0 0) 0x7f8b600cece0 con 0x7f8b6015c7f0 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.909+0000 7f8b6692d640 1 --2- 172.21.15.106:0/3927427479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cdfb0 0x7f8b600ce440 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009e50 tx=0x7f8b5802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1ec16b3dddf803 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b6512a640 1 -- 172.21.15.106:0/3927427479 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5802cb00 con 0x7f8b600cdfb0 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b6512a640 1 -- 172.21.15.106:0/3927427479 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5802cc60 con 0x7f8b600cdfb0 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b6512a640 1 -- 172.21.15.106:0/3927427479 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580343f0 con 0x7f8b600cdfb0 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b68390640 1 -- 172.21.15.106:0/3927427479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cdfb0 msgr2=0x7f8b600ce440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b68390640 1 --2- 172.21.15.106:0/3927427479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cdfb0 0x7f8b600ce440 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009e50 tx=0x7f8b5802bf60 comp rx=0 tx=0).stop 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b68390640 1 -- 172.21.15.106:0/3927427479 shutdown_connections 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b68390640 1 --2- 172.21.15.106:0/3927427479 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cdfb0 0x7f8b600ce440 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b68390640 1 --2- 172.21.15.106:0/3927427479 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cf450 0x7f8b600cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.910+0000 7f8b68390640 1 -- 172.21.15.106:0/3927427479 >> 172.21.15.106:0/3927427479 conn(0x7f8b6006a920 msgr2=0x7f8b6006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:14.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.911+0000 7f8b68390640 1 -- 172.21.15.106:0/3927427479 shutdown_connections 2023-11-24T08:13:14.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.911+0000 7f8b68390640 1 -- 172.21.15.106:0/3927427479 wait complete. 2023-11-24T08:13:14.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.912+0000 7f8b68390640 1 Processor -- start 2023-11-24T08:13:14.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.912+0000 7f8b68390640 1 -- start start 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.913+0000 7f8b68390640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cdfb0 0x7f8b600cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.913+0000 7f8b68390640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 0x7f8b601493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.913+0000 7f8b68390640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b6015c7f0 0x7f8b600b3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.913+0000 7f8b68390640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6015a570 con 0x7f8b600cf450 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.913+0000 7f8b68390640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c87d0 con 0x7f8b6015c7f0 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.913+0000 7f8b68390640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600cb1f0 con 0x7f8b600cdfb0 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 0x7f8b601493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 0x7f8b601493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45530/0 (socket says 172.21.15.106:45530) 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 -- 172.21.15.106:0/763996407 learned_addr learned my addr 172.21.15.106:0/763996407 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 -- 172.21.15.106:0/763996407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cdfb0 msgr2=0x7f8b600cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cdfb0 0x7f8b600cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 -- 172.21.15.106:0/763996407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b6015c7f0 msgr2=0x7f8b600b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b6015c7f0 0x7f8b600b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 -- 172.21.15.106:0/763996407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b58009b30 con 0x7f8b600cf450 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b6592b640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 0x7f8b601493c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400b4a0 tx=0x7f8b5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.914+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54010610 con 0x7f8b600cf450 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.915+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54010770 con 0x7f8b600cf450 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.915+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54010b20 con 0x7f8b600cf450 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.915+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b60159bd0 con 0x7f8b600cf450 2023-11-24T08:13:14.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.915+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6015a190 con 0x7f8b600cf450 2023-11-24T08:13:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.916+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b60067470 con 0x7f8b600cf450 2023-11-24T08:13:14.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.917+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f8b54010c80 con 0x7f8b600cf450 2023-11-24T08:13:14.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.918+0000 7f8b52ffd640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b4406b4a0 0x7f8b4406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:14.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.918+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b54089d80 con 0x7f8b600cf450 2023-11-24T08:13:14.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.918+0000 7f8b6612c640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b4406b4a0 0x7f8b4406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:14.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.918+0000 7f8b6612c640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b4406b4a0 0x7f8b4406d960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c001560 tx=0x7f8b5c007930 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:14.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:14.922+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8b54054420 con 0x7f8b600cf450 2023-11-24T08:13:15.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.150+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8b600cd730 con 0x7f8b600cf450 2023-11-24T08:13:15.153 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.151+0000 7f8b52ffd640 1 -- 172.21.15.106:0/763996407 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f8b5401e020 con 0x7f8b600cf450 2023-11-24T08:13:15.154 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:15.154 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:56.174099+0000","modified":"2023-11-24T08:12:59.244264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1896010072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1896010072},{"type":"v1","addr":"172.21.15.106:6837","nonce":1896010072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:13:15.154 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 54 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b4406b4a0 msgr2=0x7f8b4406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b4406b4a0 0x7f8b4406d960 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c001560 tx=0x7f8b5c007930 comp rx=0 tx=0).stop 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 msgr2=0x7f8b601493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 0x7f8b601493c0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400b4a0 tx=0x7f8b5400b960 comp rx=0 tx=0).stop 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 shutdown_connections 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b6015c7f0 0x7f8b600b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f8b600cf450 0x7f8b601493c0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f8b4406b4a0 0x7f8b4406d960 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 --2- 172.21.15.106:0/763996407 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f8b600cdfb0 0x7f8b600cacb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.157 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 >> 172.21.15.106:0/763996407 conn(0x7f8b6006a920 msgr2=0x7f8b600ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.154+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 shutdown_connections 2023-11-24T08:13:15.158 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.155+0000 7f8b68390640 1 -- 172.21.15.106:0/763996407 wait complete. 2023-11-24T08:13:15.177 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:15.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.385+0000 7f2fc85c9640 1 Processor -- start 2023-11-24T08:13:15.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.385+0000 7f2fc85c9640 1 -- start start 2023-11-24T08:13:15.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc85c9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fc015c7f0 0x7f2fc006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc85c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00ce6c0 0x7f2fc00cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc85c9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fc00cece0 con 0x7f2fc00cf450 2023-11-24T08:13:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc85c9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fc00cfe40 con 0x7f2fc015c7f0 2023-11-24T08:13:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc85c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc0156010 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc6365640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2fc015c7f0 0x7f2fc006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc5b64640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f2fc00cf450 0x7f2fc00cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50676/0 (socket says 172.21.15.106:50676) 2023-11-24T08:13:15.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc5b64640 1 -- 172.21.15.106:0/208155743 learned_addr learned my addr 172.21.15.106:0/208155743 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:15.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.387+0000 7f2fc6b66640 1 --2- 172.21.15.106:0/208155743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00ce6c0 0x7f2fc00cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.388+0000 7f2fc5363640 1 -- 172.21.15.106:0/208155743 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595280062 0 0) 0x7f2fc00cece0 con 0x7f2fc00cf450 2023-11-24T08:13:15.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.388+0000 7f2fc6365640 1 -- 172.21.15.106:0/208155743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00ce6c0 msgr2=0x7f2fc00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:15.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.388+0000 7f2fc6365640 1 --2- 172.21.15.106:0/208155743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00ce6c0 0x7f2fc00cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.388+0000 7f2fc6365640 1 -- 172.21.15.106:0/208155743 >> v1:172.21.15.106:6789/0 conn(0x7f2fc00cf450 legacy=0x7f2fc00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.388+0000 7f2fc6365640 1 -- 172.21.15.106:0/208155743 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fc006d040 con 0x7f2fc015c7f0 2023-11-24T08:13:15.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.389+0000 7f2fc6365640 1 --2- 172.21.15.106:0/208155743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc015c7f0 0x7f2fc006db20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33840ed777c548ef server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:15.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.389+0000 7f2fc5363640 1 -- 172.21.15.106:0/208155743 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc02d040 con 0x7f2fc015c7f0 2023-11-24T08:13:15.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.389+0000 7f2fc5363640 1 -- 172.21.15.106:0/208155743 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc031810 con 0x7f2fc015c7f0 2023-11-24T08:13:15.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.389+0000 7f2fc5363640 1 -- 172.21.15.106:0/208155743 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc031ba0 con 0x7f2fc015c7f0 2023-11-24T08:13:15.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 -- 172.21.15.106:0/208155743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc015c7f0 msgr2=0x7f2fc006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/208155743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc015c7f0 0x7f2fc006db20 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc0097a0 tx=0x7f2fbc02b8b0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 -- 172.21.15.106:0/208155743 shutdown_connections 2023-11-24T08:13:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/208155743 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00ce6c0 0x7f2fc00cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/208155743 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc015c7f0 0x7f2fc006db20 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 -- 172.21.15.106:0/208155743 >> 172.21.15.106:0/208155743 conn(0x7f2fc006a920 msgr2=0x7f2fc006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 -- 172.21.15.106:0/208155743 shutdown_connections 2023-11-24T08:13:15.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.390+0000 7f2fc85c9640 1 -- 172.21.15.106:0/208155743 wait complete. 2023-11-24T08:13:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.392+0000 7f2fc85c9640 1 Processor -- start 2023-11-24T08:13:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.392+0000 7f2fc85c9640 1 -- start start 2023-11-24T08:13:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc85c9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 0x7f2fc01f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc85c9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00cf450 0x7f2fc00cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc85c9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fc015c7f0 0x7f2fc015a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc85c9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc00c8470 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc85c9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01572f0 con 0x7f2fc015c7f0 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc85c9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fc01f76d0 con 0x7f2fc00cf450 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.393+0000 7f2fc6365640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 0x7f2fc01f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 0x7f2fc01f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45560/0 (socket says 172.21.15.106:45560) 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 -- 172.21.15.106:0/1551835262 learned_addr learned my addr 172.21.15.106:0/1551835262 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 -- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00cf450 msgr2=0x7f2fc00cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc5b64640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00cf450 0x7f2fc00cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6b66640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fc015c7f0 0x7f2fc015a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00cf450 0x7f2fc00cada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 -- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fc015c7f0 msgr2=0x7f2fc015a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fc015c7f0 0x7f2fc015a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 -- 172.21.15.106:0/1551835262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fbc009400 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.394+0000 7f2fc6365640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 0x7f2fc01f7190 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc000c00 tx=0x7f2fbc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.395+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc0443f0 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.395+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fbc044550 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.395+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fbc044900 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.395+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fc0154ac0 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.395+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc0155080 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.396+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fc0067470 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.397+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2fbc04d450 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.398+0000 7f2fb6ffd640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fa406b470 0x7f2fa406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.398+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2fbc0ac5a0 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.404 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.398+0000 7f2fc5b64640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fa406b470 0x7f2fa406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.399+0000 7f2fc5b64640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fa406b470 0x7f2fa406d930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0007fd0 tx=0x7f2fb0009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:15.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.402+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2fbc076c50 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.595+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2fc00cb7e0 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.596+0000 7f2fb6ffd640 1 -- 172.21.15.106:0/1551835262 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16608 (secure 0 0 0) 0x7f2fbc035090 con 0x7f2fc00ce6c0 2023-11-24T08:13:15.599 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:15.600 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:57.727441+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:15.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fa406b470 msgr2=0x7f2fa406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fa406b470 0x7f2fa406d930 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0007fd0 tx=0x7f2fb0009040 comp rx=0 tx=0).stop 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 msgr2=0x7f2fc01f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 0x7f2fc01f7190 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f2fbc000c00 tx=0x7f2fbc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 shutdown_connections 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fc015c7f0 0x7f2fc015a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2fa406b470 0x7f2fa406d930 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2fc00cf450 0x7f2fc00cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.603 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 --2- 172.21.15.106:0/1551835262 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2fc00ce6c0 0x7f2fc01f7190 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 >> 172.21.15.106:0/1551835262 conn(0x7f2fc006a920 msgr2=0x7f2fc00b0410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:15.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 shutdown_connections 2023-11-24T08:13:15.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.599+0000 7f2fc85c9640 1 -- 172.21.15.106:0/1551835262 wait complete. 2023-11-24T08:13:15.621 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-11-24T08:08:41.348031+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-24T08:12:53.987858+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-11-24T08:12:55.193239+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-24T08:13:15.622 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-24T08:13:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.835+0000 7f07859a9640 1 Processor -- start 2023-11-24T08:13:15.837 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.835+0000 7f07859a9640 1 -- start start 2023-11-24T08:13:15.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f07859a9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f078006ac10 0x7f07800b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f07859a9640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0780072160 con 0x7f078015b530 2023-11-24T08:13:15.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f07859a9640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f078015b910 con 0x7f07800afa50 2023-11-24T08:13:15.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f07859a9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07800b3c80 con 0x7f078006ac10 2023-11-24T08:13:15.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f077effd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f078015b530 0x7f078006a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50692/0 (socket says 172.21.15.106:50692) 2023-11-24T08:13:15.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f077effd640 1 -- 172.21.15.106:0/3561357059 learned_addr learned my addr 172.21.15.106:0/3561357059 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:15.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.837+0000 7f07767fc640 1 --2- 172.21.15.106:0/3561357059 >> v2:172.21.15.156:3300/0 conn(0x7f078006ac10 0x7f07800b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 122977366 0 0) 0x7f0780072160 con 0x7f078015b530 2023-11-24T08:13:15.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f076c003520 con 0x7f078015b530 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1654844481 0 0) 0x7f076c003520 con 0x7f078015b530 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 >> v2:172.21.15.156:3300/0 conn(0x7f078006ac10 msgr2=0x7f07800b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 --2- 172.21.15.106:0/3561357059 >> v2:172.21.15.156:3300/0 conn(0x7f078006ac10 0x7f07800b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 >> v1:172.21.15.156:6789/0 conn(0x7f07800afa50 legacy=0x7f07800b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07800b27b0 con 0x7f078015b530 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f07700019f0 con 0x7f078015b530 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 >> v1:172.21.15.106:6789/0 conn(0x7f078015b530 legacy=0x7f078006a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c003970 0x7f076c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f076c005180 0x7f076c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07800b3c80 con 0x7f076c003970 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c00c020 con 0x7f076c0047c0 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.838+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076c00c1e0 con 0x7f076c005180 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077effd640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c003970 0x7f076c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f07767fc640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077effd640 1 -- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f076c005180 msgr2=0x7f076c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077f7fe640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f076c005180 0x7f076c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077effd640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f076c005180 0x7f076c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077effd640 1 -- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c0047c0 msgr2=0x7f076c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077effd640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c0047c0 0x7f076c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077effd640 1 -- 172.21.15.106:0/3561357059 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076c003520 con 0x7f076c003970 2023-11-24T08:13:15.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.839+0000 7f077f7fe640 1 -- 172.21.15.106:0/3561357059 reap_dead start 2023-11-24T08:13:15.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.840+0000 7f077effd640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c003970 0x7f076c003d90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f077000a830 tx=0x7f076c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31734499c76d588a server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700072d0 con 0x7f076c003970 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077002f070 con 0x7f076c003970 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f077e7fc640 1 -- 172.21.15.106:0/3561357059 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07700076f0 con 0x7f076c003970 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 -- 172.21.15.106:0/3561357059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c003970 msgr2=0x7f076c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c003970 0x7f076c003d90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f077000a830 tx=0x7f076c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 -- 172.21.15.106:0/3561357059 shutdown_connections 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 --2- 172.21.15.106:0/3561357059 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f076c003970 0x7f076c003d90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 -- 172.21.15.106:0/3561357059 >> 172.21.15.106:0/3561357059 conn(0x7f078006c090 msgr2=0x7f07800b1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 -- 172.21.15.106:0/3561357059 shutdown_connections 2023-11-24T08:13:15.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.841+0000 7f07859a9640 1 -- 172.21.15.106:0/3561357059 wait complete. 2023-11-24T08:13:15.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.843+0000 7f07859a9640 1 Processor -- start 2023-11-24T08:13:15.846 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.843+0000 7f07859a9640 1 -- start start 2023-11-24T08:13:15.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07859a9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c003970 0x7f078015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07859a9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 0x7f07800ca880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07859a9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07801f3920 0x7f07800cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07859a9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07801f4b10 con 0x7f07801f34a0 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07859a9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07801f3e90 con 0x7f076c003970 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07859a9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07801f4170 con 0x7f07801f3920 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07767fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 0x7f07800ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07767fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 0x7f07800ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45570/0 (socket says 172.21.15.106:45570) 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.844+0000 7f07767fc640 1 -- 172.21.15.106:0/3501877527 learned_addr learned my addr 172.21.15.106:0/3501877527 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07767fc640 1 -- 172.21.15.106:0/3501877527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07801f3920 msgr2=0x7f07800cadc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07767fc640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07801f3920 0x7f07800cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07767fc640 1 -- 172.21.15.106:0/3501877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c003970 msgr2=0x7f078015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07767fc640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c003970 0x7f078015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:15.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07767fc640 1 -- 172.21.15.106:0/3501877527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07801f4450 con 0x7f07801f34a0 2023-11-24T08:13:15.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07767fc640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 0x7f07800ca880 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f07680108e0 tx=0x7f0768010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:15.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076800eb60 con 0x7f07801f34a0 2023-11-24T08:13:15.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076800ecc0 con 0x7f07801f34a0 2023-11-24T08:13:15.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07801f4730 con 0x7f07801f34a0 2023-11-24T08:13:15.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.845+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076801c400 con 0x7f07801f34a0 2023-11-24T08:13:15.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.846+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f078015e380 con 0x7f07801f34a0 2023-11-24T08:13:15.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.847+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f078004e8b0 con 0x7f07801f34a0 2023-11-24T08:13:15.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.848+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f076801c560 con 0x7f07801f34a0 2023-11-24T08:13:15.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.849+0000 7f077cff9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f076006b4a0 0x7f076006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:15.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.849+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f076808a6a0 con 0x7f07801f34a0 2023-11-24T08:13:15.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.849+0000 7f077effd640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f076006b4a0 0x7f076006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:15.852 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.850+0000 7f077effd640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f076006b4a0 0x7f076006d960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0770034000 tx=0x7f077002f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:15.857 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:15.853+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0768054f10 con 0x7f07801f34a0 2023-11-24T08:13:16.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.042+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07801f4ed0 con 0x7f07801f34a0 2023-11-24T08:13:16.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.043+0000 7f077cff9640 1 -- 172.21.15.106:0/3501877527 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0768058530 con 0x7f07801f34a0 2023-11-24T08:13:16.046 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 1 2023-11-24T08:13:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f076006b4a0 msgr2=0x7f076006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f076006b4a0 0x7f076006d960 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0770034000 tx=0x7f077002f000 comp rx=0 tx=0).stop 2023-11-24T08:13:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 msgr2=0x7f07800ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 0x7f07800ca880 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f07680108e0 tx=0x7f0768010da0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 shutdown_connections 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f076006b4a0 0x7f076006d960 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f07801f3920 0x7f07800cadc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f07801f34a0 0x7f07800ca880 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 --2- 172.21.15.106:0/3501877527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f076c003970 0x7f078015dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 >> 172.21.15.106:0/3501877527 conn(0x7f078006c090 msgr2=0x7f078006d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 shutdown_connections 2023-11-24T08:13:16.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.046+0000 7f07859a9640 1 -- 172.21.15.106:0/3501877527 wait complete. 2023-11-24T08:13:16.066 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-24T08:13:16.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.267+0000 7f9e5589b640 1 Processor -- start 2023-11-24T08:13:16.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.267+0000 7f9e5589b640 1 -- start start 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e5589b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e5015d720 0x7f9e500d3330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e5589b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e500b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e5589b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5015db00 con 0x7f9e500d3870 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e5589b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e500d3d20 con 0x7f9e5015d720 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e5589b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500d0ce0 con 0x7f9e5006ca90 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e4effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f9e5015d720 0x7f9e500d3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e4e7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f9e500d3870 0x7f9e500b0ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50708/0 (socket says 172.21.15.106:50708) 2023-11-24T08:13:16.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.269+0000 7f9e4e7fc640 1 -- 172.21.15.106:0/3479549895 learned_addr learned my addr 172.21.15.106:0/3479549895 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.270+0000 7f9e4effd640 1 -- 172.21.15.106:0/3479549895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 msgr2=0x7f9e500b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.270+0000 7f9e4effd640 1 --2- 172.21.15.106:0/3479549895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e500b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.270+0000 7f9e4effd640 1 -- 172.21.15.106:0/3479549895 >> v1:172.21.15.106:6789/0 conn(0x7f9e500d3870 legacy=0x7f9e500b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.270+0000 7f9e4effd640 1 -- 172.21.15.106:0/3479549895 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5006d400 con 0x7f9e5015d720 2023-11-24T08:13:16.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.270+0000 7f9e4dffb640 1 -- 172.21.15.106:0/3479549895 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401222717 0 0) 0x7f9e5015db00 con 0x7f9e500d3870 2023-11-24T08:13:16.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.271+0000 7f9e4effd640 1 --2- 172.21.15.106:0/3479549895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500d3330 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009d90 tx=0x7f9e4002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de49dd700dae0886 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:16.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.271+0000 7f9e4dffb640 1 -- 172.21.15.106:0/3479549895 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4002cbc0 con 0x7f9e5015d720 2023-11-24T08:13:16.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.271+0000 7f9e4dffb640 1 -- 172.21.15.106:0/3479549895 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4002cd20 con 0x7f9e5015d720 2023-11-24T08:13:16.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.271+0000 7f9e4dffb640 1 -- 172.21.15.106:0/3479549895 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e40034660 con 0x7f9e5015d720 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 -- 172.21.15.106:0/3479549895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 msgr2=0x7f9e500d3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 --2- 172.21.15.106:0/3479549895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500d3330 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f9e40009d90 tx=0x7f9e4002be10 comp rx=0 tx=0).stop 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 -- 172.21.15.106:0/3479549895 shutdown_connections 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 --2- 172.21.15.106:0/3479549895 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e500b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 --2- 172.21.15.106:0/3479549895 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500d3330 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 -- 172.21.15.106:0/3479549895 >> 172.21.15.106:0/3479549895 conn(0x7f9e5008af60 msgr2=0x7f9e5009c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 -- 172.21.15.106:0/3479549895 shutdown_connections 2023-11-24T08:13:16.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.272+0000 7f9e5589b640 1 -- 172.21.15.106:0/3479549895 wait complete. 2023-11-24T08:13:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.274+0000 7f9e5589b640 1 Processor -- start 2023-11-24T08:13:16.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.274+0000 7f9e5589b640 1 -- start start 2023-11-24T08:13:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e5589b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e5015db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e5589b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e500d3870 0x7f9e5014d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e5589b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500b7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e5589b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500b6980 con 0x7f9e5015d720 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e5589b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500b6c60 con 0x7f9e500d3870 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e5589b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e500b6f40 con 0x7f9e5006ca90 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500b7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500b7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45580/0 (socket says 172.21.15.106:45580) 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 -- 172.21.15.106:0/4281993053 learned_addr learned my addr 172.21.15.106:0/4281993053 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4effd640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e5015db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4e7fc640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e500d3870 0x7f9e5014d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 -- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 msgr2=0x7f9e5015db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e5015db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 -- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e500d3870 msgr2=0x7f9e5014d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e500d3870 0x7f9e5014d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.275+0000 7f9e4f7fe640 1 -- 172.21.15.106:0/4281993053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e40009a70 con 0x7f9e5015d720 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.276+0000 7f9e4f7fe640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500b7a20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b410 tx=0x7f9e4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.276+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48011830 con 0x7f9e5015d720 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.276+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e480119b0 con 0x7f9e5015d720 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.276+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e500b7220 con 0x7f9e5015d720 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.276+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e48011d60 con 0x7f9e5015d720 2023-11-24T08:13:16.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.276+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5015e040 con 0x7f9e5015d720 2023-11-24T08:13:16.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.277+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e500b7740 con 0x7f9e5015d720 2023-11-24T08:13:16.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.278+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f9e4801a400 con 0x7f9e5015d720 2023-11-24T08:13:16.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.279+0000 7f9e47fff640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e3006b470 0x7f9e3006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.279+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e4808a200 con 0x7f9e5015d720 2023-11-24T08:13:16.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.279+0000 7f9e4effd640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e3006b470 0x7f9e3006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.280+0000 7f9e4effd640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e3006b470 0x7f9e3006d930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9e40036000 tx=0x7f9e40030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:16.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.283+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9e500b7740 con 0x7f9e5015d720 2023-11-24T08:13:16.474 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:13:16.484 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:13:16.484 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 32 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.477+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9e500b7740 con 0x7f9e5015d720 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.478+0000 7f9e47fff640 1 -- 172.21.15.106:0/4281993053 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9e500b7740 con 0x7f9e5015d720 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e3006b470 msgr2=0x7f9e3006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e3006b470 0x7f9e3006d930 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9e40036000 tx=0x7f9e40030000 comp rx=0 tx=0).stop 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 msgr2=0x7f9e500b7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500b7a20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f9e4800b410 tx=0x7f9e4800b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 shutdown_connections 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f9e5015d720 0x7f9e500b7a20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e500d3870 0x7f9e5014d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f9e3006b470 0x7f9e3006d930 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 --2- 172.21.15.106:0/4281993053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f9e5006ca90 0x7f9e5015db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.480+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 >> 172.21.15.106:0/4281993053 conn(0x7f9e5008af60 msgr2=0x7f9e500ce150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.481+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 shutdown_connections 2023-11-24T08:13:16.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.481+0000 7f9e5589b640 1 -- 172.21.15.106:0/4281993053 wait complete. 2023-11-24T08:13:16.499 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-24T08:13:16.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.719+0000 7fe483fff640 1 Processor -- start 2023-11-24T08:13:16.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.720+0000 7fe483fff640 1 -- start start 2023-11-24T08:13:16.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.721+0000 7fe483fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 0x7fe4840cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.721+0000 7fe483fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840d15e0 con 0x7fe48415d3f0 2023-11-24T08:13:16.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.721+0000 7fe483fff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4840d0280 con 0x7fe4840d1f40 2023-11-24T08:13:16.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.721+0000 7fe483fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840b0eb0 con 0x7fe48415d7d0 2023-11-24T08:13:16.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.721+0000 7fe482ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe48415d3f0 0x7fe4840d1830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50712/0 (socket says 172.21.15.106:50712) 2023-11-24T08:13:16.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.721+0000 7fe482ffd640 1 -- 172.21.15.106:0/177257008 learned_addr learned my addr 172.21.15.106:0/177257008 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427297860 0 0) 0x7fe4840d15e0 con 0x7fe48415d3f0 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe470003520 con 0x7fe48415d3f0 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe4827fc640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 0x7fe4840cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3123897483 0 0) 0x7fe470003520 con 0x7fe48415d3f0 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 msgr2=0x7fe4840cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 0x7fe4840cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 >> v1:172.21.15.156:6789/0 conn(0x7fe4840d1f40 legacy=0x7fe4840ce280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840d1300 con 0x7fe48415d3f0 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fe46c001670 con 0x7fe48415d3f0 2023-11-24T08:13:16.726 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 >> v1:172.21.15.106:6789/0 conn(0x7fe48415d3f0 legacy=0x7fe4840d1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 0x7fe4700039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470004000 0x7fe4700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe470004d00 0x7fe470005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840b0eb0 con 0x7fe470004d00 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47000bb70 con 0x7fe470004000 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.722+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47000bd30 con 0x7fe48415d7d0 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.723+0000 7fe4837fe640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe470004d00 0x7fe470005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.723+0000 7fe4837fe640 1 -- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 msgr2=0x7fe4700039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.723+0000 7fe4837fe640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe48415d7d0 0x7fe4700039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.723+0000 7fe4837fe640 1 -- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470004000 msgr2=0x7fe4700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.723+0000 7fe4837fe640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe470004000 0x7fe4700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.727 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.723+0000 7fe4837fe640 1 -- 172.21.15.106:0/177257008 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe470003520 con 0x7fe470004d00 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.724+0000 7fe4837fe640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe470004d00 0x7fe470005180 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe47c003d80 tx=0x7fe4700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f758e3fdfc60b146 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.724+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c032600 con 0x7fe470004d00 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.724+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c02e050 con 0x7fe470004d00 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.724+0000 7fe481ffb640 1 -- 172.21.15.106:0/177257008 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0329e0 con 0x7fe470004d00 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe483fff640 1 -- 172.21.15.106:0/177257008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe470004d00 msgr2=0x7fe470005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe483fff640 1 --2- 172.21.15.106:0/177257008 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe470004d00 0x7fe470005180 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fe47c003d80 tx=0x7fe4700057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe4837fe640 1 -- 172.21.15.106:0/177257008 reap_dead start 2023-11-24T08:13:16.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe483fff640 1 -- 172.21.15.106:0/177257008 shutdown_connections 2023-11-24T08:13:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe483fff640 1 -- 172.21.15.106:0/177257008 >> 172.21.15.106:0/177257008 conn(0x7fe48409c5b0 msgr2=0x7fe4840d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe483fff640 1 -- 172.21.15.106:0/177257008 shutdown_connections 2023-11-24T08:13:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.725+0000 7fe483fff640 1 -- 172.21.15.106:0/177257008 wait complete. 2023-11-24T08:13:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.726+0000 7fe483fff640 1 Processor -- start 2023-11-24T08:13:16.730 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.727+0000 7fe483fff640 1 -- start start 2023-11-24T08:13:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.727+0000 7fe483fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe470004d00 0x7fe4840c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.731 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe483fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4841605c0 0x7fe484159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe483fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 0x7fe48415ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe483fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840cb860 con 0x7fe4840b3330 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe483fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4840cd550 con 0x7fe4841605c0 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe483fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe484159ed0 con 0x7fe470004d00 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 0x7fe48415ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 0x7fe48415ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45600/0 (socket says 172.21.15.106:45600) 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 -- 172.21.15.106:0/1853581128 learned_addr learned my addr 172.21.15.106:0/1853581128 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 -- 172.21.15.106:0/1853581128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe470004d00 msgr2=0x7fe4840c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe470004d00 0x7fe4840c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 -- 172.21.15.106:0/1853581128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4841605c0 msgr2=0x7fe484159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.732 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4841605c0 0x7fe484159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.728+0000 7fe4837fe640 1 -- 172.21.15.106:0/1853581128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4840ccf90 con 0x7fe4840b3330 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.729+0000 7fe4837fe640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 0x7fe48415ab30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe47c003d80 tx=0x7fe47c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.729+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c03d630 con 0x7fe4840b3330 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.729+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c03d790 con 0x7fe4840b3330 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.729+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c03db40 con 0x7fe4840b3330 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.729+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4840cd270 con 0x7fe4840b3330 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.729+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4840b37c0 con 0x7fe4840b3330 2023-11-24T08:13:16.733 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.730+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4840d06f0 con 0x7fe4840b3330 2023-11-24T08:13:16.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.731+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe47c03dca0 con 0x7fe4840b3330 2023-11-24T08:13:16.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.732+0000 7fe47b7fe640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe46406b470 0x7fe46406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:16.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.732+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe47c000e40 con 0x7fe4840b3330 2023-11-24T08:13:16.734 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.732+0000 7fe482ffd640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe46406b470 0x7fe46406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:16.735 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.733+0000 7fe482ffd640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe46406b470 0x7fe46406d930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe46c001eb0 tx=0x7fe46c001600 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:16.738 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.736+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fe4840d06f0 con 0x7fe4840b3330 2023-11-24T08:13:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.926+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe4840d06f0 con 0x7fe4840b3330 2023-11-24T08:13:16.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.927+0000 7fe47b7fe640 1 -- 172.21.15.106:0/1853581128 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe4840d06f0 con 0x7fe4840b3330 2023-11-24T08:13:16.930 INFO:teuthology.orchestra.run.smithi106.stdout:pg_num: 64 2023-11-24T08:13:16.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.929+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe46406b470 msgr2=0x7fe46406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.929+0000 7fe483fff640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe46406b470 0x7fe46406d930 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fe46c001eb0 tx=0x7fe46c001600 comp rx=0 tx=0).stop 2023-11-24T08:13:16.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.929+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 msgr2=0x7fe48415ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:16.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.929+0000 7fe483fff640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 0x7fe48415ab30 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fe47c003d80 tx=0x7fe47c0036f0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 shutdown_connections 2023-11-24T08:13:16.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe4840b3330 0x7fe48415ab30 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe4841605c0 0x7fe484159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe46406b470 0x7fe46406d930 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 --2- 172.21.15.106:0/1853581128 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe470004d00 0x7fe4840c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:16.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 >> 172.21.15.106:0/1853581128 conn(0x7fe48409c5b0 msgr2=0x7fe4840cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:16.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 shutdown_connections 2023-11-24T08:13:16.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:16.930+0000 7fe483fff640 1 -- 172.21.15.106:0/1853581128 wait complete. 2023-11-24T08:13:16.950 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:17.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.158+0000 7fda96af8640 1 Processor -- start 2023-11-24T08:13:17.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.158+0000 7fda96af8640 1 -- start start 2023-11-24T08:13:17.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.159+0000 7fda96af8640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 0x7fda9015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.159+0000 7fda96af8640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fda9006db20 0x7fda900b1180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.166 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.159+0000 7fda96af8640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda900cec00 con 0x7fda9006d6c0 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.159+0000 7fda96af8640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9013f9e0 con 0x7fda9006db20 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.159+0000 7fda96af8640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9013fcc0 con 0x7fda9013d120 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda952f5640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fda9006d6c0 0x7fda900b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50714/0 (socket says 172.21.15.106:50714) 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda952f5640 1 -- 172.21.15.106:0/2968566021 learned_addr learned my addr 172.21.15.106:0/2968566021 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda95af6640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 0x7fda9015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda962f7640 1 --2- 172.21.15.106:0/2968566021 >> v2:172.21.15.156:3300/0 conn(0x7fda9006db20 0x7fda900b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529705260 0 0) 0x7fda900cec00 con 0x7fda9006d6c0 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda7c003520 con 0x7fda9006d6c0 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339888566 0 0) 0x7fda7c003520 con 0x7fda9006d6c0 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 msgr2=0x7fda9015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:17.167 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 0x7fda9015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 >> v2:172.21.15.156:3300/0 conn(0x7fda9006db20 msgr2=0x7fda900b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 --2- 172.21.15.106:0/2968566021 >> v2:172.21.15.156:3300/0 conn(0x7fda9006db20 0x7fda900b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.160+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9013ffa0 con 0x7fda9006d6c0 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fda80001670 con 0x7fda9006d6c0 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 >> v1:172.21.15.106:6789/0 conn(0x7fda9006d6c0 legacy=0x7fda900b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda7c003970 0x7fda7c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 0x7fda7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c004d00 0x7fda7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9013f9e0 con 0x7fda7c003970 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9013fcc0 con 0x7fda7c004d00 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda7c00b980 con 0x7fda9013d120 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda95af6640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda7c003970 0x7fda7c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.168 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda95af6640 1 -- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 msgr2=0x7fda7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda95af6640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda9013d120 0x7fda7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda95af6640 1 -- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c004d00 msgr2=0x7fda7c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda95af6640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda7c004d00 0x7fda7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.161+0000 7fda95af6640 1 -- 172.21.15.106:0/2968566021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda7c003520 con 0x7fda7c003970 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.162+0000 7fda95af6640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda7c003970 0x7fda7c003d90 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009ac0 tx=0x7fda7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=570798630b2e157c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c02c710 con 0x7fda7c003970 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c02c870 con 0x7fda7c003970 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda94af4640 1 -- 172.21.15.106:0/2968566021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c02cc00 con 0x7fda7c003970 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda96af8640 1 -- 172.21.15.106:0/2968566021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda7c003970 msgr2=0x7fda7c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda96af8640 1 --2- 172.21.15.106:0/2968566021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda7c003970 0x7fda7c003d90 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fda8c009ac0 tx=0x7fda7c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.169 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda962f7640 1 -- 172.21.15.106:0/2968566021 reap_dead start 2023-11-24T08:13:17.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda96af8640 1 -- 172.21.15.106:0/2968566021 shutdown_connections 2023-11-24T08:13:17.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda96af8640 1 -- 172.21.15.106:0/2968566021 >> 172.21.15.106:0/2968566021 conn(0x7fda900cb830 msgr2=0x7fda900cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:17.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda96af8640 1 -- 172.21.15.106:0/2968566021 shutdown_connections 2023-11-24T08:13:17.170 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.163+0000 7fda96af8640 1 -- 172.21.15.106:0/2968566021 wait complete. 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.165+0000 7fda96af8640 1 Processor -- start 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.165+0000 7fda96af8640 1 -- start start 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda96af8640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c003970 0x7fda901396f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda96af8640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9013b690 0x7fda9013bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda96af8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 0x7fda900acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda96af8640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda900ae2a0 con 0x7fda900ac8b0 2023-11-24T08:13:17.172 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda96af8640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9013c050 con 0x7fda9013b690 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda96af8640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda900ad620 con 0x7fda7c003970 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda962f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 0x7fda900acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda962f7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 0x7fda900acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45610/0 (socket says 172.21.15.106:45610) 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.166+0000 7fda962f7640 1 -- 172.21.15.106:0/3099407819 learned_addr learned my addr 172.21.15.106:0/3099407819 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda95af6640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c003970 0x7fda901396f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda952f5640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9013b690 0x7fda9013bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda962f7640 1 -- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c003970 msgr2=0x7fda901396f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda962f7640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c003970 0x7fda901396f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda962f7640 1 -- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9013b690 msgr2=0x7fda9013bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda962f7640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9013b690 0x7fda9013bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda962f7640 1 -- 172.21.15.106:0/3099407819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900ad900 con 0x7fda900ac8b0 2023-11-24T08:13:17.173 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda962f7640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 0x7fda900acd30 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fda90158170 tx=0x7fda8800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda88010040 con 0x7fda900ac8b0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8800bae0 con 0x7fda900ac8b0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8800be70 con 0x7fda900ac8b0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.167+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda900adbe0 con 0x7fda900ac8b0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.168+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda90139c30 con 0x7fda900ac8b0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.169+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9006d4e0 con 0x7fda900ac8b0 2023-11-24T08:13:17.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.170+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fda8801d400 con 0x7fda900ac8b0 2023-11-24T08:13:17.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.170+0000 7fda867fc640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fda7406b400 0x7fda7406d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.171+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda8808a8a0 con 0x7fda900ac8b0 2023-11-24T08:13:17.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.171+0000 7fda95af6640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fda7406b400 0x7fda7406d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.172+0000 7fda95af6640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fda7406b400 0x7fda7406d8c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fda8c036000 tx=0x7fda8c030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:17.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.175+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fda880589e0 con 0x7fda900ac8b0 2023-11-24T08:13:17.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.402+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda90140b80 con 0x7fda900ac8b0 2023-11-24T08:13:17.406 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.403+0000 7fda867fc640 1 -- 172.21.15.106:0/3099407819 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4921 (secure 0 0 0) 0x7fda88058570 con 0x7fda900ac8b0 2023-11-24T08:13:17.406 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:17.406 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:56.174099+0000","modified":"2023-11-24T08:12:59.244264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1896010072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1896010072},{"type":"v1","addr":"172.21.15.106:6837","nonce":1896010072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:13:17.407 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 54 2023-11-24T08:13:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fda7406b400 msgr2=0x7fda7406d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fda7406b400 0x7fda7406d8c0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fda8c036000 tx=0x7fda8c030000 comp rx=0 tx=0).stop 2023-11-24T08:13:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 msgr2=0x7fda900acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 0x7fda900acd30 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fda90158170 tx=0x7fda8800b480 comp rx=0 tx=0).stop 2023-11-24T08:13:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 shutdown_connections 2023-11-24T08:13:17.409 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fda900ac8b0 0x7fda900acd30 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fda9013b690 0x7fda9013bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fda7406b400 0x7fda7406d8c0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 --2- 172.21.15.106:0/3099407819 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fda7c003970 0x7fda901396f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.406+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 >> 172.21.15.106:0/3099407819 conn(0x7fda900cb830 msgr2=0x7fda9013e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:17.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.407+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 shutdown_connections 2023-11-24T08:13:17.410 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.407+0000 7fda96af8640 1 -- 172.21.15.106:0/3099407819 wait complete. 2023-11-24T08:13:17.428 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:17.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.627+0000 7f0f9594e640 1 Processor -- start 2023-11-24T08:13:17.630 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.627+0000 7f0f9594e640 1 -- start start 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9594e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f9013d120 0x7f0f9015af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9594e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f0f9006d6c0 0x7f0f9006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9594e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f900b0a70 0x7f0f900b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9594e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9006e060 con 0x7f0f9013d120 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9594e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f9013f9e0 con 0x7f0f9006d6c0 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9594e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9013fcc0 con 0x7f0f900b0a70 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f9013d120 0x7f0f9015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.632 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f0f9013d120 0x7f0f9015af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45618/0 (socket says 172.21.15.106:45618) 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 -- 172.21.15.106:0/3759263408 learned_addr learned my addr 172.21.15.106:0/3759263408 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f8ffff640 1 --2- 172.21.15.106:0/3759263408 >> v2:172.21.15.156:3300/0 conn(0x7f0f9006d6c0 0x7f0f9006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9514d640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f900b0a70 0x7f0f900b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 -- 172.21.15.106:0/3759263408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f900b0a70 msgr2=0x7f0f900b0ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f900b0a70 0x7f0f900b0ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 -- 172.21.15.106:0/3759263408 >> v2:172.21.15.156:3300/0 conn(0x7f0f9006d6c0 msgr2=0x7f0f9006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.629+0000 7f0f9494c640 1 --2- 172.21.15.106:0/3759263408 >> v2:172.21.15.156:3300/0 conn(0x7f0f9006d6c0 0x7f0f9006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.630+0000 7f0f9494c640 1 -- 172.21.15.106:0/3759263408 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f9013ffa0 con 0x7f0f9013d120 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.630+0000 7f0f9514d640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f900b0a70 0x7f0f900b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:17.633 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.630+0000 7f0f8ffff640 1 --2- 172.21.15.106:0/3759263408 >> v2:172.21.15.156:3300/0 conn(0x7f0f9006d6c0 0x7f0f9006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.631+0000 7f0f9494c640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9013d120 0x7f0f9015af40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0f90072a70 tx=0x7f0f8402b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2be52476f21d83d server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.631+0000 7f0f8f7fe640 1 -- 172.21.15.106:0/3759263408 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840317d0 con 0x7f0f9013d120 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.631+0000 7f0f8f7fe640 1 -- 172.21.15.106:0/3759263408 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84031930 con 0x7f0f9013d120 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.631+0000 7f0f8f7fe640 1 -- 172.21.15.106:0/3759263408 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84031cc0 con 0x7f0f9013d120 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 -- 172.21.15.106:0/3759263408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9013d120 msgr2=0x7f0f9015af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9013d120 0x7f0f9015af40 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f0f90072a70 tx=0x7f0f8402b870 comp rx=0 tx=0).stop 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 -- 172.21.15.106:0/3759263408 shutdown_connections 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f900b0a70 0x7f0f900b0ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 --2- 172.21.15.106:0/3759263408 >> v2:172.21.15.156:3300/0 conn(0x7f0f9006d6c0 0x7f0f9006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.635 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 --2- 172.21.15.106:0/3759263408 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9013d120 0x7f0f9015af40 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 -- 172.21.15.106:0/3759263408 >> 172.21.15.106:0/3759263408 conn(0x7f0f900cb830 msgr2=0x7f0f900cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:17.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 -- 172.21.15.106:0/3759263408 shutdown_connections 2023-11-24T08:13:17.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.632+0000 7f0f9594e640 1 -- 172.21.15.106:0/3759263408 wait complete. 2023-11-24T08:13:17.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.634+0000 7f0f9594e640 1 Processor -- start 2023-11-24T08:13:17.636 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.634+0000 7f0f9594e640 1 -- start start 2023-11-24T08:13:17.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9594e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 0x7f0f901659c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.637 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9594e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900b0a70 0x7f0f901506a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9594e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f9013d120 0x7f0f900cf310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9594e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f90164ac0 con 0x7f0f9006d6c0 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9594e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900cfa10 con 0x7f0f900b0a70 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9594e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901632e0 con 0x7f0f9013d120 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 0x7f0f901659c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 0x7f0f901659c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45626/0 (socket says 172.21.15.106:45626) 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 -- 172.21.15.106:0/1134201007 learned_addr learned my addr 172.21.15.106:0/1134201007 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:17.638 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f8ffff640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900b0a70 0x7f0f901506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9514d640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f9013d120 0x7f0f900cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.639 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 -- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f9013d120 msgr2=0x7f0f900cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f9013d120 0x7f0f900cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 -- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900b0a70 msgr2=0x7f0f901506a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.635+0000 7f0f9494c640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900b0a70 0x7f0f901506a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f9494c640 1 -- 172.21.15.106:0/1134201007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f840094d0 con 0x7f0f9006d6c0 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f9494c640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 0x7f0f901659c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0f8402b8b0 tx=0x7f0f840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f840328f0 con 0x7f0f9006d6c0 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84032a70 con 0x7f0f9006d6c0 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84032e20 con 0x7f0f9006d6c0 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f90163580 con 0x7f0f9006d6c0 2023-11-24T08:13:17.640 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.636+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f90163b40 con 0x7f0f9006d6c0 2023-11-24T08:13:17.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.637+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f9004e840 con 0x7f0f9006d6c0 2023-11-24T08:13:17.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.638+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f0f84030070 con 0x7f0f9006d6c0 2023-11-24T08:13:17.641 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.639+0000 7f0f8d7fa640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f7406b470 0x7f0f7406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:17.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.640+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f840ac2d0 con 0x7f0f9006d6c0 2023-11-24T08:13:17.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.640+0000 7f0f8ffff640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f7406b470 0x7f0f7406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:17.642 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.640+0000 7f0f8ffff640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f7406b470 0x7f0f7406d930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f9013d7f0 tx=0x7f0f7c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:17.646 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.644+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f0f840768f0 con 0x7f0f9006d6c0 2023-11-24T08:13:17.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.822+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f90164d90 con 0x7f0f9006d6c0 2023-11-24T08:13:17.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.824+0000 7f0f8d7fa640 1 -- 172.21.15.106:0/1134201007 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16608 (secure 0 0 0) 0x7f0f84079f10 con 0x7f0f9006d6c0 2023-11-24T08:13:17.827 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:17.827 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:57.727441+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:17.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f7406b470 msgr2=0x7f0f7406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f7406b470 0x7f0f7406d930 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0f9013d7f0 tx=0x7f0f7c008040 comp rx=0 tx=0).stop 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 msgr2=0x7f0f901659c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 0x7f0f901659c0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f0f8402b8b0 tx=0x7f0f840046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 shutdown_connections 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f0f7406b470 0x7f0f7406d930 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f0f9013d120 0x7f0f900cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0f900b0a70 0x7f0f901506a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 --2- 172.21.15.106:0/1134201007 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f0f9006d6c0 0x7f0f901659c0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 >> 172.21.15.106:0/1134201007 conn(0x7f0f900cb830 msgr2=0x7f0f900af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 shutdown_connections 2023-11-24T08:13:17.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:17.827+0000 7f0f9594e640 1 -- 172.21.15.106:0/1134201007 wait complete. 2023-11-24T08:13:17.849 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:13:17.849 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.044+0000 7f1634a9b640 1 Processor -- start 2023-11-24T08:13:18.047 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.045+0000 7f1634a9b640 1 -- start start 2023-11-24T08:13:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f1634a9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16300b2250 0x7f163015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f1634a9b640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163015ab80 con 0x7f16300b2630 2023-11-24T08:13:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f1634a9b640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f163015e680 con 0x7f16300b2aa0 2023-11-24T08:13:18.048 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f1634a9b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f163015a990 con 0x7f16300b2250 2023-11-24T08:13:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f162e59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f16300b2250 0x7f163015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f1625d9b640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f16300b2630 0x7f16300ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50728/0 (socket says 172.21.15.106:50728) 2023-11-24T08:13:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.046+0000 7f1625d9b640 1 -- 172.21.15.106:0/1990496232 learned_addr learned my addr 172.21.15.106:0/1990496232 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.047+0000 7f162e59c640 1 -- 172.21.15.106:0/1990496232 >> v1:172.21.15.156:6789/0 conn(0x7f16300b2aa0 legacy=0x7f163015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:18.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.047+0000 7f162e59c640 1 -- 172.21.15.106:0/1990496232 >> v1:172.21.15.106:6789/0 conn(0x7f16300b2630 legacy=0x7f16300ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.047+0000 7f162e59c640 1 -- 172.21.15.106:0/1990496232 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f163015e960 con 0x7f16300b2250 2023-11-24T08:13:18.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.047+0000 7f162dd9b640 1 -- 172.21.15.106:0/1990496232 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 140171048 0 0) 0x7f163015ab80 con 0x7f16300b2630 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f162e59c640 1 --2- 172.21.15.106:0/1990496232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f163015df80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f163015ad70 tx=0x7f162002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6724e62eb27606fc server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f162dd9b640 1 -- 172.21.15.106:0/1990496232 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620031830 con 0x7f16300b2250 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f162dd9b640 1 -- 172.21.15.106:0/1990496232 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620031990 con 0x7f16300b2250 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f162dd9b640 1 -- 172.21.15.106:0/1990496232 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620031d20 con 0x7f16300b2250 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f1634a9b640 1 -- 172.21.15.106:0/1990496232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 msgr2=0x7f163015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f1634a9b640 1 --2- 172.21.15.106:0/1990496232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f163015df80 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f163015ad70 tx=0x7f162002b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.048+0000 7f1634a9b640 1 -- 172.21.15.106:0/1990496232 shutdown_connections 2023-11-24T08:13:18.051 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.049+0000 7f1634a9b640 1 --2- 172.21.15.106:0/1990496232 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f163015df80 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.049+0000 7f1634a9b640 1 -- 172.21.15.106:0/1990496232 >> 172.21.15.106:0/1990496232 conn(0x7f163006a920 msgr2=0x7f163006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.049+0000 7f1634a9b640 1 -- 172.21.15.106:0/1990496232 shutdown_connections 2023-11-24T08:13:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.049+0000 7f1634a9b640 1 -- 172.21.15.106:0/1990496232 wait complete. 2023-11-24T08:13:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.050+0000 7f1634a9b640 1 Processor -- start 2023-11-24T08:13:18.052 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.050+0000 7f1634a9b640 1 -- start start 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f1634a9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f16301509c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f1634a9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300b2630 0x7f16300c99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f1634a9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300b2aa0 0x7f16300cba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f1634a9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630150f00 con 0x7f16300b2250 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f1634a9b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16300c86b0 con 0x7f16300b2aa0 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f1634a9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16300c8990 con 0x7f16300b2630 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f162e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f16301509c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f162e59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f16301509c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45640/0 (socket says 172.21.15.106:45640) 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.051+0000 7f162e59c640 1 -- 172.21.15.106:0/2999392701 learned_addr learned my addr 172.21.15.106:0/2999392701 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f162e59c640 1 -- 172.21.15.106:0/2999392701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300b2630 msgr2=0x7f16300c99d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-24T08:13:18.055 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f162e59c640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300b2630 0x7f16300c99d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f162e59c640 1 -- 172.21.15.106:0/2999392701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300b2aa0 msgr2=0x7f16300cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f162e59c640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300b2aa0 0x7f16300cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f162e59c640 1 -- 172.21.15.106:0/2999392701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620009400 con 0x7f16300b2250 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f162e59c640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f16301509c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1620009890 tx=0x7f16200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162003ba70 con 0x7f16300b2250 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f162003bbf0 con 0x7f16300b2250 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16200453f0 con 0x7f16300b2250 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.052+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16300cd7c0 con 0x7f16300b2250 2023-11-24T08:13:18.056 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.053+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16300c8f50 con 0x7f16300b2250 2023-11-24T08:13:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.054+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16301550a0 con 0x7f16300b2250 2023-11-24T08:13:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.054+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1620045550 con 0x7f16300b2250 2023-11-24T08:13:18.057 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.055+0000 7f16277fe640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161006b470 0x7f161006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.055+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f16200ac4b0 con 0x7f16300b2250 2023-11-24T08:13:18.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.056+0000 7f1625d9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161006b470 0x7f161006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.058 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.056+0000 7f1625d9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161006b470 0x7f161006d930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f16180011e0 tx=0x7f16180078b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.062 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.060+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1620076cc0 con 0x7f16300b2250 2023-11-24T08:13:18.270 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.268+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f163015d8b0 con 0x7f16300b2250 2023-11-24T08:13:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.269+0000 7f16277fe640 1 -- 172.21.15.106:0/2999392701 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f162007a2e0 con 0x7f16300b2250 2023-11-24T08:13:18.272 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:18.272 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:56.174099+0000","modified":"2023-11-24T08:12:59.244264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1896010072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1896010072},{"type":"v1","addr":"172.21.15.106:6837","nonce":1896010072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:13:18.272 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 54 2023-11-24T08:13:18.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.271+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161006b470 msgr2=0x7f161006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.271+0000 7f1634a9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161006b470 0x7f161006d930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f16180011e0 tx=0x7f16180078b0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 msgr2=0x7f16301509c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f16301509c0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f1620009890 tx=0x7f16200046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 shutdown_connections 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16300b2aa0 0x7f16300cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f161006b470 0x7f161006d930 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f16300b2630 0x7f16300c99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 --2- 172.21.15.106:0/2999392701 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f16300b2250 0x7f16301509c0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 >> 172.21.15.106:0/2999392701 conn(0x7f163006a920 msgr2=0x7f163015b080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 shutdown_connections 2023-11-24T08:13:18.275 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.272+0000 7f1634a9b640 1 -- 172.21.15.106:0/2999392701 wait complete. 2023-11-24T08:13:18.292 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:18.498 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.496+0000 7f7c9e283640 1 Processor -- start 2023-11-24T08:13:18.499 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.496+0000 7f7c9e283640 1 -- start start 2023-11-24T08:13:18.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.497+0000 7f7c9e283640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c9806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c9e283640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c980ce6c0 0x7f7c980cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c9e283640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c980cece0 con 0x7f7c980cf450 2023-11-24T08:13:18.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c9e283640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c98156010 con 0x7f7c980ce6c0 2023-11-24T08:13:18.500 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c9e283640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c9806ce30 con 0x7f7c9815c7f0 2023-11-24T08:13:18.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c9806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c9c820640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c980ce6c0 0x7f7c980cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c9806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:56346/0 (socket says 172.21.15.106:56346) 2023-11-24T08:13:18.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c9c820640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7c980ce6c0 0x7f7c980cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.106:58086/0 (socket says 172.21.15.106:58086) 2023-11-24T08:13:18.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.498+0000 7f7c97fff640 1 -- 172.21.15.106:0/3773628308 learned_addr learned my addr 172.21.15.106:0/3773628308 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:18.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.499+0000 7f7c9c820640 1 -- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 msgr2=0x7f7c9806db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:18.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.499+0000 7f7c9c820640 1 --2- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c9806db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.499+0000 7f7c9c820640 1 -- 172.21.15.106:0/3773628308 >> v1:172.21.15.156:6789/0 conn(0x7f7c980cf450 legacy=0x7f7c980cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.499+0000 7f7c9c820640 1 -- 172.21.15.106:0/3773628308 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c9806d040 con 0x7f7c980ce6c0 2023-11-24T08:13:18.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.501+0000 7f7c9c820640 1 --2- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980ce6c0 0x7f7c980cf8d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009d90 tx=0x7f7c9002bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d3d78ff00afad529 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.501+0000 7f7c96ffd640 1 -- 172.21.15.106:0/3773628308 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9002cc50 con 0x7f7c980ce6c0 2023-11-24T08:13:18.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.501+0000 7f7c96ffd640 1 -- 172.21.15.106:0/3773628308 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9002cdb0 con 0x7f7c980ce6c0 2023-11-24T08:13:18.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.501+0000 7f7c96ffd640 1 -- 172.21.15.106:0/3773628308 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c90034660 con 0x7f7c980ce6c0 2023-11-24T08:13:18.504 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 -- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980ce6c0 msgr2=0x7f7c980cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 --2- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980ce6c0 0x7f7c980cf8d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f7c90009d90 tx=0x7f7c9002bea0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 -- 172.21.15.106:0/3773628308 shutdown_connections 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 --2- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980ce6c0 0x7f7c980cf8d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 --2- 172.21.15.106:0/3773628308 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c9806db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 -- 172.21.15.106:0/3773628308 >> 172.21.15.106:0/3773628308 conn(0x7f7c9806a920 msgr2=0x7f7c9806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 -- 172.21.15.106:0/3773628308 shutdown_connections 2023-11-24T08:13:18.505 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.502+0000 7f7c9e283640 1 -- 172.21.15.106:0/3773628308 wait complete. 2023-11-24T08:13:18.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.504+0000 7f7c9e283640 1 Processor -- start 2023-11-24T08:13:18.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.504+0000 7f7c9e283640 1 -- start start 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c9e283640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 0x7f7c981509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c9e283640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980cf450 0x7f7c98151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c9e283640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c981596d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c9e283640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980adb10 con 0x7f7c980ce6c0 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c9e283640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980aca70 con 0x7f7c980cf450 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c9e283640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c980acd50 con 0x7f7c9815c7f0 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 0x7f7c981509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c97fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 0x7f7c981509a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45654/0 (socket says 172.21.15.106:45654) 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.505+0000 7f7c97fff640 1 -- 172.21.15.106:0/903989530 learned_addr learned my addr 172.21.15.106:0/903989530 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c977fe640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980cf450 0x7f7c98151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c9c820640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c981596d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c97fff640 1 -- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 msgr2=0x7f7c981596d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c97fff640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c981596d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c97fff640 1 -- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980cf450 msgr2=0x7f7c98151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c97fff640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980cf450 0x7f7c98151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c97fff640 1 -- 172.21.15.106:0/903989530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c90009a70 con 0x7f7c980ce6c0 2023-11-24T08:13:18.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c977fe640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980cf450 0x7f7c98151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c97fff640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 0x7f7c981509a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800e840 tx=0x7f7c8800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8800c8e0 con 0x7f7c980ce6c0 2023-11-24T08:13:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c8800ca40 con 0x7f7c980ce6c0 2023-11-24T08:13:18.510 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c8800cdf0 con 0x7f7c980ce6c0 2023-11-24T08:13:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c980ad030 con 0x7f7c980ce6c0 2023-11-24T08:13:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.506+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c980ad5f0 con 0x7f7c980ce6c0 2023-11-24T08:13:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.507+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c98067470 con 0x7f7c980ce6c0 2023-11-24T08:13:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.508+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7c88013470 con 0x7f7c980ce6c0 2023-11-24T08:13:18.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.509+0000 7f7c94ff9640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c8406b470 0x7f7c8406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.510+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c8808a220 con 0x7f7c980ce6c0 2023-11-24T08:13:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.510+0000 7f7c977fe640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c8406b470 0x7f7c8406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.510+0000 7f7c977fe640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c8406b470 0x7f7c8406d930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c980cbb40 tx=0x7f7c8c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.514+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7c88054990 con 0x7f7c980ce6c0 2023-11-24T08:13:18.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.740+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c980cd730 con 0x7f7c980ce6c0 2023-11-24T08:13:18.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.741+0000 7f7c94ff9640 1 -- 172.21.15.106:0/903989530 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4921 (secure 0 0 0) 0x7f7c88057fb0 con 0x7f7c980ce6c0 2023-11-24T08:13:18.744 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:18.744 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":54,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-24T08:12:56.174099+0000","modified":"2023-11-24T08:12:59.244264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7020},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7020":{"gid":7020,"name":"a","rank":0,"incarnation":49,"state":"up:active","state_seq":3,"addr":"172.21.15.106:6837/1896010072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":1896010072},{"type":"v1","addr":"172.21.15.106:6837","nonce":1896010072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-11-24T08:13:18.744 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 54 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c8406b470 msgr2=0x7f7c8406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c8406b470 0x7f7c8406d930 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7c980cbb40 tx=0x7f7c8c009040 comp rx=0 tx=0).stop 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 msgr2=0x7f7c981509a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 0x7f7c981509a0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f7c8800e840 tx=0x7f7c8800ed00 comp rx=0 tx=0).stop 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 shutdown_connections 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7c8406b470 0x7f7c8406d930 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7c9815c7f0 0x7f7c981596d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7c980cf450 0x7f7c98151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 --2- 172.21.15.106:0/903989530 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7c980ce6c0 0x7f7c981509a0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.747 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 >> 172.21.15.106:0/903989530 conn(0x7f7c9806a920 msgr2=0x7f7c980734a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 shutdown_connections 2023-11-24T08:13:18.748 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.744+0000 7f7c9e283640 1 -- 172.21.15.106:0/903989530 wait complete. 2023-11-24T08:13:18.766 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:18.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.969+0000 7fb30147c640 1 Processor -- start 2023-11-24T08:13:18.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.969+0000 7fb30147c640 1 -- start start 2023-11-24T08:13:18.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.971+0000 7fb30147c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb2fc0b21d0 0x7fb2fc0cf450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.971+0000 7fb30147c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0b3a40 con 0x7fb2fc159ce0 2023-11-24T08:13:18.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.971+0000 7fb30147c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc0cdfa0 con 0x7fb2fc0cba40 2023-11-24T08:13:18.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.971+0000 7fb30147c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cd9e0 con 0x7fb2fc0b21d0 2023-11-24T08:13:18.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.971+0000 7fb2faffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fb2fc159ce0 0x7fb2fc06de80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50732/0 (socket says 172.21.15.106:50732) 2023-11-24T08:13:18.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.971+0000 7fb2faffd640 1 -- 172.21.15.106:0/2112712089 learned_addr learned my addr 172.21.15.106:0/2112712089 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4110985635 0 0) 0x7fb2fc0b3a40 con 0x7fb2fc159ce0 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2f27fc640 1 --2- 172.21.15.106:0/2112712089 >> v2:172.21.15.156:3300/0 conn(0x7fb2fc0b21d0 0x7fb2fc0cf450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2e8005730 con 0x7fb2fc159ce0 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 171605167 0 0) 0x7fb2e8005730 con 0x7fb2fc159ce0 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 >> v2:172.21.15.156:3300/0 conn(0x7fb2fc0b21d0 msgr2=0x7fb2fc0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 --2- 172.21.15.106:0/2112712089 >> v2:172.21.15.156:3300/0 conn(0x7fb2fc0b21d0 0x7fb2fc0cf450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 >> v1:172.21.15.156:6789/0 conn(0x7fb2fc0cba40 legacy=0x7fb2fc164af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0cdcc0 con 0x7fb2fc159ce0 2023-11-24T08:13:18.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fb2ec001670 con 0x7fb2fc159ce0 2023-11-24T08:13:18.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 >> v1:172.21.15.106:6789/0 conn(0x7fb2fc159ce0 legacy=0x7fb2fc06de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8005bd0 0x7fb2e8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e80069d0 0x7fb2e8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2e8007390 0x7fb2e800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.972+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cd9e0 con 0x7fb2e8005bd0 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e800e290 con 0x7fb2e8007390 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2e800e450 con 0x7fb2e80069d0 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2faffd640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8005bd0 0x7fb2e8005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2faffd640 1 -- 172.21.15.106:0/2112712089 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e80069d0 msgr2=0x7fb2e8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2faffd640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e80069d0 0x7fb2e8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2faffd640 1 -- 172.21.15.106:0/2112712089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2e8007390 msgr2=0x7fb2e800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2faffd640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2e8007390 0x7fb2e800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2faffd640 1 -- 172.21.15.106:0/2112712089 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2e8005730 con 0x7fb2e8005bd0 2023-11-24T08:13:18.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.973+0000 7fb2fb7fe640 1 -- 172.21.15.106:0/2112712089 reap_dead start 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.974+0000 7fb2faffd640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8005bd0 0x7fb2e8005ff0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00c0e0 tx=0x7fb2e8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c8174f61c4768a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.974+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec005ad0 con 0x7fb2e8005bd0 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.974+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec005c30 con 0x7fb2e8005bd0 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.974+0000 7fb2fa7fc640 1 -- 172.21.15.106:0/2112712089 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec02ca30 con 0x7fb2e8005bd0 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 -- 172.21.15.106:0/2112712089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8005bd0 msgr2=0x7fb2e8005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8005bd0 0x7fb2e8005ff0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec00c0e0 tx=0x7fb2e8007a00 comp rx=0 tx=0).stop 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 -- 172.21.15.106:0/2112712089 shutdown_connections 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 --2- 172.21.15.106:0/2112712089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2e8005bd0 0x7fb2e8005ff0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 -- 172.21.15.106:0/2112712089 >> 172.21.15.106:0/2112712089 conn(0x7fb2fc06a920 msgr2=0x7fb2fc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 -- 172.21.15.106:0/2112712089 shutdown_connections 2023-11-24T08:13:18.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.975+0000 7fb30147c640 1 -- 172.21.15.106:0/2112712089 wait complete. 2023-11-24T08:13:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.977+0000 7fb30147c640 1 Processor -- start 2023-11-24T08:13:18.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.977+0000 7fb30147c640 1 -- start start 2023-11-24T08:13:18.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.978+0000 7fb30147c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e8005bd0 0x7fb2fc14e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.978+0000 7fb30147c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 0x7fb2fc150d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.978+0000 7fb30147c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2fc151290 0x7fb2fc1f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.978+0000 7fb30147c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc1f3480 con 0x7fb2fc149020 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.978+0000 7fb30147c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc14fbe0 con 0x7fb2fc151290 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.978+0000 7fb30147c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc151710 con 0x7fb2e8005bd0 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 0x7fb2fc150d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 0x7fb2fc150d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45662/0 (socket says 172.21.15.106:45662) 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 -- 172.21.15.106:0/912572021 learned_addr learned my addr 172.21.15.106:0/912572021 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2faffd640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e8005bd0 0x7fb2fc14e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2fb7fe640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2fc151290 0x7fb2fc1f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 -- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e8005bd0 msgr2=0x7fb2fc14e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e8005bd0 0x7fb2fc14e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 -- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2fc151290 msgr2=0x7fb2fc1f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2fc151290 0x7fb2fc1f6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 -- 172.21.15.106:0/912572021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc14ef60 con 0x7fb2fc149020 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2faffd640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e8005bd0 0x7fb2fc14e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:18.987 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.979+0000 7fb2f27fc640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 0x7fb2fc150d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e7a0 tx=0x7fb2e400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.980+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400ee50 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.980+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e400c940 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.980+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc14f240 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.980+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400ccf0 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.980+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc14f800 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.981+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc14fd20 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.982+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb2e4013470 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.983+0000 7fb2f3fff640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb2dc06b4a0 0x7fb2dc06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.983+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2e408a020 con 0x7fb2fc149020 2023-11-24T08:13:18.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.983+0000 7fb2faffd640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb2dc06b4a0 0x7fb2dc06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:18.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.984+0000 7fb2faffd640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb2dc06b4a0 0x7fb2dc06d960 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec034000 tx=0x7fb2ec009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:18.992 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:18.989+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb2fc14fd20 con 0x7fb2fc149020 2023-11-24T08:13:19.181 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.178+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2fc14fd20 con 0x7fb2fc149020 2023-11-24T08:13:19.183 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.181+0000 7fb2f3fff640 1 -- 172.21.15.106:0/912572021 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+16608 (secure 0 0 0) 0x7fb2fc14fd20 con 0x7fb2fc149020 2023-11-24T08:13:19.184 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:19.184 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":89,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:12:57.727441+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-11-24T08:12:53.987858+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-11-24T08:12:55.193239+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:19.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb2dc06b4a0 msgr2=0x7fb2dc06d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb2dc06b4a0 0x7fb2dc06d960 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec034000 tx=0x7fb2ec009030 comp rx=0 tx=0).stop 2023-11-24T08:13:19.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 msgr2=0x7fb2fc150d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 0x7fb2fc150d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400e7a0 tx=0x7fb2e400ec60 comp rx=0 tx=0).stop 2023-11-24T08:13:19.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 shutdown_connections 2023-11-24T08:13:19.186 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2fc151290 0x7fb2fc1f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb2fc149020 0x7fb2fc150d50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb2dc06b4a0 0x7fb2dc06d960 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 --2- 172.21.15.106:0/912572021 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb2e8005bd0 0x7fb2fc14e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.187 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.183+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 >> 172.21.15.106:0/912572021 conn(0x7fb2fc06a920 msgr2=0x7fb2fc0af4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:19.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.184+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 shutdown_connections 2023-11-24T08:13:19.188 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.184+0000 7fb30147c640 1 -- 172.21.15.106:0/912572021 wait complete. 2023-11-24T08:13:19.207 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-24T08:13:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.416+0000 7f1bf3b0c640 1 Processor -- start 2023-11-24T08:13:19.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.417+0000 7f1bf3b0c640 1 -- start start 2023-11-24T08:13:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf3b0c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec1660b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf3b0c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1bec0ccfb0 0x7f1bec162000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf3b0c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f1bec162540 0x7f1bec0c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf3b0c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec1655f0 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf3b0c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1bec0cc380 con 0x7f1bec162540 2023-11-24T08:13:19.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf3b0c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec162b30 con 0x7f1bec0ccbd0 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf10a7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1bec0ccfb0 0x7f1bec162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf10a7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f1bec0ccfb0 0x7f1bec162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45670/0 (socket says 172.21.15.106:45670) 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1986088078 learned_addr learned my addr 172.21.15.106:0/1986088078 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf20a9640 1 --2- 172.21.15.106:0/1986088078 >> v2:172.21.15.156:3300/0 conn(0x7f1bec162540 0x7f1bec0c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.418+0000 7f1bf18a8640 1 --2- 172.21.15.106:0/1986088078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec1660b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.419+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1986088078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 msgr2=0x7f1bec1660b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.419+0000 7f1bf10a7640 1 --2- 172.21.15.106:0/1986088078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec1660b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.419+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1986088078 >> v2:172.21.15.156:3300/0 conn(0x7f1bec162540 msgr2=0x7f1bec0c9650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.419+0000 7f1bf10a7640 1 --2- 172.21.15.106:0/1986088078 >> v2:172.21.15.156:3300/0 conn(0x7f1bec162540 0x7f1bec0c9650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.419+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1986088078 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bec162e10 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.422 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.419+0000 7f1bf20a9640 1 --2- 172.21.15.106:0/1986088078 >> v2:172.21.15.156:3300/0 conn(0x7f1bec162540 0x7f1bec0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:19.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.420+0000 7f1bf10a7640 1 --2- 172.21.15.106:0/1986088078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec162000 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009870 tx=0x7f1be802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41118074c3f8e90 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.420+0000 7f1bf08a6640 1 -- 172.21.15.106:0/1986088078 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8031640 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.420+0000 7f1bf08a6640 1 -- 172.21.15.106:0/1986088078 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be80317a0 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.420+0000 7f1bf08a6640 1 -- 172.21.15.106:0/1986088078 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8031b30 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.420+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1986088078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 msgr2=0x7f1bec162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.420+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1986088078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec162000 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f1be8009870 tx=0x7f1be802b8f0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1986088078 shutdown_connections 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1986088078 >> v2:172.21.15.156:3300/0 conn(0x7f1bec162540 0x7f1bec0c9650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1986088078 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec162000 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1986088078 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec1660b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1986088078 >> 172.21.15.106:0/1986088078 conn(0x7f1bec0b0c50 msgr2=0x7f1bec0b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1986088078 shutdown_connections 2023-11-24T08:13:19.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.421+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1986088078 wait complete. 2023-11-24T08:13:19.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.422+0000 7f1bf3b0c640 1 Processor -- start 2023-11-24T08:13:19.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.422+0000 7f1bf3b0c640 1 -- start start 2023-11-24T08:13:19.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.423+0000 7f1bf3b0c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec071b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.423+0000 7f1bf3b0c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.423+0000 7f1bf3b0c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bec162540 0x7f1bec0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.423+0000 7f1bf3b0c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0b1d40 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.423+0000 7f1bf3b0c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0b1f90 con 0x7f1bec162540 2023-11-24T08:13:19.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.423+0000 7f1bf3b0c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1bec0b2270 con 0x7f1bec0ccbd0 2023-11-24T08:13:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec14e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45686/0 (socket says 172.21.15.106:45686) 2023-11-24T08:13:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1249398046 learned_addr learned my addr 172.21.15.106:0/1249398046 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 msgr2=0x7f1bec071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf18a8640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf20a9640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bec162540 0x7f1bec0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bec162540 msgr2=0x7f1bec0b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bec162540 0x7f1bec0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 -- 172.21.15.106:0/1249398046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be80094d0 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.424+0000 7f1bf10a7640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec14e5f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000c00 tx=0x7f1be80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:19.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.425+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be803ac60 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.425+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1be803adc0 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.425+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1be8044630 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.425+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bec0b2550 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.425+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bec14eb30 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.426+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1bec0b2a70 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.427+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f1be802f050 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.428+0000 7f1be27fc640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1bd806b470 0x7f1bd806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.428+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1be80ac340 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.428+0000 7f1bf18a8640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1bd806b470 0x7f1bd806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.429+0000 7f1bf18a8640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1bd806b470 0x7f1bd806d930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b7e0 tx=0x7f1bdc008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:19.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.432+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f1bec0b2a70 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.665 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.662+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1bec0b2a70 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.666 INFO:tasks.ceph.mon.a.smithi106.stderr:2023-11-24T08:13:19.664+0000 7fb1127dc640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-24T08:13:19.669 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.667+0000 7f1be27fc640 1 -- 172.21.15.106:0/1249398046 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1bec0b2a70 con 0x7f1bec0ccfb0 2023-11-24T08:13:19.670 INFO:teuthology.orchestra.run.smithi106.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-24T08:13:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1bd806b470 msgr2=0x7f1bd806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1bd806b470 0x7f1bd806d930 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00b7e0 tx=0x7f1bdc008040 comp rx=0 tx=0).stop 2023-11-24T08:13:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 msgr2=0x7f1bec14e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec14e5f0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f1be8000c00 tx=0x7f1be80046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 shutdown_connections 2023-11-24T08:13:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bec162540 0x7f1bec0b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f1bec0ccfb0 0x7f1bec14e5f0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f1bd806b470 0x7f1bd806d930 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 --2- 172.21.15.106:0/1249398046 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f1bec0ccbd0 0x7f1bec071b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.672+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 >> 172.21.15.106:0/1249398046 conn(0x7f1bec0b0c50 msgr2=0x7f1bec0ac7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:19.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.673+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 shutdown_connections 2023-11-24T08:13:19.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.673+0000 7f1bf3b0c640 1 -- 172.21.15.106:0/1249398046 wait complete. 2023-11-24T08:13:19.689 INFO:tasks.ceph.mds.a.smithi106.stderr: -5976> 2023-11-24T08:13:02.686+0000 7f698c883640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-24T08:13:19.692 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-24T08:13:19.751 INFO:tasks.ceph.mds.a.smithi106.stdout:starting mds.a at 2023-11-24T08:13:19.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.878+0000 7f934e0d0640 1 Processor -- start 2023-11-24T08:13:19.881 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.878+0000 7f934e0d0640 1 -- start start 2023-11-24T08:13:19.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f934e0d0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f934815b440 0x7f93480cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f934e0d0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93480cbf20 0x7f93480c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f934e0d0640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f934815eea0 con 0x7f93480c57e0 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f934e0d0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93480cc3d0 con 0x7f934815b440 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f934e0d0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934815b820 con 0x7f93480cbf20 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f93477fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f934815b440 0x7f93480cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.880+0000 7f93477fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f934815b440 0x7f93480cb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45718/0 (socket says 172.21.15.106:45718) 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.881+0000 7f93477fe640 1 -- 172.21.15.106:0/1184170310 learned_addr learned my addr 172.21.15.106:0/1184170310 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.881+0000 7f93477fe640 1 -- 172.21.15.106:0/1184170310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93480cbf20 msgr2=0x7f93480c52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.881+0000 7f9346ffd640 1 --2- 172.21.15.106:0/1184170310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93480cbf20 0x7f93480c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.881+0000 7f93477fe640 1 --2- 172.21.15.106:0/1184170310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93480cbf20 0x7f93480c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.881+0000 7f93477fe640 1 -- 172.21.15.106:0/1184170310 >> v1:172.21.15.156:6789/0 conn(0x7f93480c57e0 legacy=0x7f93480c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:19.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.881+0000 7f93477fe640 1 -- 172.21.15.106:0/1184170310 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934806d600 con 0x7f934815b440 2023-11-24T08:13:19.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.882+0000 7f93477fe640 1 --2- 172.21.15.106:0/1184170310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934815b440 0x7f93480cb9e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f933802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f2e716047718908 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.882+0000 7f93467fc640 1 -- 172.21.15.106:0/1184170310 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93380317c0 con 0x7f934815b440 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.882+0000 7f93467fc640 1 -- 172.21.15.106:0/1184170310 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338031920 con 0x7f934815b440 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.882+0000 7f93467fc640 1 -- 172.21.15.106:0/1184170310 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338031cb0 con 0x7f934815b440 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 -- 172.21.15.106:0/1184170310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934815b440 msgr2=0x7f93480cb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 --2- 172.21.15.106:0/1184170310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934815b440 0x7f93480cb9e0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f933802b860 comp rx=0 tx=0).stop 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 -- 172.21.15.106:0/1184170310 shutdown_connections 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 --2- 172.21.15.106:0/1184170310 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f93480cbf20 0x7f93480c52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 --2- 172.21.15.106:0/1184170310 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f934815b440 0x7f93480cb9e0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 -- 172.21.15.106:0/1184170310 >> 172.21.15.106:0/1184170310 conn(0x7f93480b0c50 msgr2=0x7f93480b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 -- 172.21.15.106:0/1184170310 shutdown_connections 2023-11-24T08:13:19.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.883+0000 7f934e0d0640 1 -- 172.21.15.106:0/1184170310 wait complete. 2023-11-24T08:13:19.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.885+0000 7f934e0d0640 1 Processor -- start 2023-11-24T08:13:19.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.885+0000 7f934e0d0640 1 -- start start 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f934e0d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 0x7f93480999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f934e0d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93480cbf20 0x7f9348155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f934e0d0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f934815b440 0x7f934809bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f934e0d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9348099f10 con 0x7f93480c57e0 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f934e0d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93480b2c00 con 0x7f93480cbf20 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f934e0d0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934815e200 con 0x7f934815b440 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f93477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 0x7f93480999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.889 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f93477fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 0x7f93480999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45722/0 (socket says 172.21.15.106:45722) 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f93477fe640 1 -- 172.21.15.106:0/2764375278 learned_addr learned my addr 172.21.15.106:0/2764375278 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f9347fff640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f934815b440 0x7f934809bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.886+0000 7f9346ffd640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93480cbf20 0x7f9348155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f93477fe640 1 -- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f934815b440 msgr2=0x7f934809bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f93477fe640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f934815b440 0x7f934809bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f93477fe640 1 -- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93480cbf20 msgr2=0x7f9348155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f93477fe640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93480cbf20 0x7f9348155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:19.890 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f93477fe640 1 -- 172.21.15.106:0/2764375278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380094d0 con 0x7f93480c57e0 2023-11-24T08:13:19.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f93477fe640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 0x7f93480999d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f93380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:19.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338032960 con 0x7f93480c57e0 2023-11-24T08:13:19.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338032ac0 con 0x7f93480c57e0 2023-11-24T08:13:19.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338032e70 con 0x7f93480c57e0 2023-11-24T08:13:19.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f934815e4e0 con 0x7f93480c57e0 2023-11-24T08:13:19.891 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.887+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93480b3130 con 0x7f93480c57e0 2023-11-24T08:13:19.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.888+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93480b3320 con 0x7f93480c57e0 2023-11-24T08:13:19.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.889+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f933804e450 con 0x7f93480c57e0 2023-11-24T08:13:19.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.890+0000 7f932ffff640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f933406b470 0x7f933406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:19.892 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.890+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f93380ac530 con 0x7f93480c57e0 2023-11-24T08:13:19.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.890+0000 7f9346ffd640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f933406b470 0x7f933406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:19.893 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.891+0000 7f9346ffd640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f933406b470 0x7f933406d930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f93480b25e0 tx=0x7f933c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:19.897 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:19.895+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f9338076c90 con 0x7f93480c57e0 2023-11-24T08:13:20.127 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.125+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f934804e8b0 con 0x7f93480c57e0 2023-11-24T08:13:20.673 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.670+0000 7f932ffff640 1 -- 172.21.15.106:0/2764375278 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f933807a2b0 con 0x7f93480c57e0 2023-11-24T08:13:20.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.674+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f933406b470 msgr2=0x7f933406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.674+0000 7f934e0d0640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f933406b470 0x7f933406d930 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f93480b25e0 tx=0x7f933c008040 comp rx=0 tx=0).stop 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.674+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 msgr2=0x7f93480999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.674+0000 7f934e0d0640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 0x7f93480999d0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f9338009870 tx=0x7f93380046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 shutdown_connections 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f933406b470 0x7f933406d930 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f934815b440 0x7f934809bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93480cbf20 0x7f9348155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 --2- 172.21.15.106:0/2764375278 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f93480c57e0 0x7f93480999d0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 >> 172.21.15.106:0/2764375278 conn(0x7f93480b0c50 msgr2=0x7f934806cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 shutdown_connections 2023-11-24T08:13:20.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.675+0000 7f934e0d0640 1 -- 172.21.15.106:0/2764375278 wait complete. 2023-11-24T08:13:20.698 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-24T08:13:20.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.903+0000 7f3a808da640 1 Processor -- start 2023-11-24T08:13:20.906 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.904+0000 7f3a808da640 1 -- start start 2023-11-24T08:13:20.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.905+0000 7f3a808da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 0x7f3a7c161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.905+0000 7f3a808da640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c0ce0b0 con 0x7f3a7c15e390 2023-11-24T08:13:20.907 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.905+0000 7f3a808da640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a7c161960 con 0x7f3a7c0cf450 2023-11-24T08:13:20.908 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.905+0000 7f3a808da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c15e8d0 con 0x7f3a7c157c70 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a7ad9d640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f3a7c15e390 0x7f3a7c0b31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50742/0 (socket says 172.21.15.106:50742) 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a7ad9d640 1 -- 172.21.15.106:0/238704537 learned_addr learned my addr 172.21.15.106:0/238704537 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a7a59c640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 0x7f3a7c161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2303865349 0 0) 0x7f3a7c0ce0b0 con 0x7f3a7c15e390 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a68003520 con 0x7f3a7c15e390 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 927945668 0 0) 0x7f3a68003520 con 0x7f3a7c15e390 2023-11-24T08:13:20.909 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 msgr2=0x7f3a7c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 0x7f3a7c161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 >> v1:172.21.15.156:6789/0 conn(0x7f3a7c0cf450 legacy=0x7f3a7c15dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c15ebb0 con 0x7f3a7c15e390 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f3a740019f0 con 0x7f3a7c15e390 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 >> v1:172.21.15.106:6789/0 conn(0x7f3a7c15e390 legacy=0x7f3a7c0b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.906+0000 7f3a79d9b640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a680039a0 0x7f3a68003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a79d9b640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a68004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a79d9b640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 0x7f3a68005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c15e8d0 con 0x7f3a680047c0 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800bb10 con 0x7f3a680039a0 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6800bcd0 con 0x7f3a7c157c70 2023-11-24T08:13:20.910 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a73fff640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a68004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a7a59c640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a680039a0 0x7f3a68003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a7ad9d640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 0x7f3a68005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a73fff640 1 -- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 msgr2=0x7f3a68005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a73fff640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c157c70 0x7f3a68005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a73fff640 1 -- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a680039a0 msgr2=0x7f3a68003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a73fff640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a680039a0 0x7f3a68003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.912 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.907+0000 7f3a73fff640 1 -- 172.21.15.106:0/238704537 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68003520 con 0x7f3a680047c0 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.908+0000 7f3a73fff640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a68004c40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001750 tx=0x7f3a680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ba5ef44877eba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640052d0 con 0x7f3a680047c0 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6402e050 con 0x7f3a680047c0 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a79d9b640 1 -- 172.21.15.106:0/238704537 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a640035f0 con 0x7f3a680047c0 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a808da640 1 -- 172.21.15.106:0/238704537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 msgr2=0x7f3a68004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a808da640 1 --2- 172.21.15.106:0/238704537 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a68004c40 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001750 tx=0x7f3a680057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a7ad9d640 1 -- 172.21.15.106:0/238704537 reap_dead start 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a808da640 1 -- 172.21.15.106:0/238704537 shutdown_connections 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a808da640 1 -- 172.21.15.106:0/238704537 >> 172.21.15.106:0/238704537 conn(0x7f3a7c06a920 msgr2=0x7f3a7c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.909+0000 7f3a808da640 1 -- 172.21.15.106:0/238704537 shutdown_connections 2023-11-24T08:13:20.913 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.910+0000 7f3a808da640 1 -- 172.21.15.106:0/238704537 wait complete. 2023-11-24T08:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.911+0000 7f3a808da640 1 Processor -- start 2023-11-24T08:13:20.914 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.911+0000 7f3a808da640 1 -- start start 2023-11-24T08:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.912+0000 7f3a808da640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a7c1f6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.915 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.912+0000 7f3a808da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c09b310 0x7f3a7c09b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.912+0000 7f3a808da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0c8540 0x7f3a7c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.912+0000 7f3a808da640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c09bcd0 con 0x7f3a680047c0 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.912+0000 7f3a808da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c144fa0 con 0x7f3a7c0c8540 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.912+0000 7f3a808da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c1f6d50 con 0x7f3a7c09b310 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a7c1f6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a7c1f6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45738/0 (socket says 172.21.15.106:45738) 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 -- 172.21.15.106:0/1790348344 learned_addr learned my addr 172.21.15.106:0/1790348344 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a73fff640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c09b310 0x7f3a7c09b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7ad9d640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0c8540 0x7f3a7c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 -- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c09b310 msgr2=0x7f3a7c09b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c09b310 0x7f3a7c09b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.916 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 -- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0c8540 msgr2=0x7f3a7c0c89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0c8540 0x7f3a7c0c89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 -- 172.21.15.106:0/1790348344 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7c1f7030 con 0x7f3a680047c0 2023-11-24T08:13:20.917 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7ad9d640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0c8540 0x7f3a7c0c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.913+0000 7f3a7a59c640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a7c1f6770 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011600 tx=0x7f3a6c011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.914+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00f920 con 0x7f3a680047c0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.914+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6c00fa80 con 0x7f3a680047c0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.914+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c00fdf0 con 0x7f3a680047c0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.914+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c1f7310 con 0x7f3a680047c0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.914+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c0c8f30 con 0x7f3a680047c0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.915+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c067470 con 0x7f3a680047c0 2023-11-24T08:13:20.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.916+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3a6c015460 con 0x7f3a680047c0 2023-11-24T08:13:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.917+0000 7f3a72ffd640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3a5c06b470 0x7f3a5c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.917+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3a6c08a670 con 0x7f3a680047c0 2023-11-24T08:13:20.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.917+0000 7f3a73fff640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3a5c06b470 0x7f3a5c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:20.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.918+0000 7f3a73fff640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3a5c06b470 0x7f3a5c06d930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001750 tx=0x7f3a64009030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:20.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:20.921+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3a6c054e10 con 0x7f3a680047c0 2023-11-24T08:13:21.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.114+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3a7c0b1ad0 con 0x7f3a680047c0 2023-11-24T08:13:21.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.681+0000 7f3a72ffd640 1 -- 172.21.15.106:0/1790348344 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3a6c058430 con 0x7f3a680047c0 2023-11-24T08:13:21.684 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_metadata' removed 2023-11-24T08:13:21.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.684+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3a5c06b470 msgr2=0x7f3a5c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.684+0000 7f3a808da640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3a5c06b470 0x7f3a5c06d930 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3a64001750 tx=0x7f3a64009030 comp rx=0 tx=0).stop 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.684+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 msgr2=0x7f3a7c1f6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.684+0000 7f3a808da640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a7c1f6770 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c011600 tx=0x7f3a6c011ac0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 shutdown_connections 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a7c0c8540 0x7f3a7c0c89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f3a5c06b470 0x7f3a5c06d930 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3a7c09b310 0x7f3a7c09b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 --2- 172.21.15.106:0/1790348344 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3a680047c0 0x7f3a7c1f6770 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 >> 172.21.15.106:0/1790348344 conn(0x7f3a7c06a920 msgr2=0x7f3a7c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 shutdown_connections 2023-11-24T08:13:21.688 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.685+0000 7f3a808da640 1 -- 172.21.15.106:0/1790348344 wait complete. 2023-11-24T08:13:21.713 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-24T08:13:21.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.917+0000 7f46bc41c640 1 Processor -- start 2023-11-24T08:13:21.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.917+0000 7f46bc41c640 1 -- start start 2023-11-24T08:13:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46bc41c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f46b40cf450 0x7f46b40cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46bc41c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46b40cdfb0 0x7f46b40ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:21.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46bc41c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46b40cece0 con 0x7f46b415c7f0 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46bc41c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b40cfe40 con 0x7f46b40cdfb0 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46bc41c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46b4156010 con 0x7f46b40cf450 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba9b9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f46b40cdfb0 0x7f46b40ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba1b8640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f46b415c7f0 0x7f46b406db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50752/0 (socket says 172.21.15.106:50752) 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba1b8640 1 -- 172.21.15.106:0/1414218487 learned_addr learned my addr 172.21.15.106:0/1414218487 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba9b9640 1 --2- 172.21.15.106:0/1414218487 >> v2:172.21.15.106:3300/0 conn(0x7f46b40cdfb0 0x7f46b40ce3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45752/0 (socket says 172.21.15.106:45752) 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46b99b7640 1 --2- 172.21.15.106:0/1414218487 >> v2:172.21.15.156:3300/0 conn(0x7f46b40cf450 0x7f46b40cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba9b9640 1 -- 172.21.15.106:0/1414218487 >> v2:172.21.15.156:3300/0 conn(0x7f46b40cf450 msgr2=0x7f46b40cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba9b9640 1 --2- 172.21.15.106:0/1414218487 >> v2:172.21.15.156:3300/0 conn(0x7f46b40cf450 0x7f46b40cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba9b9640 1 -- 172.21.15.106:0/1414218487 >> v1:172.21.15.106:6789/0 conn(0x7f46b415c7f0 legacy=0x7f46b406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46b91b6640 1 -- 172.21.15.106:0/1414218487 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467851830 0 0) 0x7f46b40cece0 con 0x7f46b415c7f0 2023-11-24T08:13:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46ba9b9640 1 -- 172.21.15.106:0/1414218487 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46b406d040 con 0x7f46b40cdfb0 2023-11-24T08:13:21.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.919+0000 7f46b99b7640 1 --2- 172.21.15.106:0/1414218487 >> v2:172.21.15.156:3300/0 conn(0x7f46b40cf450 0x7f46b40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.920+0000 7f46ba9b9640 1 --2- 172.21.15.106:0/1414218487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b40cdfb0 0x7f46b40ce3e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f46b4070410 tx=0x7f46ac02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2de67db5b6f4e958 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46b91b6640 1 -- 172.21.15.106:0/1414218487 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac02cc70 con 0x7f46b40cdfb0 2023-11-24T08:13:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46b91b6640 1 -- 172.21.15.106:0/1414218487 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac02cdd0 con 0x7f46b40cdfb0 2023-11-24T08:13:21.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46b91b6640 1 -- 172.21.15.106:0/1414218487 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac034660 con 0x7f46b40cdfb0 2023-11-24T08:13:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46bc41c640 1 -- 172.21.15.106:0/1414218487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b40cdfb0 msgr2=0x7f46b40ce3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46bc41c640 1 --2- 172.21.15.106:0/1414218487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b40cdfb0 0x7f46b40ce3e0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f46b4070410 tx=0x7f46ac02bec0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46bc41c640 1 -- 172.21.15.106:0/1414218487 shutdown_connections 2023-11-24T08:13:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46bc41c640 1 --2- 172.21.15.106:0/1414218487 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b40cdfb0 0x7f46b40ce3e0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46bc41c640 1 --2- 172.21.15.106:0/1414218487 >> v2:172.21.15.156:3300/0 conn(0x7f46b40cf450 0x7f46b40cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.921+0000 7f46bc41c640 1 -- 172.21.15.106:0/1414218487 >> 172.21.15.106:0/1414218487 conn(0x7f46b406a920 msgr2=0x7f46b406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:21.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.922+0000 7f46bc41c640 1 -- 172.21.15.106:0/1414218487 shutdown_connections 2023-11-24T08:13:21.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.922+0000 7f46bc41c640 1 -- 172.21.15.106:0/1414218487 wait complete. 2023-11-24T08:13:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.923+0000 7f46bc41c640 1 Processor -- start 2023-11-24T08:13:21.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.923+0000 7f46bc41c640 1 -- start start 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.924+0000 7f46bc41c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b40cdfb0 0x7f46b4158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.924+0000 7f46bc41c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b40cf450 0x7f46b4149570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.924+0000 7f46bc41c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 0x7f46b41f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.924+0000 7f46bc41c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b41f3f50 con 0x7f46b415c7f0 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.924+0000 7f46bc41c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4159770 con 0x7f46b40cf450 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.924+0000 7f46bc41c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46b4159a50 con 0x7f46b40cdfb0 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 0x7f46b41f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 0x7f46b41f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45762/0 (socket says 172.21.15.106:45762) 2023-11-24T08:13:21.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 -- 172.21.15.106:0/2651219228 learned_addr learned my addr 172.21.15.106:0/2651219228 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 -- 172.21.15.106:0/2651219228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b40cdfb0 msgr2=0x7f46b4158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b40cdfb0 0x7f46b4158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 -- 172.21.15.106:0/2651219228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b40cf450 msgr2=0x7f46b4149570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b40cf450 0x7f46b4149570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 -- 172.21.15.106:0/2651219228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac009a70 con 0x7f46b415c7f0 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.925+0000 7f46ba9b9640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 0x7f46b41f3a10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009e80 tx=0x7f46ac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.926+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac040650 con 0x7f46b415c7f0 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.926+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46ac0407b0 con 0x7f46b415c7f0 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.926+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46ac040b60 con 0x7f46b415c7f0 2023-11-24T08:13:21.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.926+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46b4159d30 con 0x7f46b415c7f0 2023-11-24T08:13:21.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.926+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46b415a2f0 con 0x7f46b415c7f0 2023-11-24T08:13:21.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.927+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46b4067470 con 0x7f46b415c7f0 2023-11-24T08:13:21.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.928+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f46ac040cc0 con 0x7f46b415c7f0 2023-11-24T08:13:21.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.929+0000 7f46aaffd640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f469806b470 0x7f469806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:21.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.929+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f46ac0acd30 con 0x7f46b415c7f0 2023-11-24T08:13:21.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.929+0000 7f46ba1b8640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f469806b470 0x7f469806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:21.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.930+0000 7f46ba1b8640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f469806b470 0x7f469806d930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f46b0007fd0 tx=0x7f46b0007960 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:21.935 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:21.933+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f46ac077690 con 0x7f46b415c7f0 2023-11-24T08:13:22.126 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.124+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f46b40cd730 con 0x7f46b415c7f0 2023-11-24T08:13:22.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.684+0000 7f46aaffd640 1 -- 172.21.15.106:0/2651219228 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) v1 ==== 169+0+0 (secure 0 0 0) 0x7f46ac07acb0 con 0x7f46b415c7f0 2023-11-24T08:13:22.687 INFO:teuthology.orchestra.run.smithi106.stderr:pool 'cephfs_data' removed 2023-11-24T08:13:22.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f469806b470 msgr2=0x7f469806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f469806b470 0x7f469806d930 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f46b0007fd0 tx=0x7f46b0007960 comp rx=0 tx=0).stop 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 msgr2=0x7f46b41f3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 0x7f46b41f3a10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f46ac009e80 tx=0x7f46ac004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 shutdown_connections 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f46b415c7f0 0x7f46b41f3a10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46b40cf450 0x7f46b4149570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f469806b470 0x7f469806d930 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.691 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 --2- 172.21.15.106:0/2651219228 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f46b40cdfb0 0x7f46b4158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.688+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 >> 172.21.15.106:0/2651219228 conn(0x7f46b406a920 msgr2=0x7f46b40b03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:22.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.689+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 shutdown_connections 2023-11-24T08:13:22.692 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.689+0000 7f46bc41c640 1 -- 172.21.15.106:0/2651219228 wait complete. 2023-11-24T08:13:22.720 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-24T08:13:22.918 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.917+0000 7f6fe9ca3640 1 Processor -- start 2023-11-24T08:13:22.919 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.917+0000 7f6fe9ca3640 1 -- start start 2023-11-24T08:13:22.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe9ca3640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6fe40b1260 0x7f6fe40718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe9ca3640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe41450d0 con 0x7f6fe4141b90 2023-11-24T08:13:22.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe9ca3640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe41422b0 con 0x7f6fe4071e10 2023-11-24T08:13:22.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe9ca3640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe40725e0 con 0x7f6fe40b1260 2023-11-24T08:13:22.920 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe8ca1640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f6fe4141b90 0x7f6fe40b0b50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50762/0 (socket says 172.21.15.106:50762) 2023-11-24T08:13:22.921 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe8ca1640 1 -- 172.21.15.106:0/2680303789 learned_addr learned my addr 172.21.15.106:0/2680303789 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:22.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487269934 0 0) 0x7f6fe41450d0 con 0x7f6fe4141b90 2023-11-24T08:13:22.922 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fcc003520 con 0x7f6fe4141b90 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.918+0000 7f6fe3fff640 1 --2- 172.21.15.106:0/2680303789 >> v2:172.21.15.156:3300/0 conn(0x7f6fe40b1260 0x7f6fe40718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1069580322 0 0) 0x7f6fcc003520 con 0x7f6fe4141b90 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 >> v2:172.21.15.156:3300/0 conn(0x7f6fe40b1260 msgr2=0x7f6fe40718d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 --2- 172.21.15.106:0/2680303789 >> v2:172.21.15.156:3300/0 conn(0x7f6fe40b1260 0x7f6fe40718d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 >> v1:172.21.15.156:6789/0 conn(0x7f6fe4071e10 legacy=0x7f6fe4142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe4142520 con 0x7f6fe4141b90 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f6fd8001670 con 0x7f6fe4141b90 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 >> v1:172.21.15.106:6789/0 conn(0x7f6fe4141b90 legacy=0x7f6fe40b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fcc0039c0 0x7f6fcc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc0047c0 0x7f6fcc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.923 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fcc005250 0x7f6fcc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40725e0 con 0x7f6fcc005250 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc00c1a0 con 0x7f6fcc0047c0 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fcc010770 con 0x7f6fcc0039c0 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe94a2640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fcc005250 0x7f6fcc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe3fff640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc0047c0 0x7f6fcc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.919+0000 7f6fe8ca1640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fcc0039c0 0x7f6fcc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.920+0000 7f6fe94a2640 1 -- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fcc0039c0 msgr2=0x7f6fcc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.920+0000 7f6fe94a2640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fcc0039c0 0x7f6fcc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.920+0000 7f6fe94a2640 1 -- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc0047c0 msgr2=0x7f6fcc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.920+0000 7f6fe94a2640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc0047c0 0x7f6fcc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.920+0000 7f6fe94a2640 1 -- 172.21.15.106:0/2680303789 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc003520 con 0x7f6fcc005250 2023-11-24T08:13:22.924 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.920+0000 7f6fe8ca1640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fcc0039c0 0x7f6fcc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:22.925 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.921+0000 7f6fe94a2640 1 -- 172.21.15.106:0/2680303789 reap_dead start 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.921+0000 7f6fe94a2640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fcc005250 0x7f6fcc00b980 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0044b0 tx=0x7f6fcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9747a44be3cac8b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.921+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc006d50 con 0x7f6fcc005250 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.921+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc007020 con 0x7f6fcc005250 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.921+0000 7f6fe37fe640 1 -- 172.21.15.106:0/2680303789 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0073d0 con 0x7f6fcc005250 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/2680303789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fcc005250 msgr2=0x7f6fcc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fcc005250 0x7f6fcc00b980 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc0044b0 tx=0x7f6fcc0057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/2680303789 shutdown_connections 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/2680303789 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fcc005250 0x7f6fcc00b980 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/2680303789 >> 172.21.15.106:0/2680303789 conn(0x7f6fe406a500 msgr2=0x7f6fe406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/2680303789 shutdown_connections 2023-11-24T08:13:22.926 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.922+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/2680303789 wait complete. 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.924+0000 7f6fe9ca3640 1 Processor -- start 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.924+0000 7f6fe9ca3640 1 -- start start 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe9ca3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc005250 0x7f6fe40ce080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe9ca3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 0x7f6fe415ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe9ca3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe415dd90 0x7f6fe415e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe9ca3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe415e750 con 0x7f6fe415c630 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe9ca3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe415cfd0 con 0x7f6fcc005250 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe9ca3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40ccd90 con 0x7f6fe415dd90 2023-11-24T08:13:22.928 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 0x7f6fe415ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 0x7f6fe415ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45782/0 (socket says 172.21.15.106:45782) 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 -- 172.21.15.106:0/1147360620 learned_addr learned my addr 172.21.15.106:0/1147360620 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 -- 172.21.15.106:0/1147360620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe415dd90 msgr2=0x7f6fe415e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe415dd90 0x7f6fe415e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 -- 172.21.15.106:0/1147360620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc005250 msgr2=0x7f6fe40ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc005250 0x7f6fe40ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.925+0000 7f6fe3fff640 1 -- 172.21.15.106:0/1147360620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe40cdbf0 con 0x7f6fe415c630 2023-11-24T08:13:22.929 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.926+0000 7f6fe3fff640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 0x7f6fe415ca90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400f5b0 tx=0x7f6fd400fa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.926+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4013810 con 0x7f6fe415c630 2023-11-24T08:13:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.926+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd4013970 con 0x7f6fe415c630 2023-11-24T08:13:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.926+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd4013ce0 con 0x7f6fe415c630 2023-11-24T08:13:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.926+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe40ccfe0 con 0x7f6fe415c630 2023-11-24T08:13:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.926+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe40cd5a0 con 0x7f6fe415c630 2023-11-24T08:13:22.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.927+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe404e840 con 0x7f6fe415c630 2023-11-24T08:13:22.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.928+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6fd401f460 con 0x7f6fe415c630 2023-11-24T08:13:22.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.929+0000 7f6fe17fa640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006b470 0x7f6fd006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:22.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.931+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6fd408a2a0 con 0x7f6fe415c630 2023-11-24T08:13:22.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.931+0000 7f6fe8ca1640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006b470 0x7f6fd006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:22.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.931+0000 7f6fe8ca1640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006b470 0x7f6fd006d930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80079b0 tx=0x7f6fd8009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:22.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:22.935+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6fd4054cd0 con 0x7f6fe415c630 2023-11-24T08:13:23.128 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.126+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6fe40cb0e0 con 0x7f6fe415c630 2023-11-24T08:13:23.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.132+0000 7f6fe17fa640 1 -- 172.21.15.106:0/1147360620 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6fe40cb0e0 con 0x7f6fe415c630 2023-11-24T08:13:23.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.134+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006b470 msgr2=0x7f6fd006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.134+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006b470 0x7f6fd006d930 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6fd80079b0 tx=0x7f6fd8009040 comp rx=0 tx=0).stop 2023-11-24T08:13:23.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.134+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 msgr2=0x7f6fe415ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.134+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 0x7f6fe415ca90 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f6fd400f5b0 tx=0x7f6fd400fa70 comp rx=0 tx=0).stop 2023-11-24T08:13:23.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 shutdown_connections 2023-11-24T08:13:23.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6fd006b470 0x7f6fd006d930 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6fe415dd90 0x7f6fe415e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6fe415c630 0x7f6fe415ca90 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 --2- 172.21.15.106:0/1147360620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6fcc005250 0x7f6fe40ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.138 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 >> 172.21.15.106:0/1147360620 conn(0x7f6fe406a500 msgr2=0x7f6fe40afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:23.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 shutdown_connections 2023-11-24T08:13:23.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.135+0000 7f6fe9ca3640 1 -- 172.21.15.106:0/1147360620 wait complete. 2023-11-24T08:13:23.155 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls' 2023-11-24T08:13:23.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.367+0000 7f349dfba640 1 Processor -- start 2023-11-24T08:13:23.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.367+0000 7f349dfba640 1 -- start start 2023-11-24T08:13:23.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f349dfba640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f349815c410 0x7f349815f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:23.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f349dfba640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f349815bf80 con 0x7f3498144b30 2023-11-24T08:13:23.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f349dfba640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34980b1560 con 0x7f34980b1de0 2023-11-24T08:13:23.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f349dfba640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34980b1280 con 0x7f349815c410 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f3497fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f349815c410 0x7f349815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f3497fff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f349815c410 0x7f349815f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45792/0 (socket says 172.21.15.106:45792) 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f3497fff640 1 -- 172.21.15.106:0/2851455165 learned_addr learned my addr 172.21.15.106:0/2851455165 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f349cfb8640 1 --1- 172.21.15.106:0/2851455165 >> v1:172.21.15.106:6789/0 conn(0x7f3498144b30 0x7f34980b0a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:50770/0 (socket says 172.21.15.106:50770) 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f3497fff640 1 -- 172.21.15.106:0/2851455165 >> v1:172.21.15.156:6789/0 conn(0x7f34980b1de0 legacy=0x7f34980b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f3497fff640 1 -- 172.21.15.106:0/2851455165 >> v1:172.21.15.106:6789/0 conn(0x7f3498144b30 legacy=0x7f34980b0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.369+0000 7f3497fff640 1 -- 172.21.15.106:0/2851455165 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f349815b930 con 0x7f349815c410 2023-11-24T08:13:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.371+0000 7f3497fff640 1 --2- 172.21.15.106:0/2851455165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f349815f920 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3484009870 tx=0x7f348402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=538206992cf78f7b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:23.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.371+0000 7f34977fe640 1 -- 172.21.15.106:0/2851455165 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484030860 con 0x7f349815c410 2023-11-24T08:13:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.371+0000 7f34977fe640 1 -- 172.21.15.106:0/2851455165 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34840309c0 con 0x7f349815c410 2023-11-24T08:13:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.371+0000 7f34977fe640 1 -- 172.21.15.106:0/2851455165 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484030d50 con 0x7f349815c410 2023-11-24T08:13:23.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 -- 172.21.15.106:0/2851455165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 msgr2=0x7f349815f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 --2- 172.21.15.106:0/2851455165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f349815f920 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f3484009870 tx=0x7f348402b9a0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 -- 172.21.15.106:0/2851455165 shutdown_connections 2023-11-24T08:13:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 --2- 172.21.15.106:0/2851455165 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f349815f920 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 -- 172.21.15.106:0/2851455165 >> 172.21.15.106:0/2851455165 conn(0x7f34980cb830 msgr2=0x7f34980cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 -- 172.21.15.106:0/2851455165 shutdown_connections 2023-11-24T08:13:23.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.372+0000 7f349dfba640 1 -- 172.21.15.106:0/2851455165 wait complete. 2023-11-24T08:13:23.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.374+0000 7f349dfba640 1 Processor -- start 2023-11-24T08:13:23.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.374+0000 7f349dfba640 1 -- start start 2023-11-24T08:13:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.375+0000 7f349dfba640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34980b1de0 0x7f3498143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.375+0000 7f349dfba640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3498144b30 0x7f349815a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.375+0000 7f349dfba640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f3498141c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.375+0000 7f349dfba640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498142a40 con 0x7f349815c410 2023-11-24T08:13:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.375+0000 7f349dfba640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3498144380 con 0x7f3498144b30 2023-11-24T08:13:23.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.375+0000 7f349dfba640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f349813ee60 con 0x7f34980b1de0 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f3498141c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f3498141c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:45798/0 (socket says 172.21.15.106:45798) 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 -- 172.21.15.106:0/901498711 learned_addr learned my addr 172.21.15.106:0/901498711 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f3497fff640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3498144b30 0x7f349815a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349cfb8640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34980b1de0 0x7f3498143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 -- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34980b1de0 msgr2=0x7f3498143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34980b1de0 0x7f3498143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 -- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3498144b30 msgr2=0x7f349815a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3498144b30 0x7f349815a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.379 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.376+0000 7f349d7b9640 1 -- 172.21.15.106:0/901498711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34840094d0 con 0x7f349815c410 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.377+0000 7f349d7b9640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f3498141c90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f349000b970 tx=0x7f349000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.377+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f349000cb40 con 0x7f349815c410 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.377+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f349000cca0 con 0x7f349815c410 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.377+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f349815ae70 con 0x7f349815c410 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.377+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3490014680 con 0x7f349815c410 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.377+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3498141780 con 0x7f349815c410 2023-11-24T08:13:23.380 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.378+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f349804e840 con 0x7f349815c410 2023-11-24T08:13:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.379+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f349001d3b0 con 0x7f349815c410 2023-11-24T08:13:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.380+0000 7f34957fa640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f347c06b470 0x7f347c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:23.381 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.380+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3490089c10 con 0x7f349815c410 2023-11-24T08:13:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.380+0000 7f349cfb8640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f347c06b470 0x7f347c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:23.382 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.381+0000 7f349cfb8640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f347c06b470 0x7f347c06d930 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f348c000f60 tx=0x7f348c007c80 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:23.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.385+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3490054710 con 0x7f349815c410 2023-11-24T08:13:23.643 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.641+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]} v 0) v1 -- 0x7f349806d860 con 0x7f349815c410 2023-11-24T08:13:23.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.688+0000 7f34957fa640 1 -- 172.21.15.106:0/901498711 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestSubvolumeSnapshots.test_subvolume_inherited_snapshot_ls"]}]=0 v273) v1 ==== 164+0+0 (secure 0 0 0) 0x7f3490057d30 con 0x7f349815c410 2023-11-24T08:13:23.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.691+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f347c06b470 msgr2=0x7f347c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.691+0000 7f349dfba640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f347c06b470 0x7f347c06d930 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f348c000f60 tx=0x7f348c007c80 comp rx=0 tx=0).stop 2023-11-24T08:13:23.693 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.691+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 msgr2=0x7f3498141c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.691+0000 7f349dfba640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f3498141c90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f349000b970 tx=0x7f349000be30 comp rx=0 tx=0).stop 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 shutdown_connections 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f349815c410 0x7f3498141c90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3498144b30 0x7f349815a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f347c06b470 0x7f347c06d930 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 --2- 172.21.15.106:0/901498711 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f34980b1de0 0x7f3498143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 >> 172.21.15.106:0/901498711 conn(0x7f34980cb830 msgr2=0x7f349815d200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 shutdown_connections 2023-11-24T08:13:23.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:23.692+0000 7f349dfba640 1 -- 172.21.15.106:0/901498711 wait complete. 2023-11-24T08:13:23.712 INFO:tasks.cephfs_test_runner:test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) ... ERROR 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner: 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 4530, in test_subvolume_inherited_snapshot_ls 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner: 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:13:23.713 INFO:tasks.cephfs_test_runner:Ran 10 tests in 238.359s 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner: 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner:FAILED (errors=1, skipped=5) 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner: 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner:ERROR: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs/test_volumes.py", line 4530, in test_subvolume_inherited_snapshot_ls 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner: snapshots = self._generate_random_snapshot_name(snap_count) 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner:AttributeError: 'TestSubvolumeSnapshots' object has no attribute '_generate_random_snapshot_name' 2023-11-24T08:13:23.714 INFO:tasks.cephfs_test_runner: 2023-11-24T08:13:23.715 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:13:23.836 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=5b9b261e36a94fc79d61d4275dd63228 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_2442239d2653456406c25ae0c71b689c8f2657b6/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_rishabh-d-dave_ceph_73843baf45f8a92d8e3fcbea3f024c97e73036da/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots) 2023-11-24T08:13:23.837 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-24T08:13:23.847 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-24T08:13:23.880 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-24T08:13:23.880 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:13:23.880 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:23.950 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:13:23.950 DEBUG:teuthology.orchestra.run.smithi106:> set -ex 2023-11-24T08:13:23.950 DEBUG:teuthology.orchestra.run.smithi106:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:24.002 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054299,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249408k,nr_inodes=812352,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498824k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-11-24T08:13:24.002 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:13:24.003 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:24.065 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:13:24.065 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-11-24T08:13:24.065 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-24T08:13:24.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054301,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249412k,nr_inodes=812353,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498828k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-11-24T08:13:24.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.119 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:13:24.178 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-24T08:13:24.179 INFO:teuthology.orchestra.run.smithi106.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-24T08:13:24.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-24T08:13:24.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-24T08:13:24.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.359 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2023-11-24T08:13:24.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.485 DEBUG:teuthology.orchestra.run.smithi106:> ip netns list 2023-11-24T08:13:24.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.530 DEBUG:teuthology.orchestra.run.smithi106:> sudo ip link delete ceph-brx 2023-11-24T08:13:24.665 INFO:teuthology.orchestra.run.smithi106.stderr:Cannot find device "ceph-brx" 2023-11-24T08:13:24.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:13:24.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.669 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:13:24.728 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-24T08:13:24.728 INFO:teuthology.orchestra.run.smithi156.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-24T08:13:24.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.729 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-24T08:13:24.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.824 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-24T08:13:24.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:24.918 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-24T08:13:25.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:25.071 DEBUG:teuthology.orchestra.run.smithi156:> ip netns list 2023-11-24T08:13:25.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-24T08:13:25.119 DEBUG:teuthology.orchestra.run.smithi156:> sudo ip link delete ceph-brx 2023-11-24T08:13:25.262 INFO:teuthology.orchestra.run.smithi156.stderr:Cannot find device "ceph-brx" 2023-11-24T08:13:25.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:13:25.266 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-24T08:13:25.276 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-24T08:13:25.277 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.499+0000 7f2273a51640 1 Processor -- start 2023-11-24T08:13:25.501 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.499+0000 7f2273a51640 1 -- start start 2023-11-24T08:13:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2273a51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.502 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2273a51640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f226c15dc80 0x7f226c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2273a51640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0ce0b0 con 0x7f226c157c70 2023-11-24T08:13:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2273a51640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c161960 con 0x7f226c15dc80 2023-11-24T08:13:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2273a51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c15e670 con 0x7f226c0cfb60 2023-11-24T08:13:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2271fee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f226c15dc80 0x7f226c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2271fee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f226c15dc80 0x7f226c15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59812/0 (socket says 172.21.15.106:59812) 2023-11-24T08:13:25.503 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2271fee640 1 -- 172.21.15.106:0/2250865653 learned_addr learned my addr 172.21.15.106:0/2250865653 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.501+0000 7f2270fec640 1 --2- 172.21.15.106:0/2250865653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.502+0000 7f2271fee640 1 -- 172.21.15.106:0/2250865653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 msgr2=0x7f226c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:25.506 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.502+0000 7f2271fee640 1 --2- 172.21.15.106:0/2250865653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.502+0000 7f2271fee640 1 -- 172.21.15.106:0/2250865653 >> v1:172.21.15.156:6789/0 conn(0x7f226c157c70 legacy=0x7f226c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.502+0000 7f2271fee640 1 -- 172.21.15.106:0/2250865653 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c15e8d0 con 0x7f226c15dc80 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.503+0000 7f2271fee640 1 --2- 172.21.15.106:0/2250865653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15e130 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2264009e50 tx=0x7f226402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8260277be401ed79 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2263fff640 1 -- 172.21.15.106:0/2250865653 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226402ca70 con 0x7f226c15dc80 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2263fff640 1 -- 172.21.15.106:0/2250865653 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226402cbd0 con 0x7f226c15dc80 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2263fff640 1 -- 172.21.15.106:0/2250865653 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22640343f0 con 0x7f226c15dc80 2023-11-24T08:13:25.507 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2273a51640 1 -- 172.21.15.106:0/2250865653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 msgr2=0x7f226c15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2273a51640 1 --2- 172.21.15.106:0/2250865653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15e130 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f2264009e50 tx=0x7f226402bed0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2273a51640 1 -- 172.21.15.106:0/2250865653 shutdown_connections 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2273a51640 1 --2- 172.21.15.106:0/2250865653 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15e130 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2273a51640 1 --2- 172.21.15.106:0/2250865653 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.504+0000 7f2273a51640 1 -- 172.21.15.106:0/2250865653 >> 172.21.15.106:0/2250865653 conn(0x7f226c06a920 msgr2=0x7f226c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.505+0000 7f2273a51640 1 -- 172.21.15.106:0/2250865653 shutdown_connections 2023-11-24T08:13:25.508 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.505+0000 7f2273a51640 1 -- 172.21.15.106:0/2250865653 wait complete. 2023-11-24T08:13:25.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.507+0000 7f2273a51640 1 Processor -- start 2023-11-24T08:13:25.509 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.507+0000 7f2273a51640 1 -- start start 2023-11-24T08:13:25.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2273a51640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c14ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.511 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2273a51640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c157c70 0x7f226c1f6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2273a51640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2273a51640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0ad540 con 0x7f226c15dc80 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2273a51640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c14cb20 con 0x7f226c157c70 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2273a51640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c14df70 con 0x7f226c0cfb60 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2271fee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2271fee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15f160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59814/0 (socket says 172.21.15.106:59814) 2023-11-24T08:13:25.512 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.508+0000 7f2271fee640 1 -- 172.21.15.106:0/1473137053 learned_addr learned my addr 172.21.15.106:0/1473137053 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:25.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2270fec640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c157c70 0x7f226c1f6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f22717ed640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c14ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.513 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2271fee640 1 -- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 msgr2=0x7f226c14ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2271fee640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c14ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2271fee640 1 -- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c157c70 msgr2=0x7f226c1f6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2271fee640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c157c70 0x7f226c1f6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2271fee640 1 -- 172.21.15.106:0/1473137053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2264009b30 con 0x7f226c15dc80 2023-11-24T08:13:25.514 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f22717ed640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:25.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.509+0000 7f2271fee640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15f160 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2264009e50 tx=0x7f2264004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:25.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.510+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264040670 con 0x7f226c15dc80 2023-11-24T08:13:25.515 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.510+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22640407d0 con 0x7f226c15dc80 2023-11-24T08:13:25.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.510+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226c0acba0 con 0x7f226c15dc80 2023-11-24T08:13:25.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.510+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264040b80 con 0x7f226c15dc80 2023-11-24T08:13:25.516 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.510+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c0ad160 con 0x7f226c15dc80 2023-11-24T08:13:25.517 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.511+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c067470 con 0x7f226c15dc80 2023-11-24T08:13:25.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.512+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2264040ce0 con 0x7f226c15dc80 2023-11-24T08:13:25.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.513+0000 7f2261ffb640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f225006b470 0x7f225006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.513+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f22640acd10 con 0x7f226c15dc80 2023-11-24T08:13:25.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.513+0000 7f22717ed640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f225006b470 0x7f225006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.518 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.514+0000 7f22717ed640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f225006b470 0x7f225006d930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f22680015e0 tx=0x7f2268009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:25.519 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.517+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2264077930 con 0x7f226c15dc80 2023-11-24T08:13:25.709 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.706+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f226c161090 con 0x7f225006b470 2023-11-24T08:13:25.713 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.712+0000 7f2261ffb640 1 -- 172.21.15.106:0/1473137053 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48730 (secure 0 0 0) 0x7f226c161090 con 0x7f225006b470 2023-11-24T08:13:25.714 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:25.715 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:25.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.714+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f225006b470 msgr2=0x7f225006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.714+0000 7f2273a51640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f225006b470 0x7f225006d930 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f22680015e0 tx=0x7f2268009040 comp rx=0 tx=0).stop 2023-11-24T08:13:25.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.714+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 msgr2=0x7f226c15f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.714+0000 7f2273a51640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15f160 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f2264009e50 tx=0x7f2264004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:25.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 shutdown_connections 2023-11-24T08:13:25.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f226c15dc80 0x7f226c15f160 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c157c70 0x7f226c1f6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f225006b470 0x7f225006d930 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 --2- 172.21.15.106:0/1473137053 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f226c0cfb60 0x7f226c14ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 >> 172.21.15.106:0/1473137053 conn(0x7f226c06a920 msgr2=0x7f226c0b0060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:25.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 shutdown_connections 2023-11-24T08:13:25.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.715+0000 7f2273a51640 1 -- 172.21.15.106:0/1473137053 wait complete. 2023-11-24T08:13:25.736 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2023-11-24T08:13:25.359744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259280,"kb_used_data":9844,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715440,"statfs":{"total":773094113280,"available":772828610560,"internally_reserved":0,"allocated":10080256,"data_stored":5005979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28337,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001992"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480888+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:58.480888+0000","last_peered":"2023-11-24T08:12:58.480888+0000","last_clean":"2023-11-24T08:12:58.480888+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:58.480888+0000","last_undegraded":"2023-11-24T08:12:58.480888+0000","last_fullsized":"2023-11-24T08:12:58.480888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1687552,"data_stored":1057347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.196,"5min":1.073,"15min":1.0129999999999999},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8580000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.75800000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.196,"5min":1.181,"15min":1.1679999999999999},"min":{"1min":0.90500000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7150000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4490000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.135,"15min":1.073},"min":{"1min":0.77300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.821,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":0.89700000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.954,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.208}]},{"osd":3,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.234},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5860000000000001,"5min":3.387,"15min":3.387},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.276,"15min":1.23},"min":{"1min":0.98699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.865,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.2210000000000001},"min":{"1min":0.93799999999999994,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9259999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.3939999999999999,"15min":1.377},"min":{"1min":0.97999999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.909,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.627}]},{"osd":5,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":0.91800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7909999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.8169999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3959999999999999,"15min":1.3640000000000001},"min":{"1min":1.081,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.984,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.542,"15min":1.4950000000000001},"min":{"1min":1.4510000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.073,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.75},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7909999999999999,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.7809999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542203,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":467603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4099999999999999,"5min":2.085,"15min":2.085},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.236,"5min":1.2110000000000001,"15min":1.236},"min":{"1min":0.745,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.96899999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.629,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.212},{"interface":"front","average":{"1min":1.048,"5min":1.145,"15min":1.163},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.593,"5min":1.98,"15min":1.98},"last":0.86499999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4199999999999999,"5min":1.778,"15min":1.778},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6659999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.335},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7250000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.395,"15min":1.4330000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3160000000000001,"15min":1.3600000000000001},"min":{"1min":0.71499999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.7709999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.45},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.581},"min":{"1min":0.84499999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1269999999999998,"5min":2.661,"15min":2.661},"last":2.004}]},{"osd":5,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.6890000000000001},"min":{"1min":1.0900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0049999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":2.181},{"interface":"front","average":{"1min":1.419,"5min":1.538,"15min":1.6339999999999999},"min":{"1min":1.0580000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.752,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6930000000000001,"15min":1.7689999999999999},"min":{"1min":1.3460000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.2370000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":2.3239999999999998},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.6759999999999999,"15min":1.7689999999999999},"min":{"1min":1.1439999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.629}]}]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1138688,"data_stored":501862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":0.82699999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.123}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.248},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.032},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.198}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1105920,"data_stored":473125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0369999999999999,"15min":0.996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6180000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.226},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.74199999999999999,"5min":0.626,"15min":0.626},"max":{"1min":1.7909999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.274}]},{"osd":1,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.157,"15min":1.155},"min":{"1min":0.90300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2090000000000001,"15min":1.1699999999999999},"min":{"1min":0.81200000000000006,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6559999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.88900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.746,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.175},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.79900000000000004,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3180000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2869999999999999},"min":{"1min":0.748,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.399,"15min":1.415},"min":{"1min":0.96199999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.901,"5min":2.746,"15min":2.746},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2889999999999999,"15min":1.3149999999999999},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.919,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4690000000000001,"15min":1.4630000000000001},"min":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.411,"15min":1.4410000000000001},"min":{"1min":0.83999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.101,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.89200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.411}]},{"osd":7,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.526,"15min":1.516},"min":{"1min":1.0820000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.0059999999999998,"5min":2.548,"15min":2.548},"last":1.024},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.5070000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.506}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1134592,"data_stored":501840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97799999999999998,"15min":1.1220000000000001},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4139999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.56100000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.86399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":0.98299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.429,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0329999999999999,"15min":1.103},"min":{"1min":0.53300000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5740000000000001,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.107,"15min":1.2030000000000001},"min":{"1min":0.57099999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.544,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.1140000000000001,"15min":1.145},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.5029999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3149999999999999,"15min":1.4350000000000001},"min":{"1min":0.72399999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.137},{"interface":"front","average":{"1min":1.202,"5min":1.2989999999999999,"15min":1.361},"min":{"1min":0.88,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6719999999999999,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4470000000000001,"15min":1.548},"min":{"1min":0.92700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.829,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.468},{"interface":"front","average":{"1min":1.351,"5min":1.444,"15min":1.522},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7410000000000001,"5min":3.339,"15min":3.339},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4750000000000001,"15min":1.637},"min":{"1min":0.89000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8440000000000001,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.341},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4850000000000001,"15min":1.5449999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0030000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.651}]},{"osd":7,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5449999999999999,"15min":1.655},"min":{"1min":1.077,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.105,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.6839999999999999,"15min":1.966},"min":{"1min":1.01,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0470000000000002,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.5069999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1093632,"data_stored":467491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.155}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.995},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.256}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":1.2110000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.3979999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542203,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":473184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.476,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2809999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0109999999999999},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1519999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6279999999999999,"5min":1.907,"15min":1.907},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.458,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.167,"15min":1.165},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.325},"min":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.248,"15min":1.296},"min":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.661,"15min":2.661},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6819999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.411}]},{"osd":6,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.321,"15min":1.349},"min":{"1min":0.78900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.633,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.669},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3400000000000001,"15min":1.3340000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.472}]},{"osd":7,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3999999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.351,"15min":1.363},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.008,"15min":2.008},"last":1.738}]}]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":1063527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5069999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.149}]},{"osd":2,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.48399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":1.0680000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.05,"15min":1.0720000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4450000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.51},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88800000000000001,"15min":0.95199999999999996},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.591,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.29},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5880000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1950000000000001,"15min":1.218},"min":{"1min":0.85799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.506,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.2529999999999999,"15min":1.2669999999999999},"min":{"1min":0.79600000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.722,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2490000000000001,"15min":1.266},"min":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.512,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.268},{"interface":"front","average":{"1min":1.294,"5min":1.2849999999999999,"15min":1.2869999999999999},"min":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9039999999999999,"5min":2.149,"15min":2.149},"last":1.46}]},{"osd":7,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.879,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7809999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6890000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.579}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:25.737 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:25.945 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.944+0000 7fa8f74da640 1 Processor -- start 2023-11-24T08:13:25.946 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.944+0000 7fa8f74da640 1 -- start start 2023-11-24T08:13:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.945+0000 7fa8f74da640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8f00b2250 0x7fa8f015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.945+0000 7fa8f74da640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f015ab80 con 0x7fa8f00b2630 2023-11-24T08:13:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.945+0000 7fa8f74da640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f015e680 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.947 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.945+0000 7fa8f74da640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f015a990 con 0x7fa8f00b2250 2023-11-24T08:13:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8f5276640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fa8f00b2250 0x7fa8f015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8f4a75640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fa8f00b2630 0x7fa8f00ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58896/0 (socket says 172.21.15.106:58896) 2023-11-24T08:13:25.948 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8f4a75640 1 -- 172.21.15.106:0/3751451498 learned_addr learned my addr 172.21.15.106:0/3751451498 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8f5276640 1 -- 172.21.15.106:0/3751451498 >> v1:172.21.15.156:6789/0 conn(0x7fa8f00b2aa0 legacy=0x7fa8f015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8f5276640 1 -- 172.21.15.106:0/3751451498 >> v1:172.21.15.106:6789/0 conn(0x7fa8f00b2630 legacy=0x7fa8f00ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8f5276640 1 -- 172.21.15.106:0/3751451498 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f015e960 con 0x7fa8f00b2250 2023-11-24T08:13:25.949 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.946+0000 7fa8e7fff640 1 -- 172.21.15.106:0/3751451498 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 264681788 0 0) 0x7fa8f015ab80 con 0x7fa8f00b2630 2023-11-24T08:13:25.950 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.947+0000 7fa8f5276640 1 --2- 172.21.15.106:0/3751451498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2250 0x7fa8f015df80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa8f015ad70 tx=0x7fa8ec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70a47a1eba407e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:25.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.947+0000 7fa8e7fff640 1 -- 172.21.15.106:0/3751451498 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec031830 con 0x7fa8f00b2250 2023-11-24T08:13:25.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8e7fff640 1 -- 172.21.15.106:0/3751451498 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ec031990 con 0x7fa8f00b2250 2023-11-24T08:13:25.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8e7fff640 1 -- 172.21.15.106:0/3751451498 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec031d20 con 0x7fa8f00b2250 2023-11-24T08:13:25.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 -- 172.21.15.106:0/3751451498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2250 msgr2=0x7fa8f015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.951 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3751451498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2250 0x7fa8f015df80 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7fa8f015ad70 tx=0x7fa8ec02b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 -- 172.21.15.106:0/3751451498 shutdown_connections 2023-11-24T08:13:25.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3751451498 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2250 0x7fa8f015df80 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 -- 172.21.15.106:0/3751451498 >> 172.21.15.106:0/3751451498 conn(0x7fa8f006a920 msgr2=0x7fa8f006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:25.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 -- 172.21.15.106:0/3751451498 shutdown_connections 2023-11-24T08:13:25.952 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.948+0000 7fa8f74da640 1 -- 172.21.15.106:0/3751451498 wait complete. 2023-11-24T08:13:25.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.950+0000 7fa8f74da640 1 Processor -- start 2023-11-24T08:13:25.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.950+0000 7fa8f74da640 1 -- start start 2023-11-24T08:13:25.954 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f74da640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f00b2250 0x7fa8f01473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f74da640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f00b2630 0x7fa8f0155af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f74da640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 0x7fa8f00c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f74da640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01556d0 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f74da640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0147920 con 0x7fa8f00b2250 2023-11-24T08:13:25.955 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f74da640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0155050 con 0x7fa8f00b2630 2023-11-24T08:13:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.951+0000 7fa8f5a77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 0x7fa8f00c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 0x7fa8f00c8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59826/0 (socket says 172.21.15.106:59826) 2023-11-24T08:13:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 -- 172.21.15.106:0/3293534490 learned_addr learned my addr 172.21.15.106:0/3293534490 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 -- 172.21.15.106:0/3293534490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f00b2630 msgr2=0x7fa8f0155af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f00b2630 0x7fa8f0155af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.956 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 -- 172.21.15.106:0/3293534490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f00b2250 msgr2=0x7fa8f01473e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:25.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f00b2250 0x7fa8f01473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:25.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 -- 172.21.15.106:0/3293534490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ec009400 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f5a77640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 0x7fa8f00c8360 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b9e0 tx=0x7fa8e800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:25.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8003b70 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.957 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e8003cd0 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e800cae0 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.952+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f00cd7c0 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.958 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.953+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f00c9100 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.954+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f0067470 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.955+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fa8e800cc40 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.955+0000 7fa8e5ffb640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8d406b4a0 0x7fa8d406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:25.959 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.956+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa8e8089bc0 con 0x7fa8f00b2aa0 2023-11-24T08:13:25.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.956+0000 7fa8f5276640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8d406b4a0 0x7fa8d406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:25.960 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.956+0000 7fa8f5276640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8d406b4a0 0x7fa8d406d960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec036000 tx=0x7fa8ec030000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:25.961 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:25.960+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fa8e8054560 con 0x7fa8f00b2aa0 2023-11-24T08:13:26.150 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.148+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa8f00cb430 con 0x7fa8d406b4a0 2023-11-24T08:13:26.155 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.154+0000 7fa8e5ffb640 1 -- 172.21.15.106:0/3293534490 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48730 (secure 0 0 0) 0x7fa8f00cb430 con 0x7fa8d406b4a0 2023-11-24T08:13:26.156 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:26.156 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:26.159 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8d406b4a0 msgr2=0x7fa8d406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8d406b4a0 0x7fa8d406d960 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec036000 tx=0x7fa8ec030000 comp rx=0 tx=0).stop 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 msgr2=0x7fa8f00c8360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 0x7fa8f00c8360 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b9e0 tx=0x7fa8e800bea0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 shutdown_connections 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fa8f00b2aa0 0x7fa8f00c8360 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fa8d406b4a0 0x7fa8d406d960 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fa8f00b2630 0x7fa8f0155af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.160 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 --2- 172.21.15.106:0/3293534490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8f00b2250 0x7fa8f01473e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 >> 172.21.15.106:0/3293534490 conn(0x7fa8f006a920 msgr2=0x7fa8f00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:26.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 shutdown_connections 2023-11-24T08:13:26.161 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.157+0000 7fa8f74da640 1 -- 172.21.15.106:0/3293534490 wait complete. 2023-11-24T08:13:26.177 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2023-11-24T08:13:25.359744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259280,"kb_used_data":9844,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715440,"statfs":{"total":773094113280,"available":772828610560,"internally_reserved":0,"allocated":10080256,"data_stored":5005979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28337,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001992"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480888+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:58.480888+0000","last_peered":"2023-11-24T08:12:58.480888+0000","last_clean":"2023-11-24T08:12:58.480888+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:58.480888+0000","last_undegraded":"2023-11-24T08:12:58.480888+0000","last_fullsized":"2023-11-24T08:12:58.480888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1687552,"data_stored":1057347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.196,"5min":1.073,"15min":1.0129999999999999},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8580000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.75800000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.196,"5min":1.181,"15min":1.1679999999999999},"min":{"1min":0.90500000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7150000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4490000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.135,"15min":1.073},"min":{"1min":0.77300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.821,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":0.89700000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.954,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.208}]},{"osd":3,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.234},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5860000000000001,"5min":3.387,"15min":3.387},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.276,"15min":1.23},"min":{"1min":0.98699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.865,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.2210000000000001},"min":{"1min":0.93799999999999994,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9259999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.3939999999999999,"15min":1.377},"min":{"1min":0.97999999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.909,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.627}]},{"osd":5,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":0.91800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7909999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.8169999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3959999999999999,"15min":1.3640000000000001},"min":{"1min":1.081,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.984,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.542,"15min":1.4950000000000001},"min":{"1min":1.4510000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.073,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.75},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7909999999999999,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.7809999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542203,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":467603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4099999999999999,"5min":2.085,"15min":2.085},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.236,"5min":1.2110000000000001,"15min":1.236},"min":{"1min":0.745,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.96899999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.629,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.212},{"interface":"front","average":{"1min":1.048,"5min":1.145,"15min":1.163},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.593,"5min":1.98,"15min":1.98},"last":0.86499999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4199999999999999,"5min":1.778,"15min":1.778},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6659999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.335},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7250000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.395,"15min":1.4330000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3160000000000001,"15min":1.3600000000000001},"min":{"1min":0.71499999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.7709999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.45},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.581},"min":{"1min":0.84499999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1269999999999998,"5min":2.661,"15min":2.661},"last":2.004}]},{"osd":5,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.6890000000000001},"min":{"1min":1.0900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0049999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":2.181},{"interface":"front","average":{"1min":1.419,"5min":1.538,"15min":1.6339999999999999},"min":{"1min":1.0580000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.752,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6930000000000001,"15min":1.7689999999999999},"min":{"1min":1.3460000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.2370000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":2.3239999999999998},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.6759999999999999,"15min":1.7689999999999999},"min":{"1min":1.1439999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.629}]}]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1138688,"data_stored":501862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":0.82699999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.123}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.248},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.032},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.198}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1105920,"data_stored":473125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0369999999999999,"15min":0.996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6180000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.226},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.74199999999999999,"5min":0.626,"15min":0.626},"max":{"1min":1.7909999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.274}]},{"osd":1,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.157,"15min":1.155},"min":{"1min":0.90300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2090000000000001,"15min":1.1699999999999999},"min":{"1min":0.81200000000000006,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6559999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.88900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.746,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.175},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.79900000000000004,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3180000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2869999999999999},"min":{"1min":0.748,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.399,"15min":1.415},"min":{"1min":0.96199999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.901,"5min":2.746,"15min":2.746},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2889999999999999,"15min":1.3149999999999999},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.919,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4690000000000001,"15min":1.4630000000000001},"min":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.411,"15min":1.4410000000000001},"min":{"1min":0.83999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.101,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.89200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.411}]},{"osd":7,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.526,"15min":1.516},"min":{"1min":1.0820000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.0059999999999998,"5min":2.548,"15min":2.548},"last":1.024},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.5070000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.506}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1134592,"data_stored":501840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97799999999999998,"15min":1.1220000000000001},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4139999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.56100000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.86399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":0.98299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.429,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0329999999999999,"15min":1.103},"min":{"1min":0.53300000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5740000000000001,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.107,"15min":1.2030000000000001},"min":{"1min":0.57099999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.544,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.1140000000000001,"15min":1.145},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.5029999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3149999999999999,"15min":1.4350000000000001},"min":{"1min":0.72399999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.137},{"interface":"front","average":{"1min":1.202,"5min":1.2989999999999999,"15min":1.361},"min":{"1min":0.88,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6719999999999999,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4470000000000001,"15min":1.548},"min":{"1min":0.92700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.829,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.468},{"interface":"front","average":{"1min":1.351,"5min":1.444,"15min":1.522},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7410000000000001,"5min":3.339,"15min":3.339},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4750000000000001,"15min":1.637},"min":{"1min":0.89000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8440000000000001,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.341},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4850000000000001,"15min":1.5449999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0030000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.651}]},{"osd":7,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5449999999999999,"15min":1.655},"min":{"1min":1.077,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.105,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.6839999999999999,"15min":1.966},"min":{"1min":1.01,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0470000000000002,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.5069999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1093632,"data_stored":467491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.155}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.995},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.256}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":1.2110000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.3979999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542203,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":473184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.476,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2809999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0109999999999999},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1519999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6279999999999999,"5min":1.907,"15min":1.907},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.458,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.167,"15min":1.165},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.325},"min":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.248,"15min":1.296},"min":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.661,"15min":2.661},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6819999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.411}]},{"osd":6,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.321,"15min":1.349},"min":{"1min":0.78900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.633,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.669},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3400000000000001,"15min":1.3340000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.472}]},{"osd":7,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3999999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.351,"15min":1.363},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.008,"15min":2.008},"last":1.738}]}]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":1063527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5069999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.149}]},{"osd":2,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.48399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":1.0680000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.05,"15min":1.0720000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4450000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.51},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88800000000000001,"15min":0.95199999999999996},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.591,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.29},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5880000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1950000000000001,"15min":1.218},"min":{"1min":0.85799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.506,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.2529999999999999,"15min":1.2669999999999999},"min":{"1min":0.79600000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.722,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2490000000000001,"15min":1.266},"min":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.512,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.268},{"interface":"front","average":{"1min":1.294,"5min":1.2849999999999999,"15min":1.2869999999999999},"min":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9039999999999999,"5min":2.149,"15min":2.149},"last":1.46}]},{"osd":7,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.879,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7809999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6890000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.579}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:26.180 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-24T08:13:26.181 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:26.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.387+0000 7f6eef0ee640 1 Processor -- start 2023-11-24T08:13:26.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.387+0000 7f6eef0ee640 1 -- start start 2023-11-24T08:13:26.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eef0ee640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ee815c7f0 0x7f6ee806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eef0ee640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6ee80cf450 0x7f6ee80cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eef0ee640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80cece0 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eef0ee640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80cfe40 con 0x7f6ee815c7f0 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eef0ee640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee80ceaf0 con 0x7f6ee80cf450 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eece8a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ee815c7f0 0x7f6ee806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eece8a640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6ee815c7f0 0x7f6ee806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59836/0 (socket says 172.21.15.106:59836) 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6eece8a640 1 -- 172.21.15.106:0/3219405362 learned_addr learned my addr 172.21.15.106:0/3219405362 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.389+0000 7f6ee7fff640 1 --2- 172.21.15.106:0/3219405362 >> v2:172.21.15.156:3300/0 conn(0x7f6ee80cf450 0x7f6ee80cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.390+0000 7f6eece8a640 1 -- 172.21.15.106:0/3219405362 >> v2:172.21.15.156:3300/0 conn(0x7f6ee80cf450 msgr2=0x7f6ee80cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:26.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.390+0000 7f6eece8a640 1 --2- 172.21.15.106:0/3219405362 >> v2:172.21.15.156:3300/0 conn(0x7f6ee80cf450 0x7f6ee80cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.390+0000 7f6eece8a640 1 -- 172.21.15.106:0/3219405362 >> v1:172.21.15.156:6789/0 conn(0x7f6ee80cdfb0 legacy=0x7f6ee80ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:26.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.390+0000 7f6eece8a640 1 -- 172.21.15.106:0/3219405362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee806d040 con 0x7f6ee815c7f0 2023-11-24T08:13:26.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.391+0000 7f6eece8a640 1 --2- 172.21.15.106:0/3219405362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee815c7f0 0x7f6ee806db20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8070410 tx=0x7f6edc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc5e6b574460065 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:26.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6ee77fe640 1 -- 172.21.15.106:0/3219405362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc031870 con 0x7f6ee815c7f0 2023-11-24T08:13:26.394 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6ee77fe640 1 -- 172.21.15.106:0/3219405362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc0319d0 con 0x7f6ee815c7f0 2023-11-24T08:13:26.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6ee77fe640 1 -- 172.21.15.106:0/3219405362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc031d60 con 0x7f6ee815c7f0 2023-11-24T08:13:26.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3219405362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee815c7f0 msgr2=0x7f6ee806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3219405362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee815c7f0 0x7f6ee806db20 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6ee8070410 tx=0x7f6edc02b910 comp rx=0 tx=0).stop 2023-11-24T08:13:26.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3219405362 shutdown_connections 2023-11-24T08:13:26.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3219405362 >> v2:172.21.15.156:3300/0 conn(0x7f6ee80cf450 0x7f6ee80cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3219405362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee815c7f0 0x7f6ee806db20 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3219405362 >> 172.21.15.106:0/3219405362 conn(0x7f6ee806a920 msgr2=0x7f6ee806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:26.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3219405362 shutdown_connections 2023-11-24T08:13:26.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.392+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3219405362 wait complete. 2023-11-24T08:13:26.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.394+0000 7f6eef0ee640 1 Processor -- start 2023-11-24T08:13:26.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.394+0000 7f6eef0ee640 1 -- start start 2023-11-24T08:13:26.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.395+0000 7f6eef0ee640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 0x7f6ee80ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.395+0000 7f6eef0ee640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80cf450 0x7f6ee8155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.395+0000 7f6eef0ee640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815c7f0 0x7f6ee814d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.395+0000 7f6eef0ee640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8157c10 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.395+0000 7f6eef0ee640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80c8da0 con 0x7f6ee815c7f0 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.395+0000 7f6eef0ee640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee8156fd0 con 0x7f6ee80cf450 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 0x7f6ee80ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 0x7f6ee80ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59844/0 (socket says 172.21.15.106:59844) 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 -- 172.21.15.106:0/3309681026 learned_addr learned my addr 172.21.15.106:0/3309681026 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eed68b640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815c7f0 0x7f6ee814d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6ee7fff640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80cf450 0x7f6ee8155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.399 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 -- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80cf450 msgr2=0x7f6ee8155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80cf450 0x7f6ee8155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 -- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815c7f0 msgr2=0x7f6ee814d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815c7f0 0x7f6ee814d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 -- 172.21.15.106:0/3309681026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0094d0 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.400 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.396+0000 7f6eece8a640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 0x7f6ee80ae190 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6edc02b950 tx=0x7f6edc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:26.401 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.397+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc03abe0 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.397+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6edc03ad40 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.397+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc044610 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.397+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee8156450 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.397+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee8156a10 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.398+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee80c8f30 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.399+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6edc044770 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.400+0000 7f6ee57fa640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6ecc06b470 0x7f6ecc06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.400+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6edc02d040 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.400+0000 7f6ee7fff640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6ecc06b470 0x7f6ecc06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.403 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.401+0000 7f6ee7fff640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6ecc06b470 0x7f6ecc06d930 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6ee81f6c70 tx=0x7f6ed4008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:26.405 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.404+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6ee80c8f30 con 0x7f6ee80cdfb0 2023-11-24T08:13:26.596 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.594+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ee806e060 con 0x7f6ecc06b470 2023-11-24T08:13:26.602 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.600+0000 7f6ee57fa640 1 -- 172.21.15.106:0/3309681026 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48730 (secure 0 0 0) 0x7f6ed4009d60 con 0x7f6ecc06b470 2023-11-24T08:13:26.602 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:26.603 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:26.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6ecc06b470 msgr2=0x7f6ecc06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.604 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6ecc06b470 0x7f6ecc06d930 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f6ee81f6c70 tx=0x7f6ed4008040 comp rx=0 tx=0).stop 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 msgr2=0x7f6ee80ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 0x7f6ee80ae190 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f6edc02b950 tx=0x7f6edc0046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 shutdown_connections 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee815c7f0 0x7f6ee814d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6ecc06b470 0x7f6ecc06d930 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6ee80cf450 0x7f6ee8155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.605 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 --2- 172.21.15.106:0/3309681026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6ee80cdfb0 0x7f6ee80ae190 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 >> 172.21.15.106:0/3309681026 conn(0x7f6ee806a920 msgr2=0x7f6ee80cc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:26.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 shutdown_connections 2023-11-24T08:13:26.606 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.603+0000 7f6eef0ee640 1 -- 172.21.15.106:0/3309681026 wait complete. 2023-11-24T08:13:26.623 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2023-11-24T08:13:25.359744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":259280,"kb_used_data":9844,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715440,"statfs":{"total":773094113280,"available":772828610560,"internally_reserved":0,"allocated":10080256,"data_stored":5005979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-28337,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-61,"num_write_kb":-49,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001992"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":219,"reported_epoch":89,"state":"active+clean","last_fresh":"2023-11-24T08:12:58.480888+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:12:58.480888+0000","last_peered":"2023-11-24T08:12:58.480888+0000","last_clean":"2023-11-24T08:12:58.480888+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:12:58.480888+0000","last_undegraded":"2023-11-24T08:12:58.480888+0000","last_fullsized":"2023-11-24T08:12:58.480888+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542204,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32828,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339012,"statfs":{"total":96636764160,"available":96603148288,"internally_reserved":0,"allocated":1687552,"data_stored":1057347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.196,"5min":1.073,"15min":1.0129999999999999},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8580000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.85899999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.75800000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.196,"5min":1.181,"15min":1.1679999999999999},"min":{"1min":0.90500000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7150000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.4490000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.135,"15min":1.073},"min":{"1min":0.77300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.821,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":0.89700000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.954,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.208}]},{"osd":3,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.234},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5860000000000001,"5min":3.387,"15min":3.387},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.276,"15min":1.23},"min":{"1min":0.98699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.865,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.325}]},{"osd":4,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.2210000000000001},"min":{"1min":0.93799999999999994,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9259999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.6000000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.3939999999999999,"15min":1.377},"min":{"1min":0.97999999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.909,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.627}]},{"osd":5,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":0.91800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7909999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.8169999999999999},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3959999999999999,"15min":1.3640000000000001},"min":{"1min":1.081,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.984,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.6879999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.542,"15min":1.4950000000000001},"min":{"1min":1.4510000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.073,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.75},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7909999999999999,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.7809999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542203,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32256,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339584,"statfs":{"total":96636764160,"available":96603734016,"internally_reserved":0,"allocated":1101824,"data_stored":467603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4099999999999999,"5min":2.085,"15min":2.085},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.236,"5min":1.2110000000000001,"15min":1.236},"min":{"1min":0.745,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.96899999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.629,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.212},{"interface":"front","average":{"1min":1.048,"5min":1.145,"15min":1.163},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.593,"5min":1.98,"15min":1.98},"last":0.86499999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4199999999999999,"5min":1.778,"15min":1.778},"last":1.1379999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6659999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":0.74199999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.335},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7250000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.395,"15min":1.4330000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2609999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3160000000000001,"15min":1.3600000000000001},"min":{"1min":0.71499999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.7709999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.45},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.581},"min":{"1min":0.84499999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1269999999999998,"5min":2.661,"15min":2.661},"last":2.004}]},{"osd":5,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.6890000000000001},"min":{"1min":1.0900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0049999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":2.181},{"interface":"front","average":{"1min":1.419,"5min":1.538,"15min":1.6339999999999999},"min":{"1min":1.0580000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.752,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.3939999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6930000000000001,"15min":1.7689999999999999},"min":{"1min":1.3460000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.2370000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":2.3239999999999998},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.6759999999999999,"15min":1.7689999999999999},"min":{"1min":1.1439999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.629}]}]},{"osd":5,"up_from":14,"seq":60129542202,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32288,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1138688,"data_stored":501862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":0.82699999999999996}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.77500000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.96999999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.123}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":0.68200000000000005}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.248},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":0.88900000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.032},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.198}]}]},{"osd":4,"up_from":14,"seq":60129542203,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339580,"statfs":{"total":96636764160,"available":96603729920,"internally_reserved":0,"allocated":1105920,"data_stored":473125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0369999999999999,"15min":0.996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6180000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.226},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.74199999999999999,"5min":0.626,"15min":0.626},"max":{"1min":1.7909999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.274}]},{"osd":1,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.157,"15min":1.155},"min":{"1min":0.90300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2090000000000001,"15min":1.1699999999999999},"min":{"1min":0.81200000000000006,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6559999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.89600000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.88900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.746,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.175},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.79900000000000004,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.3180000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2869999999999999},"min":{"1min":0.748,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.399,"15min":1.415},"min":{"1min":0.96199999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.901,"5min":2.746,"15min":2.746},"last":1.379}]},{"osd":5,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2889999999999999,"15min":1.3149999999999999},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.919,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4690000000000001,"15min":1.4630000000000001},"min":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.351}]},{"osd":6,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.411,"15min":1.4410000000000001},"min":{"1min":0.83999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.101,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.89200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.411}]},{"osd":7,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.526,"15min":1.516},"min":{"1min":1.0820000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.0059999999999998,"5min":2.548,"15min":2.548},"last":1.024},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.5070000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.506}]}]},{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32284,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339556,"statfs":{"total":96636764160,"available":96603705344,"internally_reserved":0,"allocated":1134592,"data_stored":501840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97799999999999998,"15min":1.1220000000000001},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4139999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.88,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.56100000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.86399999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":0.98299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.429,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":1.5660000000000001},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0329999999999999,"15min":1.103},"min":{"1min":0.53300000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5740000000000001,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.92100000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.107,"15min":1.2030000000000001},"min":{"1min":0.57099999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.544,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.1140000000000001,"15min":1.145},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.5029999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.72599999999999998}]},{"osd":4,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3149999999999999,"15min":1.4350000000000001},"min":{"1min":0.72399999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.137},{"interface":"front","average":{"1min":1.202,"5min":1.2989999999999999,"15min":1.361},"min":{"1min":0.88,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6719999999999999,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.2549999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4470000000000001,"15min":1.548},"min":{"1min":0.92700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.829,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.468},{"interface":"front","average":{"1min":1.351,"5min":1.444,"15min":1.522},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7410000000000001,"5min":3.339,"15min":3.339},"last":1.3080000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4750000000000001,"15min":1.637},"min":{"1min":0.89000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8440000000000001,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.341},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4850000000000001,"15min":1.5449999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0030000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.651}]},{"osd":7,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5449999999999999,"15min":1.655},"min":{"1min":1.077,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.105,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.6839999999999999,"15min":1.966},"min":{"1min":1.01,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0470000000000002,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.5069999999999999}]}]},{"osd":2,"up_from":14,"seq":60129542203,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339588,"statfs":{"total":96636764160,"available":96603738112,"internally_reserved":0,"allocated":1093632,"data_stored":467491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.93400000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1020000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":0.71599999999999997}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.155}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":0.995},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.256}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":1.2110000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.3979999999999999}]}]},{"osd":1,"up_from":14,"seq":60129542203,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32268,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339572,"statfs":{"total":96636764160,"available":96603721728,"internally_reserved":0,"allocated":1114112,"data_stored":473184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.476,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.80900000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2809999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.86199999999999999},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0109999999999999},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.0489999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1519999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6279999999999999,"5min":1.907,"15min":1.907},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.458,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.0920000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.167,"15min":1.165},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.325},"min":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.306}]},{"osd":5,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.248,"15min":1.296},"min":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.661,"15min":2.661},"last":1.7989999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6819999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.411}]},{"osd":6,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.321,"15min":1.349},"min":{"1min":0.78900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.633,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.669},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3400000000000001,"15min":1.3340000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.472}]},{"osd":7,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3999999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.1990000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.351,"15min":1.363},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.008,"15min":2.008},"last":1.738}]}]},{"osd":0,"up_from":14,"seq":60129542203,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32844,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338996,"statfs":{"total":96636764160,"available":96603131904,"internally_reserved":0,"allocated":1703936,"data_stored":1063527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.76600000000000001},{"interface":"front","average":{"1min":1.006,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5069999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.149}]},{"osd":2,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.48399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":1.0680000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.05,"15min":1.0720000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4450000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.51},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88800000000000001,"15min":0.95199999999999996},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.82399999999999995}]},{"osd":4,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.591,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.29},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5880000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.298}]},{"osd":5,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1950000000000001,"15min":1.218},"min":{"1min":0.85799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.506,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.2529999999999999,"15min":1.2669999999999999},"min":{"1min":0.79600000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.722,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.6739999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2490000000000001,"15min":1.266},"min":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.512,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.268},{"interface":"front","average":{"1min":1.294,"5min":1.2849999999999999,"15min":1.2869999999999999},"min":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9039999999999999,"5min":2.149,"15min":2.149},"last":1.46}]},{"osd":7,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.879,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7809999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.7410000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6890000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.579}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:26.625 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-24T08:13:26.838 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.837+0000 7f32384e5640 1 Processor -- start 2023-11-24T08:13:26.839 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.837+0000 7f32384e5640 1 -- start start 2023-11-24T08:13:26.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.838+0000 7f32384e5640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32300b2250 0x7f323015df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.838+0000 7f32384e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2aa0 0x7f323015b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.838+0000 7f32384e5640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f323015ab80 con 0x7f32300b2630 2023-11-24T08:13:26.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.838+0000 7f32384e5640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f323015a990 con 0x7f32300b2250 2023-11-24T08:13:26.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.838+0000 7f32384e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f323015e680 con 0x7f32300b2aa0 2023-11-24T08:13:26.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.838+0000 7f3236281640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f32300b2250 0x7f323015df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3235a80640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f32300b2630 0x7f32300ccf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58906/0 (socket says 172.21.15.106:58906) 2023-11-24T08:13:26.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3235a80640 1 -- 172.21.15.106:0/999939734 learned_addr learned my addr 172.21.15.106:0/999939734 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:26.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3236a82640 1 --2- 172.21.15.106:0/999939734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2aa0 0x7f323015b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3236281640 1 -- 172.21.15.106:0/999939734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2aa0 msgr2=0x7f323015b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:26.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3236281640 1 --2- 172.21.15.106:0/999939734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2aa0 0x7f323015b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3236281640 1 -- 172.21.15.106:0/999939734 >> v1:172.21.15.106:6789/0 conn(0x7f32300b2630 legacy=0x7f32300ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f3236281640 1 -- 172.21.15.106:0/999939734 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f323015e960 con 0x7f32300b2250 2023-11-24T08:13:26.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.839+0000 7f323527f640 1 -- 172.21.15.106:0/999939734 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3863995999 0 0) 0x7f323015ab80 con 0x7f32300b2630 2023-11-24T08:13:26.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.840+0000 7f3236281640 1 --2- 172.21.15.106:0/999939734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2250 0x7f323015df80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f322c0097a0 tx=0x7f322c02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394f7a313c8164e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:26.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.840+0000 7f323527f640 1 -- 172.21.15.106:0/999939734 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c02d040 con 0x7f32300b2250 2023-11-24T08:13:26.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.840+0000 7f323527f640 1 -- 172.21.15.106:0/999939734 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f322c031810 con 0x7f32300b2250 2023-11-24T08:13:26.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.840+0000 7f323527f640 1 -- 172.21.15.106:0/999939734 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322c031ba0 con 0x7f32300b2250 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 -- 172.21.15.106:0/999939734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2250 msgr2=0x7f323015df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 --2- 172.21.15.106:0/999939734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2250 0x7f323015df80 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f322c0097a0 tx=0x7f322c02b8b0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 -- 172.21.15.106:0/999939734 shutdown_connections 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 --2- 172.21.15.106:0/999939734 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2aa0 0x7f323015b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 --2- 172.21.15.106:0/999939734 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2250 0x7f323015df80 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 -- 172.21.15.106:0/999939734 >> 172.21.15.106:0/999939734 conn(0x7f323006a920 msgr2=0x7f323006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:26.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 -- 172.21.15.106:0/999939734 shutdown_connections 2023-11-24T08:13:26.845 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.841+0000 7f32384e5640 1 -- 172.21.15.106:0/999939734 wait complete. 2023-11-24T08:13:26.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.843+0000 7f32384e5640 1 Processor -- start 2023-11-24T08:13:26.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.843+0000 7f32384e5640 1 -- start start 2023-11-24T08:13:26.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f32384e5640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2250 0x7f32301539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f32384e5640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32300b2630 0x7f32300c8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f32384e5640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 0x7f32300c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.847 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f32384e5640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230157020 con 0x7f32300b2aa0 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f32384e5640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3230157300 con 0x7f32300b2630 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f32384e5640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32301575e0 con 0x7f32300b2250 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f3236a82640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 0x7f32300c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f3236a82640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 0x7f32300c8d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59864/0 (socket says 172.21.15.106:59864) 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.844+0000 7f3236a82640 1 -- 172.21.15.106:0/2901366139 learned_addr learned my addr 172.21.15.106:0/2901366139 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3235a80640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32300b2630 0x7f32300c8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236281640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2250 0x7f32301539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.848 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236a82640 1 -- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2250 msgr2=0x7f32301539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236a82640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2250 0x7f32301539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236a82640 1 -- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32300b2630 msgr2=0x7f32300c8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236a82640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32300b2630 0x7f32300c8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236a82640 1 -- 172.21.15.106:0/2901366139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c009400 con 0x7f32300b2aa0 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236281640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2250 0x7f32301539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3235a80640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32300b2630 0x7f32300c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3236a82640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 0x7f32300c8d00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f322800cc80 tx=0x7f322800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322800ce30 con 0x7f32300b2aa0 2023-11-24T08:13:26.849 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3228003990 con 0x7f32300b2aa0 2023-11-24T08:13:26.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3228003d20 con 0x7f32300b2aa0 2023-11-24T08:13:26.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.845+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32300cd7c0 con 0x7f32300b2aa0 2023-11-24T08:13:26.850 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.846+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3230153f30 con 0x7f32300b2aa0 2023-11-24T08:13:26.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.847+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3230067470 con 0x7f32300b2aa0 2023-11-24T08:13:26.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.848+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3228003e80 con 0x7f32300b2aa0 2023-11-24T08:13:26.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.848+0000 7f3226ffd640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f321406b470 0x7f321406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:26.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.848+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f322808a1d0 con 0x7f32300b2aa0 2023-11-24T08:13:26.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.849+0000 7f3236281640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f321406b470 0x7f321406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:26.851 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.849+0000 7f3236281640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f321406b470 0x7f321406d930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f322c036000 tx=0x7f322c008030 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:26.854 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:26.853+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f3228054bc0 con 0x7f32300b2aa0 2023-11-24T08:13:27.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.042+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f323015dac0 con 0x7f32300b2aa0 2023-11-24T08:13:27.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.045+0000 7f3226ffd640 1 -- 172.21.15.106:0/2901366139 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13037 (secure 0 0 0) 0x7f322801d090 con 0x7f32300b2aa0 2023-11-24T08:13:27.047 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:27.047 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":92,"fsid":"d1a897c2-15ce-4272-ad66-4b3723970cb1","created":"2023-11-24T08:08:28.262452+0000","modified":"2023-11-24T08:13:22.682580+0000","last_up_change":"2023-11-24T08:08:39.815404+0000","last_in_change":"2023-11-24T08:08:31.674429+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-11-24T08:08:41.348031+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"22cb81b7-10cd-4ba1-9b9c-6f5a45440014","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6808","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6809","nonce":3403388309}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6811","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6813","nonce":3403388309}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6822","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6825","nonce":3403388309}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6816","nonce":3403388309},{"type":"v1","addr":"172.21.15.106:6819","nonce":3403388309}]},"public_addr":"172.21.15.106:6809/3403388309","cluster_addr":"172.21.15.106:6813/3403388309","heartbeat_back_addr":"172.21.15.106:6825/3403388309","heartbeat_front_addr":"172.21.15.106:6819/3403388309","state":["exists","up"]},{"osd":1,"uuid":"6e9a4ad1-3a61-4b1f-ac3d-d2ef7a9019bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6800","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6801","nonce":1347023088}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6802","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6803","nonce":1347023088}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6806","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6807","nonce":1347023088}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6804","nonce":1347023088},{"type":"v1","addr":"172.21.15.106:6805","nonce":1347023088}]},"public_addr":"172.21.15.106:6801/1347023088","cluster_addr":"172.21.15.106:6803/1347023088","heartbeat_back_addr":"172.21.15.106:6807/1347023088","heartbeat_front_addr":"172.21.15.106:6805/1347023088","state":["exists","up"]},{"osd":2,"uuid":"3891f879-7b91-401a-83f2-fcf5bde64519","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6810","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6812","nonce":1671152390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6814","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6815","nonce":1671152390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6824","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6827","nonce":1671152390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6818","nonce":1671152390},{"type":"v1","addr":"172.21.15.106:6821","nonce":1671152390}]},"public_addr":"172.21.15.106:6812/1671152390","cluster_addr":"172.21.15.106:6815/1671152390","heartbeat_back_addr":"172.21.15.106:6827/1671152390","heartbeat_front_addr":"172.21.15.106:6821/1671152390","state":["exists","up"]},{"osd":3,"uuid":"e9297628-723d-459b-8077-39d9e0b212ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6817","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6820","nonce":906310218}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6823","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6826","nonce":906310218}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6830","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6831","nonce":906310218}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6828","nonce":906310218},{"type":"v1","addr":"172.21.15.106:6829","nonce":906310218}]},"public_addr":"172.21.15.106:6820/906310218","cluster_addr":"172.21.15.106:6826/906310218","heartbeat_back_addr":"172.21.15.106:6831/906310218","heartbeat_front_addr":"172.21.15.106:6829/906310218","state":["exists","up"]},{"osd":4,"uuid":"3fa594b0-7f7c-4bf3-9cec-88e41b45dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6801","nonce":2602399217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6803","nonce":2602399217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6807","nonce":2602399217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2602399217},{"type":"v1","addr":"172.21.15.156:6805","nonce":2602399217}]},"public_addr":"172.21.15.156:6801/2602399217","cluster_addr":"172.21.15.156:6803/2602399217","heartbeat_back_addr":"172.21.15.156:6807/2602399217","heartbeat_front_addr":"172.21.15.156:6805/2602399217","state":["exists","up"]},{"osd":5,"uuid":"6055d743-bcd9-4e65-af0b-5b99fae68c9d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6817","nonce":3158094073}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6819","nonce":3158094073}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6823","nonce":3158094073}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":3158094073},{"type":"v1","addr":"172.21.15.156:6821","nonce":3158094073}]},"public_addr":"172.21.15.156:6817/3158094073","cluster_addr":"172.21.15.156:6819/3158094073","heartbeat_back_addr":"172.21.15.156:6823/3158094073","heartbeat_front_addr":"172.21.15.156:6821/3158094073","state":["exists","up"]},{"osd":6,"uuid":"1516397b-f6b3-4f54-b7a2-c5e0de5811c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6809","nonce":2357036477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6811","nonce":2357036477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6815","nonce":2357036477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":2357036477},{"type":"v1","addr":"172.21.15.156:6813","nonce":2357036477}]},"public_addr":"172.21.15.156:6809/2357036477","cluster_addr":"172.21.15.156:6811/2357036477","heartbeat_back_addr":"172.21.15.156:6815/2357036477","heartbeat_front_addr":"172.21.15.156:6813/2357036477","state":["exists","up"]},{"osd":7,"uuid":"c8bb85e2-5999-4d43-9c54-e60aa7949caf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6825","nonce":1912715038}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6827","nonce":1912715038}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6831","nonce":1912715038}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":1912715038},{"type":"v1","addr":"172.21.15.156:6829","nonce":1912715038}]},"public_addr":"172.21.15.156:6825/1912715038","cluster_addr":"172.21.15.156:6827/1912715038","heartbeat_back_addr":"172.21.15.156:6831/1912715038","heartbeat_front_addr":"172.21.15.156:6829/1912715038","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.066156+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.212122+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.047365+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.994388+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.312434+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.307380+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:36.966495+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-24T08:08:37.053610+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.106:6837/1896010072":"2023-11-25T08:13:19.665250+0000","172.21.15.106:6835/1896010072":"2023-11-25T08:13:19.665250+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-24T08:13:27.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.047+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f321406b470 msgr2=0x7f321406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:27.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.047+0000 7f32384e5640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f321406b470 0x7f321406d930 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f322c036000 tx=0x7f322c008030 comp rx=0 tx=0).stop 2023-11-24T08:13:27.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.047+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 msgr2=0x7f32300c8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:27.049 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.047+0000 7f32384e5640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 0x7f32300c8d00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f322800cc80 tx=0x7f322800a5c0 comp rx=0 tx=0).stop 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 shutdown_connections 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32300b2aa0 0x7f32300c8d00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f32300b2630 0x7f32300c8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f321406b470 0x7f321406d930 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 --2- 172.21.15.106:0/2901366139 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32300b2250 0x7f32301539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 >> 172.21.15.106:0/2901366139 conn(0x7f323006a920 msgr2=0x7f32300ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 shutdown_connections 2023-11-24T08:13:27.050 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:27.048+0000 7f32384e5640 1 -- 172.21.15.106:0/2901366139 wait complete. 2023-11-24T08:13:28.069 INFO:tasks.ceph:Scrubbing osd.0 2023-11-24T08:13:28.070 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.299+0000 7fae4199e640 1 Processor -- start 2023-11-24T08:13:28.301 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.299+0000 7fae4199e640 1 -- start start 2023-11-24T08:13:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae4199e640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fae3c0cfb60 0x7fae3c161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.302 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae4199e640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fae3c15dc80 0x7fae3c15e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae4199e640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae3c0ce0b0 con 0x7fae3c157c70 2023-11-24T08:13:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae4199e640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c161960 con 0x7fae3c15dc80 2023-11-24T08:13:28.303 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae4199e640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae3c15e670 con 0x7fae3c0cfb60 2023-11-24T08:13:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae3b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fae3c15dc80 0x7fae3c15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae3b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fae3c15dc80 0x7fae3c15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59870/0 (socket says 172.21.15.106:59870) 2023-11-24T08:13:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.301+0000 7fae3b7fe640 1 -- 172.21.15.106:0/2086358805 learned_addr learned my addr 172.21.15.106:0/2086358805 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:28.304 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.302+0000 7fae3a7fc640 1 --2- 172.21.15.106:0/2086358805 >> v2:172.21.15.156:3300/0 conn(0x7fae3c0cfb60 0x7fae3c161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.302+0000 7fae3b7fe640 1 -- 172.21.15.106:0/2086358805 >> v2:172.21.15.156:3300/0 conn(0x7fae3c0cfb60 msgr2=0x7fae3c161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.302+0000 7fae3b7fe640 1 --2- 172.21.15.106:0/2086358805 >> v2:172.21.15.156:3300/0 conn(0x7fae3c0cfb60 0x7fae3c161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.302+0000 7fae3b7fe640 1 -- 172.21.15.106:0/2086358805 >> v1:172.21.15.156:6789/0 conn(0x7fae3c157c70 legacy=0x7fae3c0cf450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.302+0000 7fae3b7fe640 1 -- 172.21.15.106:0/2086358805 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c15e8d0 con 0x7fae3c15dc80 2023-11-24T08:13:28.305 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.302+0000 7fae3a7fc640 1 --2- 172.21.15.106:0/2086358805 >> v2:172.21.15.156:3300/0 conn(0x7fae3c0cfb60 0x7fae3c161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.303+0000 7fae3b7fe640 1 --2- 172.21.15.106:0/2086358805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c15dc80 0x7fae3c15e130 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b44b0 tx=0x7fae3402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586bb8892e93158b server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.304+0000 7fae39ffb640 1 -- 172.21.15.106:0/2086358805 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3402c9f0 con 0x7fae3c15dc80 2023-11-24T08:13:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.304+0000 7fae39ffb640 1 -- 172.21.15.106:0/2086358805 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3402cb50 con 0x7fae3c15dc80 2023-11-24T08:13:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.304+0000 7fae39ffb640 1 -- 172.21.15.106:0/2086358805 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340343f0 con 0x7fae3c15dc80 2023-11-24T08:13:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.304+0000 7fae4199e640 1 -- 172.21.15.106:0/2086358805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c15dc80 msgr2=0x7fae3c15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.307 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.304+0000 7fae4199e640 1 --2- 172.21.15.106:0/2086358805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c15dc80 0x7fae3c15e130 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fae3c0b44b0 tx=0x7fae3402be50 comp rx=0 tx=0).stop 2023-11-24T08:13:28.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.305+0000 7fae4199e640 1 -- 172.21.15.106:0/2086358805 shutdown_connections 2023-11-24T08:13:28.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.305+0000 7fae4199e640 1 --2- 172.21.15.106:0/2086358805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c15dc80 0x7fae3c15e130 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.305+0000 7fae4199e640 1 --2- 172.21.15.106:0/2086358805 >> v2:172.21.15.156:3300/0 conn(0x7fae3c0cfb60 0x7fae3c161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.305+0000 7fae4199e640 1 -- 172.21.15.106:0/2086358805 >> 172.21.15.106:0/2086358805 conn(0x7fae3c06a920 msgr2=0x7fae3c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:28.308 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.305+0000 7fae4199e640 1 -- 172.21.15.106:0/2086358805 shutdown_connections 2023-11-24T08:13:28.309 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.305+0000 7fae4199e640 1 -- 172.21.15.106:0/2086358805 wait complete. 2023-11-24T08:13:28.311 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.306+0000 7fae4199e640 1 Processor -- start 2023-11-24T08:13:28.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.307+0000 7fae4199e640 1 -- start start 2023-11-24T08:13:28.312 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.307+0000 7fae4199e640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0cfb60 0x7fae3c09ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae4199e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c157c70 0x7fae3c152db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae4199e640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae3c15dc80 0x7fae3c1f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae4199e640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c14fe10 con 0x7fae3c157c70 2023-11-24T08:13:28.313 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae4199e640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c1e9160 con 0x7fae3c0cfb60 2023-11-24T08:13:28.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae4199e640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae3c1e8550 con 0x7fae3c15dc80 2023-11-24T08:13:28.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c157c70 0x7fae3c152db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.314 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c157c70 0x7fae3c152db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59874/0 (socket says 172.21.15.106:59874) 2023-11-24T08:13:28.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 -- 172.21.15.106:0/712241622 learned_addr learned my addr 172.21.15.106:0/712241622 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:28.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3b7fe640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae3c15dc80 0x7fae3c1f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3affd640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0cfb60 0x7fae3c09ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.315 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 -- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae3c15dc80 msgr2=0x7fae3c1f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae3c15dc80 0x7fae3c1f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 -- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0cfb60 msgr2=0x7fae3c09ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0cfb60 0x7fae3c09ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3a7fc640 1 -- 172.21.15.106:0/712241622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae34009b30 con 0x7fae3c157c70 2023-11-24T08:13:28.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3b7fe640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fae3c15dc80 0x7fae3c1f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:28.316 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.308+0000 7fae3affd640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fae3c0cfb60 0x7fae3c09ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.309+0000 7fae3a7fc640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c157c70 0x7fae3c152db0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.309+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30007820 con 0x7fae3c157c70 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.309+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae30007980 con 0x7fae3c157c70 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.309+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae3c1e8830 con 0x7fae3c157c70 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.309+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae30007cf0 con 0x7fae3c157c70 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.309+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae3c1e8df0 con 0x7fae3c157c70 2023-11-24T08:13:28.317 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.310+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fae3c1615f0 con 0x7fae3c157c70 2023-11-24T08:13:28.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.311+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fae300113f0 con 0x7fae3c157c70 2023-11-24T08:13:28.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.312+0000 7fae23fff640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae2806b470 0x7fae2806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.312+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fae3008a2b0 con 0x7fae3c157c70 2023-11-24T08:13:28.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.312+0000 7fae3affd640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae2806b470 0x7fae2806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.319 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.312+0000 7fae23fff640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fae280712d0 0x7fae280736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.312+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 --> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae28073da0 con 0x7fae280712d0 2023-11-24T08:13:28.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.312+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae3008c7c0 con 0x7fae3c157c70 2023-11-24T08:13:28.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.313+0000 7fae3b7fe640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fae280712d0 0x7fae280736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.313+0000 7fae3affd640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae2806b470 0x7fae2806d930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00a810 tx=0x7fae2c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.313+0000 7fae3b7fe640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fae280712d0 0x7fae280736f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.320 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.314+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== osd.0 v2:172.21.15.106:6808/3403388309 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fae28073da0 con 0x7fae280712d0 2023-11-24T08:13:28.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.328+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 --> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fae3c161310 con 0x7fae280712d0 2023-11-24T08:13:28.330 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.329+0000 7fae23fff640 1 -- 172.21.15.106:0/712241622 <== osd.0 v2:172.21.15.106:6808/3403388309 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fae34040410 con 0x7fae280712d0 2023-11-24T08:13:28.331 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:28.331 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:28.331 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:28.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.329+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fae280712d0 msgr2=0x7fae280736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.329+0000 7fae4199e640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6808/3403388309,v1:172.21.15.106:6809/3403388309] conn(0x7fae280712d0 0x7fae280736f0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.332 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae2806b470 msgr2=0x7fae2806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fae2806b470 0x7fae2806d930 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fae2c00a810 tx=0x7fae2c008040 comp rx=0 tx=0).stop 2023-11-24T08:13:28.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c157c70 msgr2=0x7fae3c152db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 --2- 172.21.15.106:0/712241622 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fae3c157c70 0x7fae3c152db0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fae3000c7f0 tx=0x7fae3000ccb0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae3b7fe640 1 -- 172.21.15.106:0/712241622 reap_dead start 2023-11-24T08:13:28.333 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 shutdown_connections 2023-11-24T08:13:28.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 >> 172.21.15.106:0/712241622 conn(0x7fae3c06a920 msgr2=0x7fae3c0cf420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:28.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 shutdown_connections 2023-11-24T08:13:28.334 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.330+0000 7fae4199e640 1 -- 172.21.15.106:0/712241622 wait complete. 2023-11-24T08:13:28.349 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-24T08:13:28.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.562+0000 7f327ac56640 1 Processor -- start 2023-11-24T08:13:28.564 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.562+0000 7f327ac56640 1 -- start start 2023-11-24T08:13:28.565 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f327ac56640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f327415df80 0x7f32740ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f327ac56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f327415b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f327ac56640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f327415e3f0 con 0x7f32740b2250 2023-11-24T08:13:28.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f327ac56640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f327415a990 con 0x7f327415df80 2023-11-24T08:13:28.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f327ac56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f327415ab80 con 0x7f32740cd470 2023-11-24T08:13:28.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f32789f2640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f32740b2250 0x7f32740b2630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58912/0 (socket says 172.21.15.106:58912) 2023-11-24T08:13:28.566 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f32789f2640 1 -- 172.21.15.106:0/2386820599 learned_addr learned my addr 172.21.15.106:0/2386820599 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:28.568 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f3273fff640 1 --2- 172.21.15.106:0/2386820599 >> v2:172.21.15.156:3300/0 conn(0x7f327415df80 0x7f32740ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f32791f3640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f327415b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.564+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045586991 0 0) 0x7f327415e3f0 con 0x7f32740b2250 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3264003520 con 0x7f32740b2250 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2497150984 0 0) 0x7f3264003520 con 0x7f32740b2250 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 msgr2=0x7f327415b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f327415b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.569 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 >> v2:172.21.15.156:3300/0 conn(0x7f327415df80 msgr2=0x7f32740ccf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 --2- 172.21.15.106:0/2386820599 >> v2:172.21.15.156:3300/0 conn(0x7f327415df80 0x7f32740ccf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32740b2d40 con 0x7f32740b2250 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7f3268001670 con 0x7f32740b2250 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 >> v1:172.21.15.106:6789/0 conn(0x7f32740b2250 legacy=0x7f32740b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f3264003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3264003ef0 0x7f32640047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3264004d00 0x7f3264005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.570 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327415ab80 con 0x7f3264004d00 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326400b980 con 0x7f3264003ef0 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f326400bb10 con 0x7f32740cd470 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.565+0000 7f3273fff640 1 --2- 172.21.15.106:0/2386820599 >> v2:172.21.15.156:3300/0 conn(0x7f327415df80 0x7f32740ccf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32791f3640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3264004d00 0x7f3264005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32789f2640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f3264003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32791f3640 1 -- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 msgr2=0x7f3264003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32791f3640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f3264003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f3273fff640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3264003ef0 0x7f32640047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32791f3640 1 -- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3264003ef0 msgr2=0x7f32640047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:28.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32791f3640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3264003ef0 0x7f32640047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32791f3640 1 -- 172.21.15.106:0/2386820599 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3264003520 con 0x7f3264004d00 2023-11-24T08:13:28.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f32789f2640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f32740cd470 0x7f3264003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:28.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.566+0000 7f3273fff640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3264003ef0 0x7f32640047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:28.572 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.567+0000 7f32791f3640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3264004d00 0x7f3264005180 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f326c009fc0 tx=0x7f32640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d61e6977750d9c server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c02cc20 con 0x7f3264004d00 2023-11-24T08:13:28.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326c00f040 con 0x7f3264004d00 2023-11-24T08:13:28.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f32737fe640 1 -- 172.21.15.106:0/2386820599 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326c034440 con 0x7f3264004d00 2023-11-24T08:13:28.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f327ac56640 1 -- 172.21.15.106:0/2386820599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3264004d00 msgr2=0x7f3264005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f327ac56640 1 --2- 172.21.15.106:0/2386820599 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f3264004d00 0x7f3264005180 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f326c009fc0 tx=0x7f32640057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.577 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f32791f3640 1 -- 172.21.15.106:0/2386820599 reap_dead start 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f327ac56640 1 -- 172.21.15.106:0/2386820599 shutdown_connections 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.568+0000 7f327ac56640 1 -- 172.21.15.106:0/2386820599 >> 172.21.15.106:0/2386820599 conn(0x7f327406a920 msgr2=0x7f327406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.569+0000 7f327ac56640 1 -- 172.21.15.106:0/2386820599 shutdown_connections 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.569+0000 7f327ac56640 1 -- 172.21.15.106:0/2386820599 wait complete. 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.571+0000 7f327ac56640 1 Processor -- start 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.571+0000 7f327ac56640 1 -- start start 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f327ac56640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3264004d00 0x7f3274095cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.578 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f327ac56640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 0x7f3274149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f327ac56640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274095000 0x7f32740954b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f327ac56640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327414fe70 con 0x7f32740b17f0 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f327ac56640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3274145f80 con 0x7f3274095000 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f327ac56640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3274145af0 con 0x7f3264004d00 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f3273fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 0x7f3274149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f3273fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 0x7f3274149ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59882/0 (socket says 172.21.15.106:59882) 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.572+0000 7f3273fff640 1 -- 172.21.15.106:0/371695026 learned_addr learned my addr 172.21.15.106:0/371695026 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:28.579 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f32789f2640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3264004d00 0x7f3274095cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f32791f3640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274095000 0x7f32740954b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f3273fff640 1 -- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3264004d00 msgr2=0x7f3274095cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f3273fff640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3264004d00 0x7f3274095cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f3273fff640 1 -- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274095000 msgr2=0x7f32740954b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f3273fff640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274095000 0x7f32740954b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f3273fff640 1 -- 172.21.15.106:0/371695026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f327414f8b0 con 0x7f32740b17f0 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.573+0000 7f3273fff640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 0x7f3274149ba0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f326000f650 tx=0x7f326000fb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.580 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.574+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600098e0 con 0x7f32740b17f0 2023-11-24T08:13:28.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.574+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f326000c030 con 0x7f32740b17f0 2023-11-24T08:13:28.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.574+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3260009c40 con 0x7f32740b17f0 2023-11-24T08:13:28.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.574+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327414fb90 con 0x7f32740b17f0 2023-11-24T08:13:28.581 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.574+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32740959f0 con 0x7f32740b17f0 2023-11-24T08:13:28.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.575+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32741455e0 con 0x7f32740b17f0 2023-11-24T08:13:28.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.576+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f3260014480 con 0x7f32740b17f0 2023-11-24T08:13:28.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.577+0000 7f32717fa640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f325806b4a0 0x7f325806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:28.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.577+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f326008a2f0 con 0x7f32740b17f0 2023-11-24T08:13:28.582 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.577+0000 7f32789f2640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f325806b4a0 0x7f325806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:28.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.578+0000 7f32789f2640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f325806b4a0 0x7f325806d960 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f32680018a0 tx=0x7f3268009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:28.583 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.582+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f32741455e0 con 0x7f32740b17f0 2023-11-24T08:13:28.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.774+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f327415ec40 con 0x7f325806b4a0 2023-11-24T08:13:28.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.774+0000 7f32717fa640 1 -- 172.21.15.106:0/371695026 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f326800d050 con 0x7f325806b4a0 2023-11-24T08:13:28.776 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 0 to deep-scrub 2023-11-24T08:13:28.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f325806b4a0 msgr2=0x7f325806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f325806b4a0 0x7f325806d960 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f32680018a0 tx=0x7f3268009040 comp rx=0 tx=0).stop 2023-11-24T08:13:28.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 msgr2=0x7f3274149ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:28.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 0x7f3274149ba0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f326000f650 tx=0x7f326000fb10 comp rx=0 tx=0).stop 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 shutdown_connections 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3274095000 0x7f32740954b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f32740b17f0 0x7f3274149ba0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f325806b4a0 0x7f325806d960 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 --2- 172.21.15.106:0/371695026 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f3264004d00 0x7f3274095cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.777+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 >> 172.21.15.106:0/371695026 conn(0x7f327406a920 msgr2=0x7f32740705c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.778+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 shutdown_connections 2023-11-24T08:13:28.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:28.778+0000 7f327ac56640 1 -- 172.21.15.106:0/371695026 wait complete. 2023-11-24T08:13:28.797 INFO:tasks.ceph:Scrubbing osd.1 2023-11-24T08:13:28.797 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:29.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.012+0000 7fcb408d4640 1 Processor -- start 2023-11-24T08:13:29.014 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.013+0000 7fcb408d4640 1 -- start start 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.014+0000 7fcb408d4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb3c06ac10 0x7fcb3c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.014+0000 7fcb408d4640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0cfed0 con 0x7fcb3c15d920 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.014+0000 7fcb408d4640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0caae0 con 0x7fcb3c0b2400 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.014+0000 7fcb408d4640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0cadc0 con 0x7fcb3c06ac10 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.014+0000 7fcb3ad9d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fcb3c06ac10 0x7fcb3c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.014+0000 7fcb3a59c640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fcb3c15d920 0x7fcb3c0acfd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58928/0 (socket says 172.21.15.106:58928) 2023-11-24T08:13:29.016 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.015+0000 7fcb3a59c640 1 -- 172.21.15.106:0/1868718891 learned_addr learned my addr 172.21.15.106:0/1868718891 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:29.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.015+0000 7fcb3ad9d640 1 -- 172.21.15.106:0/1868718891 >> v1:172.21.15.156:6789/0 conn(0x7fcb3c0b2400 legacy=0x7fcb3c06a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:29.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.015+0000 7fcb3ad9d640 1 -- 172.21.15.106:0/1868718891 >> v1:172.21.15.106:6789/0 conn(0x7fcb3c15d920 legacy=0x7fcb3c0acfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.017 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.015+0000 7fcb3ad9d640 1 -- 172.21.15.106:0/1868718891 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0cb0a0 con 0x7fcb3c06ac10 2023-11-24T08:13:29.018 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.016+0000 7fcb3ad9d640 1 --2- 172.21.15.106:0/1868718891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c06ac10 0x7fcb3c0ccb90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcb340097a0 tx=0x7fcb3402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd58abca5bca824 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.019 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.016+0000 7fcb3959a640 1 -- 172.21.15.106:0/1868718891 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34030610 con 0x7fcb3c06ac10 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.016+0000 7fcb3959a640 1 -- 172.21.15.106:0/1868718891 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb34030770 con 0x7fcb3c06ac10 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.016+0000 7fcb3959a640 1 -- 172.21.15.106:0/1868718891 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34030b00 con 0x7fcb3c06ac10 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 -- 172.21.15.106:0/1868718891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c06ac10 msgr2=0x7fcb3c0ccb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 --2- 172.21.15.106:0/1868718891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c06ac10 0x7fcb3c0ccb90 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fcb340097a0 tx=0x7fcb3402b960 comp rx=0 tx=0).stop 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 -- 172.21.15.106:0/1868718891 shutdown_connections 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 --2- 172.21.15.106:0/1868718891 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c06ac10 0x7fcb3c0ccb90 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 -- 172.21.15.106:0/1868718891 >> 172.21.15.106:0/1868718891 conn(0x7fcb3c0999d0 msgr2=0x7fcb3c0b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 -- 172.21.15.106:0/1868718891 shutdown_connections 2023-11-24T08:13:29.021 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.017+0000 7fcb408d4640 1 -- 172.21.15.106:0/1868718891 wait complete. 2023-11-24T08:13:29.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.019+0000 7fcb408d4640 1 Processor -- start 2023-11-24T08:13:29.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.019+0000 7fcb408d4640 1 -- start start 2023-11-24T08:13:29.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb408d4640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb3c06ac10 0x7fcb3c1595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb408d4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c0b2400 0x7fcb3c0c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb408d4640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb3c15d920 0x7fcb3c0c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.023 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb408d4640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0ca090 con 0x7fcb3c0b2400 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb408d4640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c0c6b20 con 0x7fcb3c06ac10 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb408d4640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c15a070 con 0x7fcb3c15d920 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c0b2400 0x7fcb3c0c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb39d9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c0b2400 0x7fcb3c0c8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59902/0 (socket says 172.21.15.106:59902) 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.020+0000 7fcb39d9b640 1 -- 172.21.15.106:0/364958319 learned_addr learned my addr 172.21.15.106:0/364958319 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb3a59c640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb3c06ac10 0x7fcb3c1595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb3ad9d640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb3c15d920 0x7fcb3c0c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.024 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb39d9b640 1 -- 172.21.15.106:0/364958319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb3c15d920 msgr2=0x7fcb3c0c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb39d9b640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fcb3c15d920 0x7fcb3c0c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb39d9b640 1 -- 172.21.15.106:0/364958319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb3c06ac10 msgr2=0x7fcb3c1595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb39d9b640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb3c06ac10 0x7fcb3c1595c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb39d9b640 1 -- 172.21.15.106:0/364958319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009400 con 0x7fcb3c0b2400 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb39d9b640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c0b2400 0x7fcb3c0c8d50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c430 tx=0x7fcb2400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24011790 con 0x7fcb3c0b2400 2023-11-24T08:13:29.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb240118f0 con 0x7fcb3c0b2400 2023-11-24T08:13:29.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24011ce0 con 0x7fcb3c0b2400 2023-11-24T08:13:29.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c159b00 con 0x7fcb3c0b2400 2023-11-24T08:13:29.026 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.021+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c0c8060 con 0x7fcb3c0b2400 2023-11-24T08:13:29.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.022+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb3c0ceba0 con 0x7fcb3c0b2400 2023-11-24T08:13:29.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.023+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fcb2401c460 con 0x7fcb3c0b2400 2023-11-24T08:13:29.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.024+0000 7fcb32ffd640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb1c06b470 0x7fcb1c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.024+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcb24089d90 con 0x7fcb3c0b2400 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.025+0000 7fcb3a59c640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb1c06b470 0x7fcb1c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.025+0000 7fcb32ffd640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7fcb1c0712d0 0x7fcb1c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.025+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 --> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb1c073da0 con 0x7fcb1c0712d0 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.025+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb2408f2d0 con 0x7fcb3c0b2400 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.025+0000 7fcb3ad9d640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7fcb1c0712d0 0x7fcb1c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.025+0000 7fcb3a59c640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb1c06b470 0x7fcb1c06d930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c005c00 tx=0x7fcb2c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.026+0000 7fcb3ad9d640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7fcb1c0712d0 0x7fcb1c0736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.026+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== osd.1 v2:172.21.15.106:6800/1347023088 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fcb1c073da0 con 0x7fcb1c0712d0 2023-11-24T08:13:29.042 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.041+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 --> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcb3c0ce260 con 0x7fcb1c0712d0 2023-11-24T08:13:29.043 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.041+0000 7fcb32ffd640 1 -- 172.21.15.106:0/364958319 <== osd.1 v2:172.21.15.106:6800/1347023088 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcb3c0ce260 con 0x7fcb1c0712d0 2023-11-24T08:13:29.043 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:29.043 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:29.044 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:29.044 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7fcb1c0712d0 msgr2=0x7fcb1c0736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6800/1347023088,v1:172.21.15.106:6801/1347023088] conn(0x7fcb1c0712d0 0x7fcb1c0736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb1c06b470 msgr2=0x7fcb1c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fcb1c06b470 0x7fcb1c06d930 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c005c00 tx=0x7fcb2c009040 comp rx=0 tx=0).stop 2023-11-24T08:13:29.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c0b2400 msgr2=0x7fcb3c0c8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.045 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 --2- 172.21.15.106:0/364958319 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fcb3c0b2400 0x7fcb3c0c8d50 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400c430 tx=0x7fcb2400c8f0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb3ad9d640 1 -- 172.21.15.106:0/364958319 reap_dead start 2023-11-24T08:13:29.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 shutdown_connections 2023-11-24T08:13:29.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 >> 172.21.15.106:0/364958319 conn(0x7fcb3c0999d0 msgr2=0x7fcb3c15b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 shutdown_connections 2023-11-24T08:13:29.046 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.042+0000 7fcb408d4640 1 -- 172.21.15.106:0/364958319 wait complete. 2023-11-24T08:13:29.062 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-24T08:13:29.273 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.272+0000 7f54ca9a7640 1 Processor -- start 2023-11-24T08:13:29.274 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.272+0000 7f54ca9a7640 1 -- start start 2023-11-24T08:13:29.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54ca9a7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54c40b1260 0x7f54c40718d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54ca9a7640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f54c4071e10 0x7f54c4142d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54ca9a7640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54c41450d0 con 0x7f54c4141b90 2023-11-24T08:13:29.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54ca9a7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c41422b0 con 0x7f54c40b1260 2023-11-24T08:13:29.276 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54ca9a7640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54c40725e0 con 0x7f54c4071e10 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54c91a4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54c40b1260 0x7f54c40718d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54c91a4640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f54c40b1260 0x7f54c40718d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59910/0 (socket says 172.21.15.106:59910) 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54c91a4640 1 -- 172.21.15.106:0/2365462095 learned_addr learned my addr 172.21.15.106:0/2365462095 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54ca1a6640 1 --2- 172.21.15.106:0/2365462095 >> v2:172.21.15.156:3300/0 conn(0x7f54c4071e10 0x7f54c4142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54c91a4640 1 -- 172.21.15.106:0/2365462095 >> v2:172.21.15.156:3300/0 conn(0x7f54c4071e10 msgr2=0x7f54c4142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54c91a4640 1 --2- 172.21.15.106:0/2365462095 >> v2:172.21.15.156:3300/0 conn(0x7f54c4071e10 0x7f54c4142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.277 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.274+0000 7f54c91a4640 1 -- 172.21.15.106:0/2365462095 >> v1:172.21.15.156:6789/0 conn(0x7f54c4141b90 legacy=0x7f54c40b0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:29.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.275+0000 7f54c91a4640 1 -- 172.21.15.106:0/2365462095 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c4142520 con 0x7f54c40b1260 2023-11-24T08:13:29.278 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.275+0000 7f54ca1a6640 1 --2- 172.21.15.106:0/2365462095 >> v2:172.21.15.156:3300/0 conn(0x7f54c4071e10 0x7f54c4142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.276+0000 7f54c91a4640 1 --2- 172.21.15.106:0/2365462095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c40718d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f54c4160120 tx=0x7f54b402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac1ea1816daaf48 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.276+0000 7f54c89a3640 1 -- 172.21.15.106:0/2365462095 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b40317e0 con 0x7f54c40b1260 2023-11-24T08:13:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.276+0000 7f54c89a3640 1 -- 172.21.15.106:0/2365462095 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4031940 con 0x7f54c40b1260 2023-11-24T08:13:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.276+0000 7f54c89a3640 1 -- 172.21.15.106:0/2365462095 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4031cd0 con 0x7f54c40b1260 2023-11-24T08:13:29.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 -- 172.21.15.106:0/2365462095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 msgr2=0x7f54c40718d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/2365462095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c40718d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f54c4160120 tx=0x7f54b402b880 comp rx=0 tx=0).stop 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 -- 172.21.15.106:0/2365462095 shutdown_connections 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/2365462095 >> v2:172.21.15.156:3300/0 conn(0x7f54c4071e10 0x7f54c4142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/2365462095 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c40718d0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 -- 172.21.15.106:0/2365462095 >> 172.21.15.106:0/2365462095 conn(0x7f54c406a500 msgr2=0x7f54c406a8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 -- 172.21.15.106:0/2365462095 shutdown_connections 2023-11-24T08:13:29.280 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.277+0000 7f54ca9a7640 1 -- 172.21.15.106:0/2365462095 wait complete. 2023-11-24T08:13:29.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.279+0000 7f54ca9a7640 1 Processor -- start 2023-11-24T08:13:29.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.279+0000 7f54ca9a7640 1 -- start start 2023-11-24T08:13:29.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca9a7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c4071e10 0x7f54c40b2410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca9a7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c415d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca9a7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c4141b90 0x7f54c41f2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca9a7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c415d650 con 0x7f54c40b1260 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca9a7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b2aa0 con 0x7f54c4141b90 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca9a7640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54c40b2d80 con 0x7f54c4071e10 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54c91a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c415d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54c91a4640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c415d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59920/0 (socket says 172.21.15.106:59920) 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54c91a4640 1 -- 172.21.15.106:0/3641944524 learned_addr learned my addr 172.21.15.106:0/3641944524 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:29.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54c99a5640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c4071e10 0x7f54c40b2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.280+0000 7f54ca1a6640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c4141b90 0x7f54c41f2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54c91a4640 1 -- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c4071e10 msgr2=0x7f54c40b2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54c91a4640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c4071e10 0x7f54c40b2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54c91a4640 1 -- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c4141b90 msgr2=0x7f54c41f2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54c91a4640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c4141b90 0x7f54c41f2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54c91a4640 1 -- 172.21.15.106:0/3641944524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b40094d0 con 0x7f54c40b1260 2023-11-24T08:13:29.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54ca1a6640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c4141b90 0x7f54c41f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:29.285 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54c91a4640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c415d110 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f54b402b8c0 tx=0x7f54b40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b40329c0 con 0x7f54c40b1260 2023-11-24T08:13:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b4032b40 con 0x7f54c40b1260 2023-11-24T08:13:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b40453f0 con 0x7f54c40b1260 2023-11-24T08:13:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54c40b3060 con 0x7f54c40b1260 2023-11-24T08:13:29.286 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.281+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54c40b3620 con 0x7f54c40b1260 2023-11-24T08:13:29.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.283+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54c404e840 con 0x7f54c40b1260 2023-11-24T08:13:29.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.283+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f54b404d3f0 con 0x7f54c40b1260 2023-11-24T08:13:29.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.284+0000 7f54ba7fc640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54a806b470 0x7f54a806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.284+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f54b40abee0 con 0x7f54c40b1260 2023-11-24T08:13:29.287 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.285+0000 7f54c99a5640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54a806b470 0x7f54a806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.288 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.285+0000 7f54c99a5640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54a806b470 0x7f54a806d930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f54c0006fd0 tx=0x7f54c0008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.290 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.288+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f54b40768f0 con 0x7f54c40b1260 2023-11-24T08:13:29.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.479+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f54c40ccf70 con 0x7f54a806b470 2023-11-24T08:13:29.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.479+0000 7f54ba7fc640 1 -- 172.21.15.106:0/3641944524 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f54c40ccf70 con 0x7f54a806b470 2023-11-24T08:13:29.481 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 1 to deep-scrub 2023-11-24T08:13:29.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54a806b470 msgr2=0x7f54a806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54a806b470 0x7f54a806d930 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f54c0006fd0 tx=0x7f54c0008040 comp rx=0 tx=0).stop 2023-11-24T08:13:29.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 msgr2=0x7f54c415d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c415d110 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f54b402b8c0 tx=0x7f54b40046c0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 shutdown_connections 2023-11-24T08:13:29.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54c4141b90 0x7f54c41f2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f54c40b1260 0x7f54c415d110 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f54a806b470 0x7f54a806d930 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 --2- 172.21.15.106:0/3641944524 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f54c4071e10 0x7f54c40b2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 >> 172.21.15.106:0/3641944524 conn(0x7f54c406a500 msgr2=0x7f54c40ac840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 shutdown_connections 2023-11-24T08:13:29.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.482+0000 7f54ca9a7640 1 -- 172.21.15.106:0/3641944524 wait complete. 2023-11-24T08:13:29.501 INFO:tasks.ceph:Scrubbing osd.2 2023-11-24T08:13:29.501 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.710+0000 7ff741690640 1 Processor -- start 2023-11-24T08:13:29.712 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.710+0000 7ff741690640 1 -- start start 2023-11-24T08:13:29.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.712+0000 7ff741690640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff73c0ce6c0 0x7ff73c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.712+0000 7ff741690640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c0cece0 con 0x7ff73c15c7f0 2023-11-24T08:13:29.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.712+0000 7ff741690640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff73c06d040 con 0x7ff73c0cf450 2023-11-24T08:13:29.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.712+0000 7ff741690640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff73c0cfe40 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.714 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff73b7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff73c0ce6c0 0x7ff73c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff73affd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff73c15c7f0 0x7ff73c06db20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58934/0 (socket says 172.21.15.106:58934) 2023-11-24T08:13:29.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff73affd640 1 -- 172.21.15.106:0/3902770894 learned_addr learned my addr 172.21.15.106:0/3902770894 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:29.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff73b7fe640 1 -- 172.21.15.106:0/3902770894 >> v1:172.21.15.156:6789/0 conn(0x7ff73c0cf450 legacy=0x7ff73c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:29.715 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff73b7fe640 1 -- 172.21.15.106:0/3902770894 >> v1:172.21.15.106:6789/0 conn(0x7ff73c15c7f0 legacy=0x7ff73c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff73b7fe640 1 -- 172.21.15.106:0/3902770894 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff73c06d320 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.716 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.713+0000 7ff739ffb640 1 -- 172.21.15.106:0/3902770894 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225951819 0 0) 0x7ff73c0cece0 con 0x7ff73c15c7f0 2023-11-24T08:13:29.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.714+0000 7ff73b7fe640 1 --2- 172.21.15.106:0/3902770894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c0cf900 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff7340097a0 tx=0x7ff73402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc1d7468b3565f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.715+0000 7ff739ffb640 1 -- 172.21.15.106:0/3902770894 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734031830 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.715+0000 7ff739ffb640 1 -- 172.21.15.106:0/3902770894 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff734031990 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.717 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.715+0000 7ff739ffb640 1 -- 172.21.15.106:0/3902770894 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff734031d20 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.715+0000 7ff741690640 1 -- 172.21.15.106:0/3902770894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 msgr2=0x7ff73c0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.715+0000 7ff741690640 1 --2- 172.21.15.106:0/3902770894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c0cf900 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff7340097a0 tx=0x7ff73402b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.716+0000 7ff741690640 1 -- 172.21.15.106:0/3902770894 shutdown_connections 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.716+0000 7ff741690640 1 --2- 172.21.15.106:0/3902770894 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c0cf900 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.716+0000 7ff741690640 1 -- 172.21.15.106:0/3902770894 >> 172.21.15.106:0/3902770894 conn(0x7ff73c06a920 msgr2=0x7ff73c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.716+0000 7ff741690640 1 -- 172.21.15.106:0/3902770894 shutdown_connections 2023-11-24T08:13:29.718 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.716+0000 7ff741690640 1 -- 172.21.15.106:0/3902770894 wait complete. 2023-11-24T08:13:29.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.718+0000 7ff741690640 1 Processor -- start 2023-11-24T08:13:29.719 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.718+0000 7ff741690640 1 -- start start 2023-11-24T08:13:29.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.719+0000 7ff741690640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c14caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.719+0000 7ff741690640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cf450 0x7ff73c0cac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.719+0000 7ff741690640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff73c15c7f0 0x7ff73c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.721 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.719+0000 7ff741690640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c14d030 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.719+0000 7ff741690640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c0b4110 con 0x7ff73c0cf450 2023-11-24T08:13:29.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.719+0000 7ff741690640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73c1f6ac0 con 0x7ff73c15c7f0 2023-11-24T08:13:29.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c14caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c14caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59934/0 (socket says 172.21.15.106:59934) 2023-11-24T08:13:29.722 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 -- 172.21.15.106:0/3799811896 learned_addr learned my addr 172.21.15.106:0/3799811896 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:29.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 -- 172.21.15.106:0/3799811896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff73c15c7f0 msgr2=0x7ff73c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.723 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73b7fe640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff73c15c7f0 0x7ff73c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff73c15c7f0 0x7ff73c06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 -- 172.21.15.106:0/3799811896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cf450 msgr2=0x7ff73c0cac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73a7fc640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cf450 0x7ff73c0cac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff73c0cf450 0x7ff73c0cac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 -- 172.21.15.106:0/3799811896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734009400 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.720+0000 7ff73affd640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c14caf0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00c7c0 tx=0x7ff72c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.724 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.721+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00a640 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.721+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c00a7c0 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.721+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00ab70 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.721+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff73c1f6da0 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.721+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73c1f7360 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.725 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.722+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff73c14d1a0 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.723+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff72c00acd0 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.724+0000 7ff7337fe640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff71c06b470 0x7ff71c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.728 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.724+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff72c089390 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.724+0000 7ff73a7fc640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff71c06b470 0x7ff71c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.724+0000 7ff7337fe640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7ff71c0712d0 0x7ff71c0736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.725+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 --> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff71c073da0 con 0x7ff71c0712d0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.725+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff72c08b970 con 0x7ff73c0ce6c0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.725+0000 7ff73b7fe640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7ff71c0712d0 0x7ff71c0736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.725+0000 7ff73a7fc640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff71c06b470 0x7ff71c06d930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff73c14e7b0 tx=0x7ff724009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.725+0000 7ff73b7fe640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7ff71c0712d0 0x7ff71c0736f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.729 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.726+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== osd.2 v2:172.21.15.106:6810/1671152390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff71c073da0 con 0x7ff71c0712d0 2023-11-24T08:13:29.742 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.741+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 --> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff73c067470 con 0x7ff71c0712d0 2023-11-24T08:13:29.743 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.741+0000 7ff7337fe640 1 -- 172.21.15.106:0/3799811896 <== osd.2 v2:172.21.15.106:6810/1671152390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff73c067470 con 0x7ff71c0712d0 2023-11-24T08:13:29.743 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:29.743 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:29.743 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.741+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7ff71c0712d0 msgr2=0x7ff71c0736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.741+0000 7ff741690640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6810/1671152390,v1:172.21.15.106:6812/1671152390] conn(0x7ff71c0712d0 0x7ff71c0736f0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.744 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff71c06b470 msgr2=0x7ff71c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff71c06b470 0x7ff71c06d930 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7ff73c14e7b0 tx=0x7ff724009040 comp rx=0 tx=0).stop 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 msgr2=0x7ff73c14caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 --2- 172.21.15.106:0/3799811896 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff73c0ce6c0 0x7ff73c14caf0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00c7c0 tx=0x7ff72c00cc80 comp rx=0 tx=0).stop 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff73b7fe640 1 -- 172.21.15.106:0/3799811896 reap_dead start 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 shutdown_connections 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 >> 172.21.15.106:0/3799811896 conn(0x7ff73c06a920 msgr2=0x7ff73c06c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.745 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 shutdown_connections 2023-11-24T08:13:29.746 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.742+0000 7ff741690640 1 -- 172.21.15.106:0/3799811896 wait complete. 2023-11-24T08:13:29.759 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-24T08:13:29.970 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.968+0000 7f5a7f07c640 1 Processor -- start 2023-11-24T08:13:29.971 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.969+0000 7f5a7f07c640 1 -- start start 2023-11-24T08:13:29.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.970+0000 7f5a7f07c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f5a780d22b0 0x7f5a780ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.972 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.970+0000 7f5a7f07c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5a780ce710 0x7f5a780cfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.970+0000 7f5a7f07c640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a780b2d90 con 0x7f5a780d1ed0 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.970+0000 7f5a7f07c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780ceb80 con 0x7f5a780ce710 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.970+0000 7f5a7f07c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a780b45f0 con 0x7f5a780d22b0 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.970+0000 7f5a7d619640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5a780ce710 0x7f5a780cfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a7d619640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f5a780ce710 0x7f5a780cfdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59938/0 (socket says 172.21.15.106:59938) 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a7d619640 1 -- 172.21.15.106:0/2445026805 learned_addr learned my addr 172.21.15.106:0/2445026805 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a77fff640 1 --2- 172.21.15.106:0/2445026805 >> v2:172.21.15.156:3300/0 conn(0x7f5a780d22b0 0x7f5a780ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.973 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a7d619640 1 -- 172.21.15.106:0/2445026805 >> v2:172.21.15.156:3300/0 conn(0x7f5a780d22b0 msgr2=0x7f5a780ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:29.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a7d619640 1 --2- 172.21.15.106:0/2445026805 >> v2:172.21.15.156:3300/0 conn(0x7f5a780d22b0 0x7f5a780ce1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a7d619640 1 -- 172.21.15.106:0/2445026805 >> v1:172.21.15.156:6789/0 conn(0x7f5a780d1ed0 legacy=0x7f5a780d1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:29.974 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.971+0000 7f5a7d619640 1 -- 172.21.15.106:0/2445026805 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a780ced90 con 0x7f5a780ce710 2023-11-24T08:13:29.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.972+0000 7f5a7d619640 1 --2- 172.21.15.106:0/2445026805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780ce710 0x7f5a780cfdb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5a7808bdd0 tx=0x7f5a7002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73f6a68e3c88cfe3 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.972+0000 7f5a777fe640 1 -- 172.21.15.106:0/2445026805 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a7002c9f0 con 0x7f5a780ce710 2023-11-24T08:13:29.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a777fe640 1 -- 172.21.15.106:0/2445026805 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a7002cb50 con 0x7f5a780ce710 2023-11-24T08:13:29.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a777fe640 1 -- 172.21.15.106:0/2445026805 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a700343f0 con 0x7f5a780ce710 2023-11-24T08:13:29.975 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 -- 172.21.15.106:0/2445026805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780ce710 msgr2=0x7f5a780cfdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/2445026805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780ce710 0x7f5a780cfdb0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f5a7808bdd0 tx=0x7f5a7002be50 comp rx=0 tx=0).stop 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 -- 172.21.15.106:0/2445026805 shutdown_connections 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/2445026805 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780ce710 0x7f5a780cfdb0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/2445026805 >> v2:172.21.15.156:3300/0 conn(0x7f5a780d22b0 0x7f5a780ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 -- 172.21.15.106:0/2445026805 >> 172.21.15.106:0/2445026805 conn(0x7f5a7808af60 msgr2=0x7f5a780d3430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 -- 172.21.15.106:0/2445026805 shutdown_connections 2023-11-24T08:13:29.976 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.973+0000 7f5a7f07c640 1 -- 172.21.15.106:0/2445026805 wait complete. 2023-11-24T08:13:29.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.975+0000 7f5a7f07c640 1 Processor -- start 2023-11-24T08:13:29.977 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.975+0000 7f5a7f07c640 1 -- start start 2023-11-24T08:13:29.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.976+0000 7f5a7f07c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a780ce710 0x7f5a781d2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.976+0000 7f5a7f07c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 0x7f5a781cec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.976+0000 7f5a7f07c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a780d22b0 0x7f5a780b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.978 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.976+0000 7f5a7f07c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780b8a20 con 0x7f5a780d1ed0 2023-11-24T08:13:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.976+0000 7f5a7f07c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a781d2540 con 0x7f5a780d22b0 2023-11-24T08:13:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.976+0000 7f5a7f07c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a780b7e10 con 0x7f5a780ce710 2023-11-24T08:13:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 0x7f5a781cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 0x7f5a781cec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59954/0 (socket says 172.21.15.106:59954) 2023-11-24T08:13:29.979 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 -- 172.21.15.106:0/3126433687 learned_addr learned my addr 172.21.15.106:0/3126433687 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a7ce18640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a780ce710 0x7f5a781d2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 -- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a780ce710 msgr2=0x7f5a781d2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.980 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a7d619640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a780d22b0 0x7f5a780b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a780ce710 0x7f5a781d2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 -- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a780d22b0 msgr2=0x7f5a780b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a780d22b0 0x7f5a780b1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.977+0000 7f5a77fff640 1 -- 172.21.15.106:0/3126433687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a70009b30 con 0x7f5a780d1ed0 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.978+0000 7f5a77fff640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 0x7f5a781cec60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d5f0 tx=0x7f5a6400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.978+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a640125f0 con 0x7f5a780d1ed0 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.978+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a64012750 con 0x7f5a780d1ed0 2023-11-24T08:13:29.981 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.978+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a64012b00 con 0x7f5a780d1ed0 2023-11-24T08:13:29.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.978+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a780b80f0 con 0x7f5a780d1ed0 2023-11-24T08:13:29.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.978+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a780b86b0 con 0x7f5a780d1ed0 2023-11-24T08:13:29.982 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.979+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a780b8b90 con 0x7f5a780d1ed0 2023-11-24T08:13:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.980+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f5a64012c60 con 0x7f5a780d1ed0 2023-11-24T08:13:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.981+0000 7f5a757fa640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a5c06b470 0x7f5a5c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.981+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5a6408a340 con 0x7f5a780d1ed0 2023-11-24T08:13:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.981+0000 7f5a7ce18640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a5c06b470 0x7f5a5c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:29.983 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.982+0000 7f5a7ce18640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a5c06b470 0x7f5a5c06d930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c003d90 tx=0x7f5a6c009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:29.986 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:29.985+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f5a64054da0 con 0x7f5a780d1ed0 2023-11-24T08:13:30.174 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.172+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a780b2fe0 con 0x7f5a5c06b470 2023-11-24T08:13:30.175 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.173+0000 7f5a757fa640 1 -- 172.21.15.106:0/3126433687 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5a6c003a30 con 0x7f5a5c06b470 2023-11-24T08:13:30.175 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 2 to deep-scrub 2023-11-24T08:13:30.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.175+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a5c06b470 msgr2=0x7f5a5c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.175+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a5c06b470 0x7f5a5c06d930 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c003d90 tx=0x7f5a6c009040 comp rx=0 tx=0).stop 2023-11-24T08:13:30.177 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 msgr2=0x7f5a781cec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 0x7f5a781cec60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f5a6400d5f0 tx=0x7f5a6400dab0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 shutdown_connections 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a780d22b0 0x7f5a780b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f5a780d1ed0 0x7f5a781cec60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f5a5c06b470 0x7f5a5c06d930 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 --2- 172.21.15.106:0/3126433687 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f5a780ce710 0x7f5a781d2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 >> 172.21.15.106:0/3126433687 conn(0x7f5a7808af60 msgr2=0x7f5a781e4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:30.178 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 shutdown_connections 2023-11-24T08:13:30.179 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.176+0000 7f5a7f07c640 1 -- 172.21.15.106:0/3126433687 wait complete. 2023-11-24T08:13:30.196 INFO:tasks.ceph:Scrubbing osd.3 2023-11-24T08:13:30.196 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.414+0000 7fe735fa9640 1 Processor -- start 2023-11-24T08:13:30.416 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.415+0000 7fe735fa9640 1 -- start start 2023-11-24T08:13:30.417 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.416+0000 7fe735fa9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe730157c70 0x7fe730161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.416+0000 7fe735fa9640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fe7300cf450 0x7fe7300cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.416+0000 7fe735fa9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe73015dc80 0x7fe73015e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.416+0000 7fe735fa9640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300cfe10 con 0x7fe73015dc80 2023-11-24T08:13:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.416+0000 7fe735fa9640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7300ce0b0 con 0x7fe7300cf450 2023-11-24T08:13:30.418 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.416+0000 7fe735fa9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe730161960 con 0x7fe730157c70 2023-11-24T08:13:30.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe73015dc80 0x7fe73015e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe73015dc80 0x7fe73015e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59956/0 (socket says 172.21.15.106:59956) 2023-11-24T08:13:30.419 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 -- 172.21.15.106:0/2902115521 learned_addr learned my addr 172.21.15.106:0/2902115521 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe727fff640 1 --2- 172.21.15.106:0/2902115521 >> v2:172.21.15.156:3300/0 conn(0x7fe7300cf450 0x7fe7300cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72f7fe640 1 --2- 172.21.15.106:0/2902115521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe730157c70 0x7fe730161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 -- 172.21.15.106:0/2902115521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe730157c70 msgr2=0x7fe730161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 --2- 172.21.15.106:0/2902115521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe730157c70 0x7fe730161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 -- 172.21.15.106:0/2902115521 >> v2:172.21.15.156:3300/0 conn(0x7fe7300cf450 msgr2=0x7fe7300cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 --2- 172.21.15.106:0/2902115521 >> v2:172.21.15.156:3300/0 conn(0x7fe7300cf450 0x7fe7300cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.420 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.417+0000 7fe72ffff640 1 -- 172.21.15.106:0/2902115521 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73015e650 con 0x7fe73015dc80 2023-11-24T08:13:30.421 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.418+0000 7fe72ffff640 1 --2- 172.21.15.106:0/2902115521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe73015dc80 0x7fe73015e110 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe73006b550 tx=0x7fe72802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc5338dd86562f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.421+0000 7fe72effd640 1 -- 172.21.15.106:0/2902115521 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72802c9e0 con 0x7fe73015dc80 2023-11-24T08:13:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.421+0000 7fe72effd640 1 -- 172.21.15.106:0/2902115521 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72802cb40 con 0x7fe73015dc80 2023-11-24T08:13:30.423 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.421+0000 7fe72effd640 1 -- 172.21.15.106:0/2902115521 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7280343f0 con 0x7fe73015dc80 2023-11-24T08:13:30.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 -- 172.21.15.106:0/2902115521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe73015dc80 msgr2=0x7fe73015e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 --2- 172.21.15.106:0/2902115521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe73015dc80 0x7fe73015e110 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fe73006b550 tx=0x7fe72802be40 comp rx=0 tx=0).stop 2023-11-24T08:13:30.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 -- 172.21.15.106:0/2902115521 shutdown_connections 2023-11-24T08:13:30.424 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 --2- 172.21.15.106:0/2902115521 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe73015dc80 0x7fe73015e110 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 --2- 172.21.15.106:0/2902115521 >> v2:172.21.15.156:3300/0 conn(0x7fe7300cf450 0x7fe7300cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 --2- 172.21.15.106:0/2902115521 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe730157c70 0x7fe730161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 -- 172.21.15.106:0/2902115521 >> 172.21.15.106:0/2902115521 conn(0x7fe73006a920 msgr2=0x7fe73006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:30.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 -- 172.21.15.106:0/2902115521 shutdown_connections 2023-11-24T08:13:30.425 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.422+0000 7fe735fa9640 1 -- 172.21.15.106:0/2902115521 wait complete. 2023-11-24T08:13:30.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.424+0000 7fe735fa9640 1 Processor -- start 2023-11-24T08:13:30.426 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.424+0000 7fe735fa9640 1 -- start start 2023-11-24T08:13:30.427 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.425+0000 7fe735fa9640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7300cf450 0x7fe73009cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.425+0000 7fe735fa9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe730157c70 0x7fe73009ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.425+0000 7fe735fa9640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe73015dc80 0x7fe7300ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.425+0000 7fe735fa9640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73009b250 con 0x7fe730157c70 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.425+0000 7fe735fa9640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300af7d0 con 0x7fe73015dc80 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.425+0000 7fe735fa9640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7300aebc0 con 0x7fe7300cf450 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe730157c70 0x7fe73009ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.428 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe730157c70 0x7fe73009ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59964/0 (socket says 172.21.15.106:59964) 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 -- 172.21.15.106:0/3106283118 learned_addr learned my addr 172.21.15.106:0/3106283118 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe72ffff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe73015dc80 0x7fe7300ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe72f7fe640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7300cf450 0x7fe73009cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 -- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7300cf450 msgr2=0x7fe73009cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7300cf450 0x7fe73009cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 -- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe73015dc80 msgr2=0x7fe7300ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe73015dc80 0x7fe7300ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.429 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe727fff640 1 -- 172.21.15.106:0/3106283118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe728009b30 con 0x7fe730157c70 2023-11-24T08:13:30.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe72f7fe640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe7300cf450 0x7fe73009cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:30.430 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.426+0000 7fe72ffff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe73015dc80 0x7fe7300ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:30.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.427+0000 7fe727fff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe730157c70 0x7fe73009ad10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe71800d5f0 tx=0x7fe71800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.427+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7180125f0 con 0x7fe730157c70 2023-11-24T08:13:30.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.427+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe718012750 con 0x7fe730157c70 2023-11-24T08:13:30.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.427+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7300aeea0 con 0x7fe730157c70 2023-11-24T08:13:30.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.427+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe718012b00 con 0x7fe730157c70 2023-11-24T08:13:30.431 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.427+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7300af460 con 0x7fe730157c70 2023-11-24T08:13:30.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.428+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe7300b4200 con 0x7fe730157c70 2023-11-24T08:13:30.432 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.429+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe718012c60 con 0x7fe730157c70 2023-11-24T08:13:30.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.430+0000 7fe72cff9640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe71006b470 0x7fe71006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.430+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe71808a300 con 0x7fe730157c70 2023-11-24T08:13:30.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.430+0000 7fe72f7fe640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe71006b470 0x7fe71006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.430+0000 7fe72cff9640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7fe7100712d0 0x7fe7100736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.430+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 --> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe710073da0 con 0x7fe7100712d0 2023-11-24T08:13:30.433 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.430+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe71808c830 con 0x7fe730157c70 2023-11-24T08:13:30.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.431+0000 7fe72ffff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7fe7100712d0 0x7fe7100736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.431+0000 7fe72f7fe640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe71006b470 0x7fe71006d930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b840 tx=0x7fe720008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.431+0000 7fe72ffff640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7fe7100712d0 0x7fe7100736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.434 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.432+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== osd.3 v2:172.21.15.106:6817/906310218 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe710073da0 con 0x7fe7100712d0 2023-11-24T08:13:30.448 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.447+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 --> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe730161310 con 0x7fe7100712d0 2023-11-24T08:13:30.449 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.447+0000 7fe72cff9640 1 -- 172.21.15.106:0/3106283118 <== osd.3 v2:172.21.15.106:6817/906310218 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe728040410 con 0x7fe7100712d0 2023-11-24T08:13:30.449 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:30.449 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:30.449 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:30.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.447+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7fe7100712d0 msgr2=0x7fe7100736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.450 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.447+0000 7fe735fa9640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6817/906310218,v1:172.21.15.106:6820/906310218] conn(0x7fe7100712d0 0x7fe7100736f0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.447+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe71006b470 msgr2=0x7fe71006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.447+0000 7fe735fa9640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe71006b470 0x7fe71006d930 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fe72000b840 tx=0x7fe720008040 comp rx=0 tx=0).stop 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe730157c70 msgr2=0x7fe73009ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe735fa9640 1 --2- 172.21.15.106:0/3106283118 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe730157c70 0x7fe73009ad10 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fe71800d5f0 tx=0x7fe71800dab0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe72ffff640 1 -- 172.21.15.106:0/3106283118 reap_dead start 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 shutdown_connections 2023-11-24T08:13:30.451 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 >> 172.21.15.106:0/3106283118 conn(0x7fe73006a920 msgr2=0x7fe73015edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:30.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 shutdown_connections 2023-11-24T08:13:30.452 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.448+0000 7fe735fa9640 1 -- 172.21.15.106:0/3106283118 wait complete. 2023-11-24T08:13:30.469 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-24T08:13:30.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.668+0000 7fc407fff640 1 Processor -- start 2023-11-24T08:13:30.670 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.669+0000 7fc407fff640 1 -- start start 2023-11-24T08:13:30.671 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.670+0000 7fc407fff640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc4080d08b0 0x7fc4080b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.670+0000 7fc407fff640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4080d1d80 con 0x7fc40815d5c0 2023-11-24T08:13:30.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.670+0000 7fc407fff640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4080d2060 con 0x7fc4080d1610 2023-11-24T08:13:30.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.670+0000 7fc407fff640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4080d2340 con 0x7fc4080d08b0 2023-11-24T08:13:30.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.670+0000 7fc406ffd640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fc40815d5c0 0x7fc4080d01a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58936/0 (socket says 172.21.15.106:58936) 2023-11-24T08:13:30.672 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.670+0000 7fc406ffd640 1 -- 172.21.15.106:0/536352758 learned_addr learned my addr 172.21.15.106:0/536352758 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:30.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933402091 0 0) 0x7fc4080d1d80 con 0x7fc40815d5c0 2023-11-24T08:13:30.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f4003520 con 0x7fc40815d5c0 2023-11-24T08:13:30.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc4067fc640 1 --2- 172.21.15.106:0/536352758 >> v2:172.21.15.156:3300/0 conn(0x7fc4080d08b0 0x7fc4080b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.674 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2432941738 0 0) 0x7fc3f4003520 con 0x7fc40815d5c0 2023-11-24T08:13:30.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 >> v2:172.21.15.156:3300/0 conn(0x7fc4080d08b0 msgr2=0x7fc4080b8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:30.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 --2- 172.21.15.106:0/536352758 >> v2:172.21.15.156:3300/0 conn(0x7fc4080d08b0 0x7fc4080b8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 >> v1:172.21.15.156:6789/0 conn(0x7fc4080d1610 legacy=0x7fc4080b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4080d2620 con 0x7fc40815d5c0 2023-11-24T08:13:30.675 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fc3f80019f0 con 0x7fc40815d5c0 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 >> v1:172.21.15.106:6789/0 conn(0x7fc40815d5c0 legacy=0x7fc4080d01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f40039c0 0x7fc3f4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40047c0 0x7fc3f4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.671+0000 7fc405ffb640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40052a0 0x7fc3f400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4080d2340 con 0x7fc3f40039c0 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f400c1c0 con 0x7fc3f40047c0 2023-11-24T08:13:30.676 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f400c380 con 0x7fc3f40052a0 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc406ffd640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f40039c0 0x7fc3f4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc406ffd640 1 -- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40052a0 msgr2=0x7fc3f400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc4077fe640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40052a0 0x7fc3f400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc4067fc640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40047c0 0x7fc3f4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc406ffd640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40052a0 0x7fc3f400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc406ffd640 1 -- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40047c0 msgr2=0x7fc3f4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc406ffd640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40047c0 0x7fc3f4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.677 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc406ffd640 1 -- 172.21.15.106:0/536352758 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f4003520 con 0x7fc3f40039c0 2023-11-24T08:13:30.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc4077fe640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40052a0 0x7fc3f400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:30.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc4067fc640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40047c0 0x7fc3f4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:30.678 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.672+0000 7fc4077fe640 1 -- 172.21.15.106:0/536352758 reap_dead start 2023-11-24T08:13:30.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.673+0000 7fc406ffd640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f40039c0 0x7fc3f4003de0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800a760 tx=0x7fc3f40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac31f1e2553990cb server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.673+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8034630 con 0x7fc3f40039c0 2023-11-24T08:13:30.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f802f070 con 0x7fc3f40039c0 2023-11-24T08:13:30.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc405ffb640 1 -- 172.21.15.106:0/536352758 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f8034a90 con 0x7fc3f40039c0 2023-11-24T08:13:30.681 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 -- 172.21.15.106:0/536352758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f40039c0 msgr2=0x7fc3f4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f40039c0 0x7fc3f4003de0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fc3f800a760 tx=0x7fc3f40057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 -- 172.21.15.106:0/536352758 shutdown_connections 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 --2- 172.21.15.106:0/536352758 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc3f40039c0 0x7fc3f4003de0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 -- 172.21.15.106:0/536352758 >> 172.21.15.106:0/536352758 conn(0x7fc408061310 msgr2=0x7fc408061720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 -- 172.21.15.106:0/536352758 shutdown_connections 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.674+0000 7fc407fff640 1 -- 172.21.15.106:0/536352758 wait complete. 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.676+0000 7fc407fff640 1 Processor -- start 2023-11-24T08:13:30.682 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.676+0000 7fc407fff640 1 -- start start 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.677+0000 7fc407fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40039c0 0x7fc40814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.677+0000 7fc407fff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 0x7fc40815f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.677+0000 7fc407fff640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc408157150 0x7fc4081575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.677+0000 7fc407fff640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40815fa60 con 0x7fc40815f0a0 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.677+0000 7fc407fff640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc408157e80 con 0x7fc408157150 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.677+0000 7fc407fff640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc408158160 con 0x7fc3f40039c0 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 0x7fc40815f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.683 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 0x7fc40815f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:59984/0 (socket says 172.21.15.106:59984) 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 -- 172.21.15.106:0/3738249510 learned_addr learned my addr 172.21.15.106:0/3738249510 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4077fe640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc408157150 0x7fc4081575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc406ffd640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40039c0 0x7fc40814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 -- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40039c0 msgr2=0x7fc40814d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40039c0 0x7fc40814d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 -- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc408157150 msgr2=0x7fc4081575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc408157150 0x7fc4081575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.684 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 -- 172.21.15.106:0/3738249510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408158440 con 0x7fc40815f0a0 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4077fe640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc408157150 0x7fc4081575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.678+0000 7fc4067fc640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 0x7fc40815f520 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e540 tx=0x7fc3f000ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.679+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0013610 con 0x7fc40815f0a0 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.679+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3f0013770 con 0x7fc40815f0a0 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.679+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc408158720 con 0x7fc40815f0a0 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.679+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0013b20 con 0x7fc40815f0a0 2023-11-24T08:13:30.685 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.679+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc408157b10 con 0x7fc40815f0a0 2023-11-24T08:13:30.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.680+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc40804e840 con 0x7fc40815f0a0 2023-11-24T08:13:30.686 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.681+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc3f0013c80 con 0x7fc40815f0a0 2023-11-24T08:13:30.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.682+0000 7fc3fffff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3e806b4a0 0x7fc3e806d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:30.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.682+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc3f0089ac0 con 0x7fc40815f0a0 2023-11-24T08:13:30.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.682+0000 7fc406ffd640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3e806b4a0 0x7fc3e806d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:30.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.683+0000 7fc406ffd640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3e806b4a0 0x7fc3e806d960 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8034000 tx=0x7fc3f802f000 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:30.687 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.686+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc3f0054470 con 0x7fc40815f0a0 2023-11-24T08:13:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.877+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4080d1a20 con 0x7fc3e806b4a0 2023-11-24T08:13:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.878+0000 7fc3fffff640 1 -- 172.21.15.106:0/3738249510 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc3f8034530 con 0x7fc3e806b4a0 2023-11-24T08:13:30.879 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 3 to deep-scrub 2023-11-24T08:13:30.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3e806b4a0 msgr2=0x7fc3e806d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3e806b4a0 0x7fc3e806d960 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8034000 tx=0x7fc3f802f000 comp rx=0 tx=0).stop 2023-11-24T08:13:30.883 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 msgr2=0x7fc40815f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:30.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 0x7fc40815f520 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000e540 tx=0x7fc3f000ea00 comp rx=0 tx=0).stop 2023-11-24T08:13:30.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 shutdown_connections 2023-11-24T08:13:30.884 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc408157150 0x7fc4081575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc40815f0a0 0x7fc40815f520 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc3e806b4a0 0x7fc3e806d960 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.885 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 --2- 172.21.15.106:0/3738249510 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc3f40039c0 0x7fc40814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:30.886 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.881+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 >> 172.21.15.106:0/3738249510 conn(0x7fc408061310 msgr2=0x7fc4080d3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:30.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.882+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 shutdown_connections 2023-11-24T08:13:30.887 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:30.882+0000 7fc407fff640 1 -- 172.21.15.106:0/3738249510 wait complete. 2023-11-24T08:13:30.900 INFO:tasks.ceph:Scrubbing osd.4 2023-11-24T08:13:30.901 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:31.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.107+0000 7ff908a9b640 1 Processor -- start 2023-11-24T08:13:31.109 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.108+0000 7ff908a9b640 1 -- start start 2023-11-24T08:13:31.110 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff908a9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff90415c7f0 0x7ff90406db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff908a9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9040cf450 0x7ff9040cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff908a9b640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7ff9040cdfb0 0x7ff9040ce440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff908a9b640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff904156010 con 0x7ff9040cf450 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff908a9b640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90406ce30 con 0x7ff9040cdfb0 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff908a9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9040cfe10 con 0x7ff90415c7f0 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff901d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9040cf450 0x7ff9040cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff901d9b640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff9040cf450 0x7ff9040cf8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60000/0 (socket says 172.21.15.106:60000) 2023-11-24T08:13:31.111 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff901d9b640 1 -- 172.21.15.106:0/3556167140 learned_addr learned my addr 172.21.15.106:0/3556167140 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.109+0000 7ff90259c640 1 --2- 172.21.15.106:0/3556167140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff90415c7f0 0x7ff90406db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.110+0000 7ff902d9d640 1 --2- 172.21.15.106:0/3556167140 >> v2:172.21.15.156:3300/0 conn(0x7ff9040cdfb0 0x7ff9040ce440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.110+0000 7ff901d9b640 1 -- 172.21.15.106:0/3556167140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff90415c7f0 msgr2=0x7ff90406db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.110+0000 7ff901d9b640 1 --2- 172.21.15.106:0/3556167140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff90415c7f0 0x7ff90406db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.110+0000 7ff901d9b640 1 -- 172.21.15.106:0/3556167140 >> v2:172.21.15.156:3300/0 conn(0x7ff9040cdfb0 msgr2=0x7ff9040ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.110+0000 7ff901d9b640 1 --2- 172.21.15.106:0/3556167140 >> v2:172.21.15.156:3300/0 conn(0x7ff9040cdfb0 0x7ff9040ce440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.110+0000 7ff901d9b640 1 -- 172.21.15.106:0/3556167140 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9040cece0 con 0x7ff9040cf450 2023-11-24T08:13:31.113 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.111+0000 7ff901d9b640 1 --2- 172.21.15.106:0/3556167140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9040cf450 0x7ff9040cf8d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff9040aebc0 tx=0x7ff8ec02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bed217702abc28f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.111+0000 7ff90159a640 1 -- 172.21.15.106:0/3556167140 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec031870 con 0x7ff9040cf450 2023-11-24T08:13:31.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.111+0000 7ff90159a640 1 -- 172.21.15.106:0/3556167140 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec0319d0 con 0x7ff9040cf450 2023-11-24T08:13:31.114 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.111+0000 7ff90159a640 1 -- 172.21.15.106:0/3556167140 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec031d60 con 0x7ff9040cf450 2023-11-24T08:13:31.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 -- 172.21.15.106:0/3556167140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9040cf450 msgr2=0x7ff9040cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3556167140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9040cf450 0x7ff9040cf8d0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ff9040aebc0 tx=0x7ff8ec02b910 comp rx=0 tx=0).stop 2023-11-24T08:13:31.115 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 -- 172.21.15.106:0/3556167140 shutdown_connections 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3556167140 >> v2:172.21.15.156:3300/0 conn(0x7ff9040cdfb0 0x7ff9040ce440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3556167140 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff9040cf450 0x7ff9040cf8d0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3556167140 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff90415c7f0 0x7ff90406db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 -- 172.21.15.106:0/3556167140 >> 172.21.15.106:0/3556167140 conn(0x7ff90406a920 msgr2=0x7ff90406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 -- 172.21.15.106:0/3556167140 shutdown_connections 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.112+0000 7ff908a9b640 1 -- 172.21.15.106:0/3556167140 wait complete. 2023-11-24T08:13:31.116 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.114+0000 7ff908a9b640 1 Processor -- start 2023-11-24T08:13:31.117 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.114+0000 7ff908a9b640 1 -- start start 2023-11-24T08:13:31.118 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff908a9b640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9040cdfb0 0x7ff9041546b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.118 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff908a9b640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9040cf450 0x7ff9041580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff908a9b640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff90415c7f0 0x7ff904158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff908a9b640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90409bad0 con 0x7ff90415c7f0 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff908a9b640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90409aec0 con 0x7ff9040cdfb0 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff908a9b640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90409b1a0 con 0x7ff9040cf450 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff90415c7f0 0x7ff904158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff902d9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff90415c7f0 0x7ff904158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60006/0 (socket says 172.21.15.106:60006) 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.115+0000 7ff902d9d640 1 -- 172.21.15.106:0/3337720577 learned_addr learned my addr 172.21.15.106:0/3337720577 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:31.119 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff901d9b640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9040cf450 0x7ff9041580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff90259c640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9040cdfb0 0x7ff9041546b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff902d9d640 1 -- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9040cf450 msgr2=0x7ff9041580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff902d9d640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff9040cf450 0x7ff9041580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff902d9d640 1 -- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9040cdfb0 msgr2=0x7ff9041546b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff902d9d640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9040cdfb0 0x7ff9041546b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff902d9d640 1 -- 172.21.15.106:0/3337720577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8ec0094d0 con 0x7ff90415c7f0 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff902d9d640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff90415c7f0 0x7ff904158600 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d580 tx=0x7ff8fc00da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.120 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc014620 con 0x7ff90415c7f0 2023-11-24T08:13:31.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8fc0147a0 con 0x7ff90415c7f0 2023-11-24T08:13:31.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8fc014b50 con 0x7ff90415c7f0 2023-11-24T08:13:31.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.116+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90409b480 con 0x7ff90415c7f0 2023-11-24T08:13:31.121 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.117+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff904154bf0 con 0x7ff90415c7f0 2023-11-24T08:13:31.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.118+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff90406d600 con 0x7ff90415c7f0 2023-11-24T08:13:31.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.119+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff8fc014cb0 con 0x7ff90415c7f0 2023-11-24T08:13:31.122 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.120+0000 7ff8faffd640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8e406b470 0x7ff8e406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.120+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff8fc08a200 con 0x7ff90415c7f0 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.120+0000 7ff90259c640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8e406b470 0x7ff8e406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.120+0000 7ff8faffd640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7ff8e40712d0 0x7ff8e40736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.120+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 --> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff8e4073da0 con 0x7ff8e40712d0 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.120+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff8fc08c6c0 con 0x7ff90415c7f0 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.121+0000 7ff90259c640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8e406b470 0x7ff8e406d930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40099c0 tx=0x7ff8f4003990 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.123 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.121+0000 7ff901d9b640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7ff8e40712d0 0x7ff8e40736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.124 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.122+0000 7ff901d9b640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7ff8e40712d0 0x7ff8e40736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.125 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.124+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== osd.4 v2:172.21.15.156:6800/2602399217 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff8e4073da0 con 0x7ff8e40712d0 2023-11-24T08:13:31.139 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.138+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 --> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff90406d320 con 0x7ff8e40712d0 2023-11-24T08:13:31.140 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff8faffd640 1 -- 172.21.15.106:0/3337720577 <== osd.4 v2:172.21.15.156:6800/2602399217 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff8ec031bc0 con 0x7ff8e40712d0 2023-11-24T08:13:31.141 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:31.141 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:31.141 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:31.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7ff8e40712d0 msgr2=0x7ff8e40736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.156:6800/2602399217,v1:172.21.15.156:6801/2602399217] conn(0x7ff8e40712d0 0x7ff8e40736f0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.142 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8e406b470 msgr2=0x7ff8e406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff8e406b470 0x7ff8e406d930 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7ff8f40099c0 tx=0x7ff8f4003990 comp rx=0 tx=0).stop 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff90415c7f0 msgr2=0x7ff904158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff908a9b640 1 --2- 172.21.15.106:0/3337720577 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff90415c7f0 0x7ff904158600 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00d580 tx=0x7ff8fc00da40 comp rx=0 tx=0).stop 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.139+0000 7ff902d9d640 1 -- 172.21.15.106:0/3337720577 reap_dead start 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.140+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 shutdown_connections 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.140+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 >> 172.21.15.106:0/3337720577 conn(0x7ff90406a920 msgr2=0x7ff9040aff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.140+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 shutdown_connections 2023-11-24T08:13:31.143 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.140+0000 7ff908a9b640 1 -- 172.21.15.106:0/3337720577 wait complete. 2023-11-24T08:13:31.158 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-24T08:13:31.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.358+0000 7fc55a30f640 1 Processor -- start 2023-11-24T08:13:31.360 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.359+0000 7fc55a30f640 1 -- start start 2023-11-24T08:13:31.361 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.360+0000 7fc55a30f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc554157c70 0x7fc554161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.360+0000 7fc55a30f640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc55415e390 0x7fc5540cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.360+0000 7fc55a30f640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5540ce0b0 con 0x7fc5540cf450 2023-11-24T08:13:31.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.360+0000 7fc55a30f640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5540cfe10 con 0x7fc55415e390 2023-11-24T08:13:31.362 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.360+0000 7fc55a30f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc554161960 con 0x7fc554157c70 2023-11-24T08:13:31.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.361+0000 7fc5588ac640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fc55415e390 0x7fc5540cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.361+0000 7fc553fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc554157c70 0x7fc554161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.361+0000 7fc553fff640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc554157c70 0x7fc554161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3301/0 says I am v2:172.21.15.106:57146/0 (socket says 172.21.15.106:57146) 2023-11-24T08:13:31.363 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.361+0000 7fc553fff640 1 -- 172.21.15.106:0/1074726552 learned_addr learned my addr 172.21.15.106:0/1074726552 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:31.364 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.362+0000 7fc5588ac640 1 -- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc554157c70 msgr2=0x7fc554161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:31.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.362+0000 7fc5588ac640 1 --2- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc554157c70 0x7fc554161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.362+0000 7fc5588ac640 1 -- 172.21.15.106:0/1074726552 >> v1:172.21.15.156:6789/0 conn(0x7fc5540cf450 legacy=0x7fc55415dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.366 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.362+0000 7fc5588ac640 1 -- 172.21.15.106:0/1074726552 --> v2:172.21.15.156:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55415e8d0 con 0x7fc55415e390 2023-11-24T08:13:31.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.363+0000 7fc5588ac640 1 --2- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc5540cf8d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009d90 tx=0x7fc54c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd6363059a698131 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.363+0000 7fc5537fe640 1 -- 172.21.15.106:0/1074726552 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c02cbc0 con 0x7fc55415e390 2023-11-24T08:13:31.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc5537fe640 1 -- 172.21.15.106:0/1074726552 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c02cd20 con 0x7fc55415e390 2023-11-24T08:13:31.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc5537fe640 1 -- 172.21.15.106:0/1074726552 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c034660 con 0x7fc55415e390 2023-11-24T08:13:31.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 -- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 msgr2=0x7fc5540cf8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.368 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 --2- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc5540cf8d0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fc54c009d90 tx=0x7fc54c02be10 comp rx=0 tx=0).stop 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 -- 172.21.15.106:0/1074726552 shutdown_connections 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 --2- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc5540cf8d0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 --2- 172.21.15.106:0/1074726552 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc554157c70 0x7fc554161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 -- 172.21.15.106:0/1074726552 >> 172.21.15.106:0/1074726552 conn(0x7fc55406a920 msgr2=0x7fc55406ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 -- 172.21.15.106:0/1074726552 shutdown_connections 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.364+0000 7fc55a30f640 1 -- 172.21.15.106:0/1074726552 wait complete. 2023-11-24T08:13:31.369 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.366+0000 7fc55a30f640 1 Processor -- start 2023-11-24T08:13:31.370 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.366+0000 7fc55a30f640 1 -- start start 2023-11-24T08:13:31.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc55a30f640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5540cf450 0x7fc554160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.371 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc55a30f640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 0x7fc55406c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc55a30f640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc554155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc55a30f640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5540cb660 con 0x7fc554157c70 2023-11-24T08:13:31.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc55a30f640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541f6d20 con 0x7fc55415e390 2023-11-24T08:13:31.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc55a30f640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5541f7000 con 0x7fc5540cf450 2023-11-24T08:13:31.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 0x7fc55406c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.372 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc54bfff640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 0x7fc55406c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60016/0 (socket says 172.21.15.106:60016) 2023-11-24T08:13:31.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.367+0000 7fc54bfff640 1 -- 172.21.15.106:0/2232231763 learned_addr learned my addr 172.21.15.106:0/2232231763 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:31.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc553fff640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5540cf450 0x7fc554160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.373 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc5588ac640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc554155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc54bfff640 1 -- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5540cf450 msgr2=0x7fc554160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc54bfff640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5540cf450 0x7fc554160b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc54bfff640 1 -- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 msgr2=0x7fc554155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc54bfff640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc554155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc54bfff640 1 -- 172.21.15.106:0/2232231763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c009a70 con 0x7fc554157c70 2023-11-24T08:13:31.374 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc5588ac640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc554155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc54bfff640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 0x7fc55406c7d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c400 tx=0x7fc53c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c0052d0 con 0x7fc554157c70 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc53c005430 con 0x7fc554157c70 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.368+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc53c003830 con 0x7fc554157c70 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.369+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5541f72e0 con 0x7fc554157c70 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.369+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5540ac060 con 0x7fc554157c70 2023-11-24T08:13:31.375 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.370+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc554067470 con 0x7fc554157c70 2023-11-24T08:13:31.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.371+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fc53c003990 con 0x7fc554157c70 2023-11-24T08:13:31.376 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.372+0000 7fc5517fa640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc53406b470 0x7fc53406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.372+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc53c0892e0 con 0x7fc554157c70 2023-11-24T08:13:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.372+0000 7fc553fff640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc53406b470 0x7fc53406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.373+0000 7fc553fff640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc53406b470 0x7fc53406d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc544009aa0 tx=0x7fc544008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.377 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.376+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fc53c053e50 con 0x7fc554157c70 2023-11-24T08:13:31.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.569+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5541615f0 con 0x7fc53406b470 2023-11-24T08:13:31.571 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.570+0000 7fc5517fa640 1 -- 172.21.15.106:0/2232231763 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc544010930 con 0x7fc53406b470 2023-11-24T08:13:31.572 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 4 to deep-scrub 2023-11-24T08:13:31.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.572+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc53406b470 msgr2=0x7fc53406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.572+0000 7fc55a30f640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc53406b470 0x7fc53406d930 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc544009aa0 tx=0x7fc544008040 comp rx=0 tx=0).stop 2023-11-24T08:13:31.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.572+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 msgr2=0x7fc55406c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.574 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.572+0000 7fc55a30f640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 0x7fc55406c7d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fc53c00c400 tx=0x7fc53c00c8c0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 shutdown_connections 2023-11-24T08:13:31.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc55415e390 0x7fc554155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fc554157c70 0x7fc55406c7d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fc53406b470 0x7fc53406d930 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 --2- 172.21.15.106:0/2232231763 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fc5540cf450 0x7fc554160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.575 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 >> 172.21.15.106:0/2232231763 conn(0x7fc55406a920 msgr2=0x7fc55406d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:31.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 shutdown_connections 2023-11-24T08:13:31.576 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.573+0000 7fc55a30f640 1 -- 172.21.15.106:0/2232231763 wait complete. 2023-11-24T08:13:31.593 INFO:tasks.ceph:Scrubbing osd.5 2023-11-24T08:13:31.593 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:31.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.806+0000 7f318473c640 1 Processor -- start 2023-11-24T08:13:31.808 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.807+0000 7f318473c640 1 -- start start 2023-11-24T08:13:31.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.808+0000 7f318473c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f317c15c7f0 0x7f317c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.808+0000 7f318473c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f317c0ce6c0 0x7f317c0cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.810 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.808+0000 7f318473c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f317c0cece0 con 0x7f317c0cf450 2023-11-24T08:13:31.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.808+0000 7f318473c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c0cfe40 con 0x7f317c0ce6c0 2023-11-24T08:13:31.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.808+0000 7f318473c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f317c0ceaf0 con 0x7f317c15c7f0 2023-11-24T08:13:31.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f317c0ce6c0 0x7f317c0cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f317c0ce6c0 0x7f317c0cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60028/0 (socket says 172.21.15.106:60028) 2023-11-24T08:13:31.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 -- 172.21.15.106:0/135889467 learned_addr learned my addr 172.21.15.106:0/135889467 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:31.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f31824d8640 1 --2- 172.21.15.106:0/135889467 >> v2:172.21.15.156:3300/0 conn(0x7f317c15c7f0 0x7f317c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 -- 172.21.15.106:0/135889467 >> v2:172.21.15.156:3300/0 conn(0x7f317c15c7f0 msgr2=0x7f317c06db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 --2- 172.21.15.106:0/135889467 >> v2:172.21.15.156:3300/0 conn(0x7f317c15c7f0 0x7f317c06db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 -- 172.21.15.106:0/135889467 >> v1:172.21.15.106:6789/0 conn(0x7f317c0cf450 legacy=0x7f317c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.812 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.809+0000 7f3182cd9640 1 -- 172.21.15.106:0/135889467 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c06d040 con 0x7f317c0ce6c0 2023-11-24T08:13:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.810+0000 7f3182cd9640 1 --2- 172.21.15.106:0/135889467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c0ce6c0 0x7f317c0cf900 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f317c0d0130 tx=0x7f317402bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b48184c7f4658028 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.810+0000 7f31814d6640 1 -- 172.21.15.106:0/135889467 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317402cb10 con 0x7f317c0ce6c0 2023-11-24T08:13:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f31814d6640 1 -- 172.21.15.106:0/135889467 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317402cc70 con 0x7f317c0ce6c0 2023-11-24T08:13:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f31814d6640 1 -- 172.21.15.106:0/135889467 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31740343f0 con 0x7f317c0ce6c0 2023-11-24T08:13:31.813 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f318473c640 1 -- 172.21.15.106:0/135889467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c0ce6c0 msgr2=0x7f317c0cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f318473c640 1 --2- 172.21.15.106:0/135889467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c0ce6c0 0x7f317c0cf900 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f317c0d0130 tx=0x7f317402bf70 comp rx=0 tx=0).stop 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f318473c640 1 -- 172.21.15.106:0/135889467 shutdown_connections 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f318473c640 1 --2- 172.21.15.106:0/135889467 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c0ce6c0 0x7f317c0cf900 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f318473c640 1 --2- 172.21.15.106:0/135889467 >> v2:172.21.15.156:3300/0 conn(0x7f317c15c7f0 0x7f317c06db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.811+0000 7f318473c640 1 -- 172.21.15.106:0/135889467 >> 172.21.15.106:0/135889467 conn(0x7f317c06a920 msgr2=0x7f317c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.812+0000 7f318473c640 1 -- 172.21.15.106:0/135889467 shutdown_connections 2023-11-24T08:13:31.814 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.812+0000 7f318473c640 1 -- 172.21.15.106:0/135889467 wait complete. 2023-11-24T08:13:31.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.814+0000 7f318473c640 1 Processor -- start 2023-11-24T08:13:31.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.814+0000 7f318473c640 1 -- start start 2023-11-24T08:13:31.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.815+0000 7f318473c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f317c0ce6c0 0x7f317c1f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.815+0000 7f318473c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f317c0cf450 0x7f317c0b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.815+0000 7f318473c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c15c7f0 0x7f317c0b28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.815+0000 7f318473c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c0b2480 con 0x7f317c15c7f0 2023-11-24T08:13:31.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.815+0000 7f318473c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f317c0b20e0 con 0x7f317c0cf450 2023-11-24T08:13:31.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.815+0000 7f318473c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c1f3a60 con 0x7f317c0ce6c0 2023-11-24T08:13:31.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c15c7f0 0x7f317c0b28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c15c7f0 0x7f317c0b28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60040/0 (socket says 172.21.15.106:60040) 2023-11-24T08:13:31.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 -- 172.21.15.106:0/1969182009 learned_addr learned my addr 172.21.15.106:0/1969182009 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:31.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f31824d8640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f317c0ce6c0 0x7f317c1f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3181cd7640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f317c0cf450 0x7f317c0b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 -- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f317c0ce6c0 msgr2=0x7f317c1f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f317c0ce6c0 0x7f317c1f31a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 -- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f317c0cf450 msgr2=0x7f317c0b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f317c0cf450 0x7f317c0b38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 -- 172.21.15.106:0/1969182009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3174009b30 con 0x7f317c15c7f0 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f31824d8640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f317c0ce6c0 0x7f317c1f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3181cd7640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f317c0cf450 0x7f317c0b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:31.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.816+0000 7f3182cd9640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c15c7f0 0x7f317c0b28a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f3174036040 tx=0x7f3174004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.817+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3174009510 con 0x7f317c15c7f0 2023-11-24T08:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.817+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317403e450 con 0x7f317c15c7f0 2023-11-24T08:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.817+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317403e800 con 0x7f317c15c7f0 2023-11-24T08:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.817+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f317c1f3d40 con 0x7f317c15c7f0 2023-11-24T08:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.817+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f317c1f36e0 con 0x7f317c15c7f0 2023-11-24T08:13:31.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.818+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f317c06e060 con 0x7f317c15c7f0 2023-11-24T08:13:31.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.819+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f317403e960 con 0x7f317c15c7f0 2023-11-24T08:13:31.824 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.820+0000 7f3172ffd640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f316006b470 0x7f316006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.820+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f31740acb20 con 0x7f317c15c7f0 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.820+0000 7f3172ffd640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7f31600712d0 0x7f31600736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.820+0000 7f31824d8640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f316006b470 0x7f316006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.820+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 --> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3160073da0 con 0x7f31600712d0 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.820+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7f31740b12a0 con 0x7f317c15c7f0 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.821+0000 7f31824d8640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f316006b470 0x7f316006d930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3178006fd0 tx=0x7f3178008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.821+0000 7f3181cd7640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7f31600712d0 0x7f31600736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:31.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.822+0000 7f3181cd7640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7f31600712d0 0x7f31600736f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:31.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.824+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== osd.5 v2:172.21.15.156:6816/3158094073 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7f3160073da0 con 0x7f31600712d0 2023-11-24T08:13:31.840 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.838+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 --> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f317c06d600 con 0x7f31600712d0 2023-11-24T08:13:31.841 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.839+0000 7f3172ffd640 1 -- 172.21.15.106:0/1969182009 <== osd.5 v2:172.21.15.156:6816/3158094073 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f316c0018b0 con 0x7f31600712d0 2023-11-24T08:13:31.841 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:31.841 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:31.841 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:31.842 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7f31600712d0 msgr2=0x7f31600736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.156:6816/3158094073,v1:172.21.15.156:6817/3158094073] conn(0x7f31600712d0 0x7f31600736f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f316006b470 msgr2=0x7f316006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f316006b470 0x7f316006d930 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f3178006fd0 tx=0x7f3178008040 comp rx=0 tx=0).stop 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c15c7f0 msgr2=0x7f317c0b28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 --2- 172.21.15.106:0/1969182009 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f317c15c7f0 0x7f317c0b28a0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f3174036040 tx=0x7f3174004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f3182cd9640 1 -- 172.21.15.106:0/1969182009 reap_dead start 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 shutdown_connections 2023-11-24T08:13:31.843 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 >> 172.21.15.106:0/1969182009 conn(0x7f317c06a920 msgr2=0x7f317c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:31.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.840+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 shutdown_connections 2023-11-24T08:13:31.844 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:31.841+0000 7f318473c640 1 -- 172.21.15.106:0/1969182009 wait complete. 2023-11-24T08:13:31.860 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-24T08:13:32.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.068+0000 7f7d0fad3640 1 Processor -- start 2023-11-24T08:13:32.070 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.069+0000 7f7d0fad3640 1 -- start start 2023-11-24T08:13:32.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0fad3640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f7d08157c70 0x7f7d08161df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0fad3640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d0815e390 0x7f7d080cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0fad3640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d080ce0b0 con 0x7f7d080cf450 2023-11-24T08:13:32.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0fad3640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d080cfe40 con 0x7f7d0815e390 2023-11-24T08:13:32.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0fad3640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d08161960 con 0x7f7d08157c70 2023-11-24T08:13:32.072 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0e070640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d0815e390 0x7f7d080cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0e070640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f7d0815e390 0x7f7d080cf900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60056/0 (socket says 172.21.15.106:60056) 2023-11-24T08:13:32.073 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.070+0000 7f7d0e070640 1 -- 172.21.15.106:0/738539413 learned_addr learned my addr 172.21.15.106:0/738539413 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.071+0000 7f7d0d86f640 1 --2- 172.21.15.106:0/738539413 >> v2:172.21.15.156:3300/0 conn(0x7f7d08157c70 0x7f7d08161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.071+0000 7f7d0e070640 1 -- 172.21.15.106:0/738539413 >> v2:172.21.15.156:3300/0 conn(0x7f7d08157c70 msgr2=0x7f7d08161df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.071+0000 7f7d0e070640 1 --2- 172.21.15.106:0/738539413 >> v2:172.21.15.156:3300/0 conn(0x7f7d08157c70 0x7f7d08161df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.071+0000 7f7d0e070640 1 -- 172.21.15.106:0/738539413 >> v1:172.21.15.156:6789/0 conn(0x7f7d080cf450 legacy=0x7f7d0815dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.071+0000 7f7d0e070640 1 -- 172.21.15.106:0/738539413 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d0815e8d0 con 0x7f7d0815e390 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.072+0000 7f7d0e070640 1 --2- 172.21.15.106:0/738539413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815e390 0x7f7d080cf900 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7d0806d5e0 tx=0x7f7d0002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a7c1c212a46a999 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.074 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.072+0000 7f7d0c86d640 1 -- 172.21.15.106:0/738539413 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0002c9f0 con 0x7f7d0815e390 2023-11-24T08:13:32.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.072+0000 7f7d0c86d640 1 -- 172.21.15.106:0/738539413 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0002cb50 con 0x7f7d0815e390 2023-11-24T08:13:32.075 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.072+0000 7f7d0c86d640 1 -- 172.21.15.106:0/738539413 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d000343f0 con 0x7f7d0815e390 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 -- 172.21.15.106:0/738539413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815e390 msgr2=0x7f7d080cf900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/738539413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815e390 0x7f7d080cf900 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7d0806d5e0 tx=0x7f7d0002be50 comp rx=0 tx=0).stop 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 -- 172.21.15.106:0/738539413 shutdown_connections 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/738539413 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d0815e390 0x7f7d080cf900 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/738539413 >> v2:172.21.15.156:3300/0 conn(0x7f7d08157c70 0x7f7d08161df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 -- 172.21.15.106:0/738539413 >> 172.21.15.106:0/738539413 conn(0x7f7d0806a920 msgr2=0x7f7d0806ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 -- 172.21.15.106:0/738539413 shutdown_connections 2023-11-24T08:13:32.076 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.073+0000 7f7d0fad3640 1 -- 172.21.15.106:0/738539413 wait complete. 2023-11-24T08:13:32.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.075+0000 7f7d0fad3640 1 Processor -- start 2023-11-24T08:13:32.077 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.075+0000 7f7d0fad3640 1 -- start start 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.076+0000 7f7d0fad3640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d080cf450 0x7f7d08099140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.076+0000 7f7d0fad3640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 0x7f7d08144840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.076+0000 7f7d0fad3640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d0815e390 0x7f7d0806d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.076+0000 7f7d0fad3640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08148180 con 0x7f7d08157c70 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.076+0000 7f7d0fad3640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08147570 con 0x7f7d080cf450 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.076+0000 7f7d0fad3640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d08147850 con 0x7f7d0815e390 2023-11-24T08:13:32.079 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 0x7f7d08144840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 0x7f7d08144840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60060/0 (socket says 172.21.15.106:60060) 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 -- 172.21.15.106:0/1673997615 learned_addr learned my addr 172.21.15.106:0/1673997615 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0e070640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d0815e390 0x7f7d0806d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d86f640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d080cf450 0x7f7d08099140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 -- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d0815e390 msgr2=0x7f7d0806d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d0815e390 0x7f7d0806d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 -- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d080cf450 msgr2=0x7f7d08099140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.080 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d080cf450 0x7f7d08099140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 -- 172.21.15.106:0/1673997615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00009b30 con 0x7f7d08157c70 2023-11-24T08:13:32.081 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.077+0000 7f7d0d06e640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 0x7f7d08144840 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00b3e0 tx=0x7f7cfc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.078+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc011830 con 0x7f7d08157c70 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.078+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cfc0119b0 con 0x7f7d08157c70 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.078+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc011d60 con 0x7f7d08157c70 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.078+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d08147b30 con 0x7f7d08157c70 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.078+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d08147200 con 0x7f7d08157c70 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.079+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d080b09f0 con 0x7f7d08157c70 2023-11-24T08:13:32.082 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.080+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f7cfc01a400 con 0x7f7d08157c70 2023-11-24T08:13:32.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.081+0000 7f7cfa7fc640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7cec06b470 0x7f7cec06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.081+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7cfc089fe0 con 0x7f7d08157c70 2023-11-24T08:13:32.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.081+0000 7f7d0d86f640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7cec06b470 0x7f7cec06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.083 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.082+0000 7f7d0d86f640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7cec06b470 0x7f7cec06d930 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7d04006fd0 tx=0x7f7d04008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.087 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.085+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f7cfc054aa0 con 0x7f7d08157c70 2023-11-24T08:13:32.279 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.277+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d081615f0 con 0x7f7cec06b470 2023-11-24T08:13:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.278+0000 7f7cfa7fc640 1 -- 172.21.15.106:0/1673997615 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7d0400b5e0 con 0x7f7cec06b470 2023-11-24T08:13:32.281 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 5 to deep-scrub 2023-11-24T08:13:32.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.280+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7cec06b470 msgr2=0x7f7cec06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.282 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.280+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7cec06b470 0x7f7cec06d930 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f7d04006fd0 tx=0x7f7d04008040 comp rx=0 tx=0).stop 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.280+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 msgr2=0x7f7d08144840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.280+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 0x7f7d08144840 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00b3e0 tx=0x7f7cfc00b8a0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 shutdown_connections 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f7cec06b470 0x7f7cec06d930 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f7d0815e390 0x7f7d0806d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f7d08157c70 0x7f7d08144840 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.283 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 --2- 172.21.15.106:0/1673997615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d080cf450 0x7f7d08099140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 >> 172.21.15.106:0/1673997615 conn(0x7f7d0806a920 msgr2=0x7f7d080b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 shutdown_connections 2023-11-24T08:13:32.284 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.281+0000 7f7d0fad3640 1 -- 172.21.15.106:0/1673997615 wait complete. 2023-11-24T08:13:32.300 INFO:tasks.ceph:Scrubbing osd.6 2023-11-24T08:13:32.300 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:32.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.520+0000 7ff9318ec640 1 Processor -- start 2023-11-24T08:13:32.522 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.521+0000 7ff9318ec640 1 -- start start 2023-11-24T08:13:32.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff9318ec640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff92c15c7f0 0x7ff92c06db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff9318ec640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c0cece0 con 0x7ff92c0cf450 2023-11-24T08:13:32.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff9318ec640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92c06d040 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.524 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff9318ec640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff92c0ceaf0 con 0x7ff92c15c7f0 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff92affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff92c15c7f0 0x7ff92c06db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff92affd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7ff92c15c7f0 0x7ff92c06db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60068/0 (socket says 172.21.15.106:60068) 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff92a7fc640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7ff92c0cf450 0x7ff92c0cdfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58942/0 (socket says 172.21.15.106:58942) 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.522+0000 7ff92affd640 1 -- 172.21.15.106:0/208274089 learned_addr learned my addr 172.21.15.106:0/208274089 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.523+0000 7ff92affd640 1 -- 172.21.15.106:0/208274089 >> v1:172.21.15.156:6789/0 conn(0x7ff92c0ce6c0 legacy=0x7ff92c0714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.523+0000 7ff92affd640 1 -- 172.21.15.106:0/208274089 >> v1:172.21.15.106:6789/0 conn(0x7ff92c0cf450 legacy=0x7ff92c0cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.525 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.523+0000 7ff92affd640 1 -- 172.21.15.106:0/208274089 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c06d320 con 0x7ff92c15c7f0 2023-11-24T08:13:32.526 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.524+0000 7ff92affd640 1 --2- 172.21.15.106:0/208274089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c15c7f0 0x7ff92c06db20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009e50 tx=0x7ff91c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bad4abc955649194 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.524+0000 7ff929ffb640 1 -- 172.21.15.106:0/208274089 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c02cb10 con 0x7ff92c15c7f0 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.524+0000 7ff929ffb640 1 -- 172.21.15.106:0/208274089 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c02cc70 con 0x7ff92c15c7f0 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.524+0000 7ff929ffb640 1 -- 172.21.15.106:0/208274089 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0343f0 con 0x7ff92c15c7f0 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 -- 172.21.15.106:0/208274089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c15c7f0 msgr2=0x7ff92c06db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 --2- 172.21.15.106:0/208274089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c15c7f0 0x7ff92c06db20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009e50 tx=0x7ff91c02bf70 comp rx=0 tx=0).stop 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 -- 172.21.15.106:0/208274089 shutdown_connections 2023-11-24T08:13:32.527 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 --2- 172.21.15.106:0/208274089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c15c7f0 0x7ff92c06db20 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 -- 172.21.15.106:0/208274089 >> 172.21.15.106:0/208274089 conn(0x7ff92c06a920 msgr2=0x7ff92c06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:32.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 -- 172.21.15.106:0/208274089 shutdown_connections 2023-11-24T08:13:32.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.525+0000 7ff9318ec640 1 -- 172.21.15.106:0/208274089 wait complete. 2023-11-24T08:13:32.528 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.527+0000 7ff9318ec640 1 Processor -- start 2023-11-24T08:13:32.529 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.527+0000 7ff9318ec640 1 -- start start 2023-11-24T08:13:32.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff9318ec640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c0ce6c0 0x7ff92c157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff9318ec640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff92c0cf450 0x7ff92c0ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff9318ec640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff92c15c7f0 0x7ff92c1480c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff9318ec640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ae0a0 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.530 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff9318ec640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ae530 con 0x7ff92c15c7f0 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff9318ec640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c0ad490 con 0x7ff92c0cf450 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c0ce6c0 0x7ff92c157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c0ce6c0 0x7ff92c157a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60080/0 (socket says 172.21.15.106:60080) 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 -- 172.21.15.106:0/2889858905 learned_addr learned my addr 172.21.15.106:0/2889858905 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 -- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff92c0cf450 msgr2=0x7ff92c0ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7ff92c0cf450 0x7ff92c0ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.531 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 -- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff92c15c7f0 msgr2=0x7ff92c1480c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff92c15c7f0 0x7ff92c1480c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.528+0000 7ff92affd640 1 -- 172.21.15.106:0/2889858905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c009b30 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.532 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.529+0000 7ff92affd640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c0ce6c0 0x7ff92c157a90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff91c036040 tx=0x7ff91c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.529+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c03e450 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.529+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c03e5f0 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.529+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c03e9a0 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.529+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c0ad770 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.529+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c0add30 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.533 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.530+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff92c06e060 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.531+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7ff91c03eb00 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.534 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.532+0000 7ff923fff640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff90c06f880 0x7ff90c071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.532+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ff91c0aced0 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.532+0000 7ff923fff640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ff90c0756e0 0x7ff90c077b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.532+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 --> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff90c0781b0 con 0x7ff90c0756e0 2023-11-24T08:13:32.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.532+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff91c0b2410 con 0x7ff92c0ce6c0 2023-11-24T08:13:32.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.532+0000 7ff92a7fc640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff90c06f880 0x7ff90c071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.535 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.533+0000 7ff92a7fc640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff90c06f880 0x7ff90c071d40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff9140015e0 tx=0x7ff9140079c0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.533+0000 7ff92b7fe640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ff90c0756e0 0x7ff90c077b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.536 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.534+0000 7ff92b7fe640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ff90c0756e0 0x7ff90c077b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.537 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.536+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== osd.6 v2:172.21.15.156:6808/2357036477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7ff90c0781b0 con 0x7ff90c0756e0 2023-11-24T08:13:32.552 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.551+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 --> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff92c0ae1e0 con 0x7ff90c0756e0 2023-11-24T08:13:32.553 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff923fff640 1 -- 172.21.15.106:0/2889858905 <== osd.6 v2:172.21.15.156:6808/2357036477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff9240050b0 con 0x7ff90c0756e0 2023-11-24T08:13:32.554 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:32.554 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:32.554 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:32.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ff90c0756e0 msgr2=0x7ff90c077b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.156:6808/2357036477,v1:172.21.15.156:6809/2357036477] conn(0x7ff90c0756e0 0x7ff90c077b00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.555 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff90c06f880 msgr2=0x7ff90c071d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7ff90c06f880 0x7ff90c071d40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff9140015e0 tx=0x7ff9140079c0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c0ce6c0 msgr2=0x7ff92c157a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 --2- 172.21.15.106:0/2889858905 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7ff92c0ce6c0 0x7ff92c157a90 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ff91c036040 tx=0x7ff91c004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff92b7fe640 1 -- 172.21.15.106:0/2889858905 reap_dead start 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 shutdown_connections 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.552+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 >> 172.21.15.106:0/2889858905 conn(0x7ff92c06a920 msgr2=0x7ff92c0afe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.553+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 shutdown_connections 2023-11-24T08:13:32.556 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.553+0000 7ff9318ec640 1 -- 172.21.15.106:0/2889858905 wait complete. 2023-11-24T08:13:32.571 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-24T08:13:32.774 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.773+0000 7f82a52d0640 1 Processor -- start 2023-11-24T08:13:32.775 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.773+0000 7f82a52d0640 1 -- start start 2023-11-24T08:13:32.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.774+0000 7f82a52d0640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82a015c7f0 0x7f82a006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.774+0000 7f82a52d0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cf450 0x7f82a00cf8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.774+0000 7f82a52d0640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82a00cece0 con 0x7f82a00cdfb0 2023-11-24T08:13:32.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.774+0000 7f82a52d0640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82a0156010 con 0x7f82a015c7f0 2023-11-24T08:13:32.776 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.774+0000 7f82a52d0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a006ce30 con 0x7f82a00cf450 2023-11-24T08:13:32.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829effd640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f82a015c7f0 0x7f82a006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829f7fe640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7f82a00cdfb0 0x7f82a00ce440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58948/0 (socket says 172.21.15.106:58948) 2023-11-24T08:13:32.777 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829f7fe640 1 -- 172.21.15.106:0/4145438741 learned_addr learned my addr 172.21.15.106:0/4145438741 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:32.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f82967fc640 1 --2- 172.21.15.106:0/4145438741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cf450 0x7f82a00cf8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829effd640 1 -- 172.21.15.106:0/4145438741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cf450 msgr2=0x7f82a00cf8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:32.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829effd640 1 --2- 172.21.15.106:0/4145438741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cf450 0x7f82a00cf8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.778 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829effd640 1 -- 172.21.15.106:0/4145438741 >> v1:172.21.15.106:6789/0 conn(0x7f82a00cdfb0 legacy=0x7f82a00ce440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829effd640 1 -- 172.21.15.106:0/4145438741 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a006d040 con 0x7f82a015c7f0 2023-11-24T08:13:32.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.775+0000 7f829e7fc640 1 -- 172.21.15.106:0/4145438741 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1230954830 0 0) 0x7f82a00cece0 con 0x7f82a00cdfb0 2023-11-24T08:13:32.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.776+0000 7f829effd640 1 --2- 172.21.15.106:0/4145438741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a006db20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f82880097a0 tx=0x7f828802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a7c422b3dd0fb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f829e7fc640 1 -- 172.21.15.106:0/4145438741 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288031830 con 0x7f82a015c7f0 2023-11-24T08:13:32.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f829e7fc640 1 -- 172.21.15.106:0/4145438741 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8288031990 con 0x7f82a015c7f0 2023-11-24T08:13:32.779 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f829e7fc640 1 -- 172.21.15.106:0/4145438741 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288031d20 con 0x7f82a015c7f0 2023-11-24T08:13:32.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 -- 172.21.15.106:0/4145438741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 msgr2=0x7f82a006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.780 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 --2- 172.21.15.106:0/4145438741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a006db20 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f82880097a0 tx=0x7f828802b8d0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 -- 172.21.15.106:0/4145438741 shutdown_connections 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 --2- 172.21.15.106:0/4145438741 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cf450 0x7f82a00cf8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 --2- 172.21.15.106:0/4145438741 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a006db20 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 -- 172.21.15.106:0/4145438741 >> 172.21.15.106:0/4145438741 conn(0x7f82a006a920 msgr2=0x7f82a006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 -- 172.21.15.106:0/4145438741 shutdown_connections 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.777+0000 7f82a52d0640 1 -- 172.21.15.106:0/4145438741 wait complete. 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.779+0000 7f82a52d0640 1 Processor -- start 2023-11-24T08:13:32.781 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.779+0000 7f82a52d0640 1 -- start start 2023-11-24T08:13:32.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f82a52d0640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cdfb0 0x7f82a0156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.783 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f82a52d0640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00cf450 0x7f82a0071b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f82a52d0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a01f35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f82a52d0640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0158070 con 0x7f82a015c7f0 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f82a52d0640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0157460 con 0x7f82a00cf450 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f82a52d0640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82a0157740 con 0x7f82a00cdfb0 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a01f35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f829f7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a01f35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60098/0 (socket says 172.21.15.106:60098) 2023-11-24T08:13:32.784 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.780+0000 7f829f7fe640 1 -- 172.21.15.106:0/1096626727 learned_addr learned my addr 172.21.15.106:0/1096626727 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829effd640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cdfb0 0x7f82a0156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829f7fe640 1 -- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cdfb0 msgr2=0x7f82a0156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f82967fc640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00cf450 0x7f82a0071b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829f7fe640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cdfb0 0x7f82a0156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829f7fe640 1 -- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00cf450 msgr2=0x7f82a0071b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829f7fe640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00cf450 0x7f82a0071b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829f7fe640 1 -- 172.21.15.106:0/1096626727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009400 con 0x7f82a015c7f0 2023-11-24T08:13:32.785 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f829f7fe640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a01f35c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8298002230 tx=0x7f829800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829800ce30 con 0x7f82a015c7f0 2023-11-24T08:13:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829800a640 con 0x7f82a015c7f0 2023-11-24T08:13:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829800a9f0 con 0x7f82a015c7f0 2023-11-24T08:13:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.781+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a0157a20 con 0x7f82a015c7f0 2023-11-24T08:13:32.786 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.782+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a0157050 con 0x7f82a015c7f0 2023-11-24T08:13:32.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.783+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a01f0c90 con 0x7f82a015c7f0 2023-11-24T08:13:32.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.784+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f829800ab50 con 0x7f82a015c7f0 2023-11-24T08:13:32.787 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.784+0000 7f8297fff640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f828006b470 0x7f828006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.785+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f82980892d0 con 0x7f82a015c7f0 2023-11-24T08:13:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.785+0000 7f829effd640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f828006b470 0x7f828006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:32.788 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.785+0000 7f829effd640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f828006b470 0x7f828006d930 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8288036000 tx=0x7f828803c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:32.792 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.791+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f8298053dd0 con 0x7f82a015c7f0 2023-11-24T08:13:32.984 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.982+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f82a0158220 con 0x7f828006b470 2023-11-24T08:13:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.983+0000 7f8297fff640 1 -- 172.21.15.106:0/1096626727 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f8288031b00 con 0x7f828006b470 2023-11-24T08:13:32.985 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 6 to deep-scrub 2023-11-24T08:13:32.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f828006b470 msgr2=0x7f828006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f828006b470 0x7f828006d930 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f8288036000 tx=0x7f828803c040 comp rx=0 tx=0).stop 2023-11-24T08:13:32.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 msgr2=0x7f82a01f35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:32.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a01f35c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f8298002230 tx=0x7f829800cbe0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 shutdown_connections 2023-11-24T08:13:32.988 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f82a015c7f0 0x7f82a01f35c0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f82a00cf450 0x7f82a0071b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f828006b470 0x7f828006d930 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 --2- 172.21.15.106:0/1096626727 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f82a00cdfb0 0x7f82a0156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:32.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 >> 172.21.15.106:0/1096626727 conn(0x7f82a006a920 msgr2=0x7f82a00ce420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:32.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 shutdown_connections 2023-11-24T08:13:32.989 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:32.986+0000 7f82a52d0640 1 -- 172.21.15.106:0/1096626727 wait complete. 2023-11-24T08:13:33.006 INFO:tasks.ceph:Scrubbing osd.7 2023-11-24T08:13:33.007 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-24T08:13:33.222 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.221+0000 7fe1b6168640 1 Processor -- start 2023-11-24T08:13:33.223 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.221+0000 7fe1b6168640 1 -- start start 2023-11-24T08:13:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.222+0000 7fe1b6168640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1b015c7f0 0x7fe1b006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.222+0000 7fe1b6168640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b00cece0 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.222+0000 7fe1b6168640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b006d040 con 0x7fe1b00cf450 2023-11-24T08:13:33.224 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.222+0000 7fe1b6168640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b00ceaf0 con 0x7fe1b015c7f0 2023-11-24T08:13:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1af7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1b015c7f0 0x7fe1b006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1af7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fe1b015c7f0 0x7fe1b006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60114/0 (socket says 172.21.15.106:60114) 2023-11-24T08:13:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1affff640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fe1b00ce6c0 0x7fe1b00714d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:58956/0 (socket says 172.21.15.106:58956) 2023-11-24T08:13:33.225 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1af7fe640 1 -- 172.21.15.106:0/685656111 learned_addr learned my addr 172.21.15.106:0/685656111 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1af7fe640 1 -- 172.21.15.106:0/685656111 >> v1:172.21.15.156:6789/0 conn(0x7fe1b00cf450 legacy=0x7fe1b00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1af7fe640 1 -- 172.21.15.106:0/685656111 >> v1:172.21.15.106:6789/0 conn(0x7fe1b00ce6c0 legacy=0x7fe1b00714d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.223+0000 7fe1af7fe640 1 -- 172.21.15.106:0/685656111 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b006d320 con 0x7fe1b015c7f0 2023-11-24T08:13:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.224+0000 7fe1af7fe640 1 --2- 172.21.15.106:0/685656111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b015c7f0 0x7fe1b006db20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009e50 tx=0x7fe1a002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a3a1fa71325896 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.226 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.224+0000 7fe1ae7fc640 1 -- 172.21.15.106:0/685656111 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a002cb10 con 0x7fe1b015c7f0 2023-11-24T08:13:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1ae7fc640 1 -- 172.21.15.106:0/685656111 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a002cc70 con 0x7fe1b015c7f0 2023-11-24T08:13:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1ae7fc640 1 -- 172.21.15.106:0/685656111 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a00343f0 con 0x7fe1b015c7f0 2023-11-24T08:13:33.227 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 -- 172.21.15.106:0/685656111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b015c7f0 msgr2=0x7fe1b006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 --2- 172.21.15.106:0/685656111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b015c7f0 0x7fe1b006db20 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009e50 tx=0x7fe1a002bf70 comp rx=0 tx=0).stop 2023-11-24T08:13:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 -- 172.21.15.106:0/685656111 shutdown_connections 2023-11-24T08:13:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 --2- 172.21.15.106:0/685656111 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b015c7f0 0x7fe1b006db20 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 -- 172.21.15.106:0/685656111 >> 172.21.15.106:0/685656111 conn(0x7fe1b006a920 msgr2=0x7fe1b006ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 -- 172.21.15.106:0/685656111 shutdown_connections 2023-11-24T08:13:33.228 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.225+0000 7fe1b6168640 1 -- 172.21.15.106:0/685656111 wait complete. 2023-11-24T08:13:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.227+0000 7fe1b6168640 1 Processor -- start 2023-11-24T08:13:33.229 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.227+0000 7fe1b6168640 1 -- start start 2023-11-24T08:13:33.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1b6168640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b00ce6c0 0x7fe1b0158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1b6168640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1b00cf450 0x7fe1b00adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.230 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1b6168640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1b015c7f0 0x7fe1b00ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1b6168640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01f1ea0 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1b6168640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b009b520 con 0x7fe1b00cf450 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1b6168640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b01f5420 con 0x7fe1b015c7f0 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1af7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b00ce6c0 0x7fe1b0158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1af7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b00ce6c0 0x7fe1b0158ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60116/0 (socket says 172.21.15.106:60116) 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.228+0000 7fe1af7fe640 1 -- 172.21.15.106:0/2434208233 learned_addr learned my addr 172.21.15.106:0/2434208233 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:33.231 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1af7fe640 1 -- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1b015c7f0 msgr2=0x7fe1b00ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1af7fe640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fe1b015c7f0 0x7fe1b00ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1af7fe640 1 -- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1b00cf450 msgr2=0x7fe1b00adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1af7fe640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1b00cf450 0x7fe1b00adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1af7fe640 1 -- 172.21.15.106:0/2434208233 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a0009b30 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1af7fe640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b00ce6c0 0x7fe1b0158ec0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009e50 tx=0x7fe1a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a003e450 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a003e5f0 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.233 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a003e9a0 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.229+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b01f1a10 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.230+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b01f13c0 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.230+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1b006e060 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.234 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.232+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fe1a003eb00 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.232+0000 7fe1a7fff640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe19006b470 0x7fe19006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.233+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe1a00ace60 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.233+0000 7fe1aeffd640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe19006b470 0x7fe19006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.233+0000 7fe1a7fff640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fe1900712d0 0x7fe1900736f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.233+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 --> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe190073da0 con 0x7fe1900712d0 2023-11-24T08:13:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.233+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_get_version_reply(handle=1 version=92) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1a00b0020 con 0x7fe1b00ce6c0 2023-11-24T08:13:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.234+0000 7fe1affff640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fe1900712d0 0x7fe1900736f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.234+0000 7fe1aeffd640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe19006b470 0x7fe19006d930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00998d0 tx=0x7fe1980079b0 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.235+0000 7fe1affff640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fe1900712d0 0x7fe1900736f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.237+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== osd.7 v2:172.21.15.156:6824/1912715038 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+26965 (crc 0 0 0) 0x7fe190073da0 con 0x7fe1900712d0 2023-11-24T08:13:33.252 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.251+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 --> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fe1b0067470 con 0x7fe1900712d0 2023-11-24T08:13:33.253 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.252+0000 7fe1a7fff640 1 -- 172.21.15.106:0/2434208233 <== osd.7 v2:172.21.15.156:6824/1912715038 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fe1b0067470 con 0x7fe1900712d0 2023-11-24T08:13:33.254 INFO:teuthology.orchestra.run.smithi106.stdout:{ 2023-11-24T08:13:33.254 INFO:teuthology.orchestra.run.smithi106.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2023-11-24T08:13:33.254 INFO:teuthology.orchestra.run.smithi106.stdout:} 2023-11-24T08:13:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fe1900712d0 msgr2=0x7fe1900736f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.156:6824/1912715038,v1:172.21.15.156:6825/1912715038] conn(0x7fe1900712d0 0x7fe1900736f0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe19006b470 msgr2=0x7fe19006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fe19006b470 0x7fe19006d930 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00998d0 tx=0x7fe1980079b0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.255 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b00ce6c0 msgr2=0x7fe1b0158ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 --2- 172.21.15.106:0/2434208233 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fe1b00ce6c0 0x7fe1b0158ec0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fe1a0009e50 tx=0x7fe1a0004a40 comp rx=0 tx=0).stop 2023-11-24T08:13:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1affff640 1 -- 172.21.15.106:0/2434208233 reap_dead start 2023-11-24T08:13:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 shutdown_connections 2023-11-24T08:13:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 >> 172.21.15.106:0/2434208233 conn(0x7fe1b006a920 msgr2=0x7fe1b015cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 shutdown_connections 2023-11-24T08:13:33.256 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.253+0000 7fe1b6168640 1 -- 172.21.15.106:0/2434208233 wait complete. 2023-11-24T08:13:33.272 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-24T08:13:33.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.476+0000 7f6e910e7640 1 Processor -- start 2023-11-24T08:13:33.478 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.477+0000 7f6e910e7640 1 -- start start 2023-11-24T08:13:33.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.478+0000 7f6e910e7640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e8c15eaf0 0x7f6e8c0b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.478+0000 7f6e910e7640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f6e8c0b3890 0x7f6e8c167110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.478+0000 7f6e910e7640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8c06d180 con 0x7f6e8c1461d0 2023-11-24T08:13:33.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.478+0000 7f6e910e7640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c0b3d20 con 0x7f6e8c15eaf0 2023-11-24T08:13:33.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.478+0000 7f6e910e7640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8c06ba70 con 0x7f6e8c0b3890 2023-11-24T08:13:33.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.478+0000 7f6e8a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e8c15eaf0 0x7f6e8c0b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8a59c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f6e8c15eaf0 0x7f6e8c0b3350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60118/0 (socket says 172.21.15.106:60118) 2023-11-24T08:13:33.481 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8a59c640 1 -- 172.21.15.106:0/3105914860 learned_addr learned my addr 172.21.15.106:0/3105914860 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:33.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8a59c640 1 -- 172.21.15.106:0/3105914860 >> v2:172.21.15.156:3300/0 conn(0x7f6e8c0b3890 msgr2=0x7f6e8c167110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:33.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8b59e640 1 --2- 172.21.15.106:0/3105914860 >> v2:172.21.15.156:3300/0 conn(0x7f6e8c0b3890 0x7f6e8c167110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8a59c640 1 --2- 172.21.15.106:0/3105914860 >> v2:172.21.15.156:3300/0 conn(0x7f6e8c0b3890 0x7f6e8c167110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8a59c640 1 -- 172.21.15.106:0/3105914860 >> v1:172.21.15.156:6789/0 conn(0x7f6e8c1461d0 legacy=0x7f6e8c0b4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:33.482 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.479+0000 7f6e8a59c640 1 -- 172.21.15.106:0/3105914860 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c06bd50 con 0x7f6e8c15eaf0 2023-11-24T08:13:33.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.480+0000 7f6e8a59c640 1 --2- 172.21.15.106:0/3105914860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c15eaf0 0x7f6e8c0b3350 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b5d30 tx=0x7f6e7402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f0a7e9877fced6 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.480+0000 7f6e89d9b640 1 -- 172.21.15.106:0/3105914860 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74031870 con 0x7f6e8c15eaf0 2023-11-24T08:13:33.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e89d9b640 1 -- 172.21.15.106:0/3105914860 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e740319d0 con 0x7f6e8c15eaf0 2023-11-24T08:13:33.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e89d9b640 1 -- 172.21.15.106:0/3105914860 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e74031d60 con 0x7f6e8c15eaf0 2023-11-24T08:13:33.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 -- 172.21.15.106:0/3105914860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c15eaf0 msgr2=0x7f6e8c0b3350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 --2- 172.21.15.106:0/3105914860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c15eaf0 0x7f6e8c0b3350 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c0b5d30 tx=0x7f6e7402b910 comp rx=0 tx=0).stop 2023-11-24T08:13:33.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 -- 172.21.15.106:0/3105914860 shutdown_connections 2023-11-24T08:13:33.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 --2- 172.21.15.106:0/3105914860 >> v2:172.21.15.156:3300/0 conn(0x7f6e8c0b3890 0x7f6e8c167110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 --2- 172.21.15.106:0/3105914860 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c15eaf0 0x7f6e8c0b3350 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 -- 172.21.15.106:0/3105914860 >> 172.21.15.106:0/3105914860 conn(0x7f6e8c061340 msgr2=0x7f6e8c061750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:33.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 -- 172.21.15.106:0/3105914860 shutdown_connections 2023-11-24T08:13:33.485 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.481+0000 7f6e910e7640 1 -- 172.21.15.106:0/3105914860 wait complete. 2023-11-24T08:13:33.486 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.483+0000 7f6e910e7640 1 Processor -- start 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.483+0000 7f6e910e7640 1 -- start start 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.484+0000 7f6e910e7640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 0x7f6e8c14f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.484+0000 7f6e910e7640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c1461d0 0x7f6e8c1f5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.484+0000 7f6e910e7640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e8c15eaf0 0x7f6e8c13f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.484+0000 7f6e910e7640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c14d710 con 0x7f6e8c0b3890 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.484+0000 7f6e910e7640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c14d130 con 0x7f6e8c1461d0 2023-11-24T08:13:33.487 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.484+0000 7f6e910e7640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c13c910 con 0x7f6e8c15eaf0 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 0x7f6e8c14f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 0x7f6e8c14f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:60120/0 (socket says 172.21.15.106:60120) 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 -- 172.21.15.106:0/357688505 learned_addr learned my addr 172.21.15.106:0/357688505 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8a59c640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c1461d0 0x7f6e8c1f5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 -- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e8c15eaf0 msgr2=0x7f6e8c13f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e8c15eaf0 0x7f6e8c13f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.488 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 -- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c1461d0 msgr2=0x7f6e8c1f5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c1461d0 0x7f6e8c1f5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 -- 172.21.15.106:0/357688505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e740094d0 con 0x7f6e8c0b3890 2023-11-24T08:13:33.489 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.485+0000 7f6e8ad9d640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 0x7f6e8c14f9d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00c7d0 tx=0x7f6e7c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.486+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c003d80 con 0x7f6e8c0b3890 2023-11-24T08:13:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.486+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c005070 con 0x7f6e8c0b3890 2023-11-24T08:13:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.486+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c005420 con 0x7f6e8c0b3890 2023-11-24T08:13:33.490 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.486+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e8c13cba0 con 0x7f6e8c0b3890 2023-11-24T08:13:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.486+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e8c13ce80 con 0x7f6e8c0b3890 2023-11-24T08:13:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.487+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8c067470 con 0x7f6e8c0b3890 2023-11-24T08:13:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.488+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f6e7c005580 con 0x7f6e8c0b3890 2023-11-24T08:13:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.489+0000 7f6e837fe640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e6c06b470 0x7f6e6c06d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.491 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.489+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6e7c089290 con 0x7f6e8c0b3890 2023-11-24T08:13:33.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.489+0000 7f6e8a59c640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e6c06b470 0x7f6e6c06d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.492 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.490+0000 7f6e8a59c640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e6c06b470 0x7f6e6c06d930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e74036000 tx=0x7f6e7403c040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.495 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.494+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f6e7c053d50 con 0x7f6e8c0b3890 2023-11-24T08:13:33.689 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.688+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e8c06c5f0 con 0x7f6e6c06b470 2023-11-24T08:13:33.690 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.689+0000 7f6e837fe640 1 -- 172.21.15.106:0/357688505 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6e74031cc0 con 0x7f6e6c06b470 2023-11-24T08:13:33.691 INFO:teuthology.orchestra.run.smithi106.stderr:instructed osd(s) 7 to deep-scrub 2023-11-24T08:13:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e6c06b470 msgr2=0x7f6e6c06d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e6c06b470 0x7f6e6c06d930 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f6e74036000 tx=0x7f6e7403c040 comp rx=0 tx=0).stop 2023-11-24T08:13:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 msgr2=0x7f6e8c14f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 0x7f6e8c14f9d0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00c7d0 tx=0x7f6e7c00cc90 comp rx=0 tx=0).stop 2023-11-24T08:13:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 shutdown_connections 2023-11-24T08:13:33.694 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f6e6c06b470 0x7f6e6c06d930 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f6e8c15eaf0 0x7f6e8c13f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6e8c1461d0 0x7f6e8c1f5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 --2- 172.21.15.106:0/357688505 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f6e8c0b3890 0x7f6e8c14f9d0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 >> 172.21.15.106:0/357688505 conn(0x7f6e8c061340 msgr2=0x7f6e8c1605c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:33.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 shutdown_connections 2023-11-24T08:13:33.695 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.692+0000 7f6e910e7640 1 -- 172.21.15.106:0/357688505 wait complete. 2023-11-24T08:13:33.712 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:33.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.926+0000 7fb355a02640 1 Processor -- start 2023-11-24T08:13:33.927 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.926+0000 7fb355a02640 1 -- start start 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb355a02640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7fb35006c4f0 0x7fb3500abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb355a02640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3500ac400 0x7fb350070aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb355a02640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3500ac830 con 0x7fb3500ce290 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb355a02640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3500ce670 con 0x7fb3500ac400 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb355a02640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb35015bcb0 con 0x7fb35006c4f0 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb34f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3500ac400 0x7fb350070aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb34f7fe640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7fb3500ac400 0x7fb350070aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36190/0 (socket says 172.21.15.106:36190) 2023-11-24T08:13:33.930 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb34f7fe640 1 -- 172.21.15.106:0/696165089 learned_addr learned my addr 172.21.15.106:0/696165089 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:33.931 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.928+0000 7fb3467fc640 1 --2- 172.21.15.106:0/696165089 >> v2:172.21.15.156:3300/0 conn(0x7fb35006c4f0 0x7fb3500abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.929+0000 7fb34f7fe640 1 -- 172.21.15.106:0/696165089 >> v2:172.21.15.156:3300/0 conn(0x7fb35006c4f0 msgr2=0x7fb3500abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.929+0000 7fb34f7fe640 1 --2- 172.21.15.106:0/696165089 >> v2:172.21.15.156:3300/0 conn(0x7fb35006c4f0 0x7fb3500abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.929+0000 7fb34f7fe640 1 -- 172.21.15.106:0/696165089 >> v1:172.21.15.156:6789/0 conn(0x7fb3500ce290 legacy=0x7fb3500ae090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.929+0000 7fb34f7fe640 1 -- 172.21.15.106:0/696165089 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb35015bf90 con 0x7fb3500ac400 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.930+0000 7fb34f7fe640 1 --2- 172.21.15.106:0/696165089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3500ac400 0x7fb350070aa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb35006e7e0 tx=0x7fb34802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc4ffc8bfd847dd server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.930+0000 7fb34e7fc640 1 -- 172.21.15.106:0/696165089 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348031890 con 0x7fb3500ac400 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.930+0000 7fb34e7fc640 1 -- 172.21.15.106:0/696165089 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3480319f0 con 0x7fb3500ac400 2023-11-24T08:13:33.932 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.930+0000 7fb34e7fc640 1 -- 172.21.15.106:0/696165089 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348031d80 con 0x7fb3500ac400 2023-11-24T08:13:33.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 -- 172.21.15.106:0/696165089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3500ac400 msgr2=0x7fb350070aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.933 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 --2- 172.21.15.106:0/696165089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3500ac400 0x7fb350070aa0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7fb35006e7e0 tx=0x7fb34802b930 comp rx=0 tx=0).stop 2023-11-24T08:13:33.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 -- 172.21.15.106:0/696165089 shutdown_connections 2023-11-24T08:13:33.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 --2- 172.21.15.106:0/696165089 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb3500ac400 0x7fb350070aa0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 --2- 172.21.15.106:0/696165089 >> v2:172.21.15.156:3300/0 conn(0x7fb35006c4f0 0x7fb3500abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 -- 172.21.15.106:0/696165089 >> 172.21.15.106:0/696165089 conn(0x7fb3500b0be0 msgr2=0x7fb3500b0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:33.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 -- 172.21.15.106:0/696165089 shutdown_connections 2023-11-24T08:13:33.934 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.931+0000 7fb355a02640 1 -- 172.21.15.106:0/696165089 wait complete. 2023-11-24T08:13:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.933+0000 7fb355a02640 1 Processor -- start 2023-11-24T08:13:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.933+0000 7fb355a02640 1 -- start start 2023-11-24T08:13:33.936 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb355a02640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 0x7fb35009b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb355a02640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ac400 0x7fb35015b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb355a02640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce290 0x7fb35014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb355a02640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35009ac10 con 0x7fb35006c4f0 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb355a02640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb350157fb0 con 0x7fb3500ce290 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb355a02640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb350150010 con 0x7fb3500ac400 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 0x7fb35009b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 0x7fb35009b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:36202/0 (socket says 172.21.15.106:36202) 2023-11-24T08:13:33.937 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 -- 172.21.15.106:0/135550110 learned_addr learned my addr 172.21.15.106:0/135550110 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 -- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ac400 msgr2=0x7fb35015b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ac400 0x7fb35015b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 -- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce290 msgr2=0x7fb35014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34f7fe640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce290 0x7fb35014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce290 0x7fb35014fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.934+0000 7fb34effd640 1 -- 172.21.15.106:0/135550110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3480094d0 con 0x7fb35006c4f0 2023-11-24T08:13:33.938 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.935+0000 7fb34effd640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 0x7fb35009b030 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb34000bab0 tx=0x7fb34000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.935+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34000cc60 con 0x7fb35006c4f0 2023-11-24T08:13:33.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.935+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34000ce00 con 0x7fb35006c4f0 2023-11-24T08:13:33.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.935+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb340003bb0 con 0x7fb35006c4f0 2023-11-24T08:13:33.939 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.935+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb350099f90 con 0x7fb35006c4f0 2023-11-24T08:13:33.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.935+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35009a550 con 0x7fb35006c4f0 2023-11-24T08:13:33.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.936+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb35009ae80 con 0x7fb35006c4f0 2023-11-24T08:13:33.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.937+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fb340003d10 con 0x7fb35006c4f0 2023-11-24T08:13:33.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.938+0000 7fb34cff9640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb33006b4a0 0x7fb33006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:33.940 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.938+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb340089d20 con 0x7fb35006c4f0 2023-11-24T08:13:33.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.939+0000 7fb3467fc640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb33006b4a0 0x7fb33006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:33.941 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.939+0000 7fb3467fc640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb33006b4a0 0x7fb33006d960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb35009b900 tx=0x7fb338008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:33.944 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:33.943+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fb3400546c0 con 0x7fb35006c4f0 2023-11-24T08:13:34.129 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.128+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb350067870 con 0x7fb33006b4a0 2023-11-24T08:13:34.133 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.132+0000 7fb34cff9640 1 -- 172.21.15.106:0/135550110 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48397 (secure 0 0 0) 0x7fb350067870 con 0x7fb33006b4a0 2023-11-24T08:13:34.134 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:34.134 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:34.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb33006b4a0 msgr2=0x7fb33006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:34.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb33006b4a0 0x7fb33006d960 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fb35009b900 tx=0x7fb338008040 comp rx=0 tx=0).stop 2023-11-24T08:13:34.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 msgr2=0x7fb35009b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:34.136 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 0x7fb35009b030 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fb34000bab0 tx=0x7fb34000bf70 comp rx=0 tx=0).stop 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 shutdown_connections 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3500ce290 0x7fb35014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fb33006b4a0 0x7fb33006d960 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fb3500ac400 0x7fb35015b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 --2- 172.21.15.106:0/135550110 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fb35006c4f0 0x7fb35009b030 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.134+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 >> 172.21.15.106:0/135550110 conn(0x7fb3500b0be0 msgr2=0x7fb35015db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.135+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 shutdown_connections 2023-11-24T08:13:34.137 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:34.135+0000 7fb355a02640 1 -- 172.21.15.106:0/135550110 wait complete. 2023-11-24T08:13:34.155 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2023-11-24T08:13:33.361623+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259496,"kb_used_data":10024,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715224,"statfs":{"total":773094113280,"available":772828389376,"internally_reserved":0,"allocated":10264576,"data_stored":5057200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.673212"},"pg_stats":[{"pgid":"1.0","version":"19'117","reported_seq":221,"reported_epoch":91,"state":"active+clean","last_fresh":"2023-11-24T08:13:21.792718+0000","last_change":"2023-11-24T08:08:43.019349+0000","last_active":"2023-11-24T08:13:21.792718+0000","last_peered":"2023-11-24T08:13:21.792718+0000","last_clean":"2023-11-24T08:13:21.792718+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:13:21.792718+0000","last_undegraded":"2023-11-24T08:13:21.792718+0000","last_fullsized":"2023-11-24T08:13:21.792718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-24T08:08:41.823474+0000","last_clean_scrub_stamp":"2023-11-24T08:08:41.823474+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T13:14:11.500183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542206,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1074926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.089,"5min":1.0289999999999999,"15min":1.0169999999999999},"min":{"1min":0.66000000000000003,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.5389999999999999,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.196,"5min":1.073,"15min":1.0129999999999999},"min":{"1min":0.61899999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8580000000000001,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":1.004}]},{"osd":1,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.131,"15min":1.125},"min":{"1min":0.75800000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.7789999999999999,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":1.484},{"interface":"front","average":{"1min":1.196,"5min":1.181,"15min":1.1679999999999999},"min":{"1min":0.90500000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7150000000000001,"5min":2.3479999999999999,"15min":2.3479999999999999},"last":1.3580000000000001}]},{"osd":2,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.135,"15min":1.073},"min":{"1min":0.77300000000000002,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.821,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.224,"15min":1.2090000000000001},"min":{"1min":0.89700000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.954,"5min":2.1720000000000002,"15min":2.1720000000000002},"last":1.5760000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.24,"15min":1.234},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.5860000000000001,"5min":3.387,"15min":3.387},"last":1.222},{"interface":"front","average":{"1min":1.359,"5min":1.276,"15min":1.23},"min":{"1min":0.98699999999999999,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.865,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.284}]},{"osd":4,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.345,"5min":1.262,"15min":1.2210000000000001},"min":{"1min":0.93799999999999994,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.9259999999999999,"5min":2.6699999999999999,"15min":2.6699999999999999},"last":1.181},{"interface":"front","average":{"1min":1.387,"5min":1.3939999999999999,"15min":1.377},"min":{"1min":0.97999999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.909,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.129}]},{"osd":5,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.4019999999999999,"15min":1.4259999999999999},"min":{"1min":0.91800000000000004,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7909999999999999,"5min":2.3170000000000002,"15min":2.3170000000000002},"last":1.643},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.3959999999999999,"15min":1.3640000000000001},"min":{"1min":1.081,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.984,"5min":2.9750000000000001,"15min":2.9750000000000001},"last":1.252}]},{"osd":6,"last update":"Fri Nov 24 08:12:44 2023","interfaces":[{"interface":"back","average":{"1min":1.649,"5min":1.542,"15min":1.4950000000000001},"min":{"1min":1.4510000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.073,"5min":2.4630000000000001,"15min":2.4630000000000001},"last":1.427},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5,"15min":1.482},"min":{"1min":1.1579999999999999,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.7909999999999999,"5min":3.8450000000000002,"15min":3.8450000000000002},"last":1.522}]}]},{"osd":6,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0509999999999999,"5min":1.0669999999999999,"15min":1.091},"min":{"1min":0.65200000000000002,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4099999999999999,"5min":2.085,"15min":2.085},"last":1.355},{"interface":"front","average":{"1min":1.236,"5min":1.2110000000000001,"15min":1.236},"min":{"1min":0.745,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.4419999999999999}]},{"osd":1,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1140000000000001,"15min":1.125},"min":{"1min":0.68999999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.629,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.542},{"interface":"front","average":{"1min":1.048,"5min":1.145,"15min":1.163},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.593,"5min":1.98,"15min":1.98},"last":0.93200000000000005}]},{"osd":2,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.214,"15min":1.2170000000000001},"min":{"1min":0.78100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4199999999999999,"5min":1.778,"15min":1.778},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.1000000000000001,"5min":1.145,"15min":1.1859999999999999},"min":{"1min":0.76500000000000001,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.6659999999999999,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.032}]},{"osd":3,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.2769999999999999,"15min":1.335},"min":{"1min":0.88700000000000001,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7250000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.294},{"interface":"front","average":{"1min":1.264,"5min":1.395,"15min":1.4330000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.6910000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.2549999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.3160000000000001,"15min":1.3600000000000001},"min":{"1min":0.71499999999999997,"5min":0.622,"15min":0.622},"max":{"1min":1.7709999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.196},{"interface":"front","average":{"1min":1.518,"5min":1.5389999999999999,"15min":1.581},"min":{"1min":0.84499999999999997,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1269999999999998,"5min":2.661,"15min":2.661},"last":1.5900000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4730000000000001,"5min":1.569,"15min":1.6890000000000001},"min":{"1min":1.0900000000000001,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.0049999999999999,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.385},{"interface":"front","average":{"1min":1.419,"5min":1.538,"15min":1.6339999999999999},"min":{"1min":1.0580000000000001,"5min":0.505,"15min":0.505},"max":{"1min":1.752,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.331}]},{"osd":7,"last update":"Fri Nov 24 08:12:47 2023","interfaces":[{"interface":"back","average":{"1min":1.6419999999999999,"5min":1.6930000000000001,"15min":1.7689999999999999},"min":{"1min":1.3460000000000001,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.2370000000000001,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.492},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.6759999999999999,"15min":1.7689999999999999},"min":{"1min":1.1439999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.895,"5min":2.657,"15min":2.657},"last":1.4159999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542204,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":0.69699999999999995}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":0.79500000000000004}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":1.03},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":1.0980000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.155},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.1850000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.478}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":1.0609999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.538},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.3759999999999999}]}]},{"osd":4,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.0369999999999999,"15min":0.996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.6180000000000001,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":1.393},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1200000000000001,"15min":1.1000000000000001},"min":{"1min":0.74199999999999999,"5min":0.626,"15min":0.626},"max":{"1min":1.7909999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.081}]},{"osd":1,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.157,"15min":1.155},"min":{"1min":0.90300000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.569},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.2090000000000001,"15min":1.1699999999999999},"min":{"1min":0.81200000000000006,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6559999999999999,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":1.254}]},{"osd":2,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.2749999999999999,"15min":1.2769999999999999},"min":{"1min":0.88900000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.746,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.79900000000000004,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3640000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":1.496}]},{"osd":3,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.2709999999999999,"15min":1.2869999999999999},"min":{"1min":0.748,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.639},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.399,"15min":1.415},"min":{"1min":0.96199999999999997,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.901,"5min":2.746,"15min":2.746},"last":1.454}]},{"osd":5,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2829999999999999,"5min":1.2889999999999999,"15min":1.3149999999999999},"min":{"1min":0.69299999999999995,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.919,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.726},{"interface":"front","average":{"1min":1.5449999999999999,"5min":1.4690000000000001,"15min":1.4630000000000001},"min":{"1min":0.79300000000000004,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":2.6930000000000001,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.696}]},{"osd":6,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.411,"15min":1.4410000000000001},"min":{"1min":0.83999999999999997,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":2.101,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.5700000000000001,"5min":1.542,"15min":1.5589999999999999},"min":{"1min":0.89200000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.79,"5min":2.79,"15min":2.79},"last":1.7450000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:45 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.526,"15min":1.516},"min":{"1min":1.0820000000000001,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.0059999999999998,"5min":2.548,"15min":2.548},"last":1.357},{"interface":"front","average":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5469999999999999},"min":{"1min":1.1539999999999999,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":2.5070000000000001,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":1.7809999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97799999999999998,"15min":1.1220000000000001},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4139999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.88,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.56100000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":0.68400000000000005}]},{"osd":1,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":0.98299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.429,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.872},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0329999999999999,"15min":1.103},"min":{"1min":0.53300000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5740000000000001,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.91400000000000003}]},{"osd":2,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.107,"15min":1.2030000000000001},"min":{"1min":0.57099999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.544,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.1140000000000001,"15min":1.145},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.5029999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.1120000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3149999999999999,"15min":1.4350000000000001},"min":{"1min":0.72399999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.196},{"interface":"front","average":{"1min":1.202,"5min":1.2989999999999999,"15min":1.361},"min":{"1min":0.88,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6719999999999999,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4470000000000001,"15min":1.548},"min":{"1min":0.92700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.829,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.351,"5min":1.444,"15min":1.522},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7410000000000001,"5min":3.339,"15min":3.339},"last":1.355}]},{"osd":6,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4750000000000001,"15min":1.637},"min":{"1min":0.89000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8440000000000001,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.294},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4850000000000001,"15min":1.5449999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0030000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.175}]},{"osd":7,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5449999999999999,"15min":1.655},"min":{"1min":1.077,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.105,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.6839999999999999,"15min":1.966},"min":{"1min":1.01,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0470000000000002,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.403}]}]},{"osd":2,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":1.284},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.409}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1919999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.4710000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.6830000000000001},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.25}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.512}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.446},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":1.5549999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.617},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.321}]}]},{"osd":1,"up_from":14,"seq":60129542205,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.82799999999999996,"15min":0.83599999999999997},"min":{"1min":0.436,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.476,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.304},{"interface":"front","average":{"1min":0.91500000000000004,"5min":0.86799999999999999,"15min":0.88400000000000001},"min":{"1min":0.43099999999999999,"5min":0.37,"15min":0.37},"max":{"1min":1.5740000000000001,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":0.871}]},{"osd":2,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.84399999999999997,"15min":0.84999999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.2809999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.79800000000000004},{"interface":"front","average":{"1min":1,"5min":1.0229999999999999,"15min":1.0109999999999999},"min":{"1min":0.53100000000000003,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.97199999999999998}]},{"osd":3,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.1259999999999999,"15min":1.1519999999999999},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.6279999999999999,"5min":1.907,"15min":1.907},"last":1.256},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.996,"15min":1.0389999999999999},"min":{"1min":0.39400000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.458,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.198,"5min":1.167,"15min":1.165},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.3089999999999999,"15min":1.325},"min":{"1min":0.80300000000000005,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.1160000000000001,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.137}]},{"osd":5,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.248,"15min":1.296},"min":{"1min":0.85199999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.6850000000000001,"5min":2.661,"15min":2.661},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.329},"min":{"1min":0.76600000000000001,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":2.6819999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.4770000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.321,"15min":1.349},"min":{"1min":0.78900000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.633,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.3180000000000001,"5min":1.3400000000000001,"15min":1.3340000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.5760000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.3740000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3999999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.222,"5min":2.222,"15min":2.222},"last":1.411},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.351,"15min":1.363},"min":{"1min":0.73199999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.8220000000000001,"5min":2.008,"15min":2.008},"last":1.6339999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542205,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32872,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338968,"statfs":{"total":96636764160,"available":96603103232,"internally_reserved":0,"allocated":1728512,"data_stored":1074926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79700000000000004,"15min":0.80200000000000005},"min":{"1min":0.39300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.6299999999999999,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":0.93300000000000005},{"interface":"front","average":{"1min":1.006,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.503,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.5069999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.054}]},{"osd":2,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.83399999999999996,"15min":0.83199999999999996},"min":{"1min":0.48399999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.3320000000000001,"5min":1.3320000000000001,"15min":1.3320000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.91000000000000003,"15min":0.92800000000000005},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":0.96199999999999997}]},{"osd":3,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.05,"15min":1.0720000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.4450000000000001,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.88800000000000001,"15min":0.95199999999999996},"min":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.3169999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.524}]},{"osd":4,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.0980000000000001,"5min":1.1499999999999999,"15min":1.1659999999999999},"min":{"1min":0.68999999999999995,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.591,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.286,"5min":1.286,"15min":1.29},"min":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"max":{"1min":1.5880000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.427}]},{"osd":5,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1950000000000001,"15min":1.218},"min":{"1min":0.85799999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.506,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.252,"5min":1.2529999999999999,"15min":1.2669999999999999},"min":{"1min":0.79600000000000004,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.722,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.4850000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.2490000000000001,"15min":1.266},"min":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.512,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.6870000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2849999999999999,"15min":1.2869999999999999},"min":{"1min":0.76200000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.9039999999999999,"5min":2.149,"15min":2.149},"last":1.3240000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:42 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.3819999999999999,"15min":1.427},"min":{"1min":0.879,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7809999999999999,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.5820000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":1,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.6890000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.776}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:34.158 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-24T08:08:41.823474+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=8, tm_min=8, tm_sec=41, tm_wday=4, tm_yday=328, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=24, tm_hour=8, tm_min=13, tm_sec=27, tm_wday=4, tm_yday=328, tm_isdst=0) 2023-11-24T08:13:34.159 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-24T08:13:46.632 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:13:46.642 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:13:54.159 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-24T08:13:54.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.382+0000 7f90ce87c640 1 Processor -- start 2023-11-24T08:13:54.385 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.382+0000 7f90ce87c640 1 -- start start 2023-11-24T08:13:54.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90ce87c640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f90c806a500 0x7f90c806d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.386 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90ce87c640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f90c80cb810 0x7f90c80cbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90ce87c640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90c81663f0 con 0x7f90c80c37c0 2023-11-24T08:13:54.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90ce87c640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c80cffe0 con 0x7f90c80cb810 2023-11-24T08:13:54.387 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90ce87c640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90c806f570 con 0x7f90c806a500 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90cce19640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f90c80cb810 0x7f90c80cbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90cce19640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f90c80cb810 0x7f90c80cbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46068/0 (socket says 172.21.15.106:46068) 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.384+0000 7f90cce19640 1 -- 172.21.15.106:0/3651776181 learned_addr learned my addr 172.21.15.106:0/3651776181 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.385+0000 7f90c7fff640 1 --2- 172.21.15.106:0/3651776181 >> v2:172.21.15.156:3300/0 conn(0x7f90c806a500 0x7f90c806d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.385+0000 7f90cce19640 1 -- 172.21.15.106:0/3651776181 >> v2:172.21.15.156:3300/0 conn(0x7f90c806a500 msgr2=0x7f90c806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.385+0000 7f90cce19640 1 --2- 172.21.15.106:0/3651776181 >> v2:172.21.15.156:3300/0 conn(0x7f90c806a500 0x7f90c806d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.388 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.385+0000 7f90cce19640 1 -- 172.21.15.106:0/3651776181 >> v1:172.21.15.106:6789/0 conn(0x7f90c80c37c0 legacy=0x7f90c806a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.389 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.385+0000 7f90cce19640 1 -- 172.21.15.106:0/3651776181 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c81657e0 con 0x7f90c80cb810 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.386+0000 7f90cce19640 1 --2- 172.21.15.106:0/3651776181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80cb810 0x7f90c80cbcc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f90c804e8b0 tx=0x7f90c002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682e938c72f93f39 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.386+0000 7f90c77fe640 1 -- 172.21.15.106:0/3651776181 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0030860 con 0x7f90c80cb810 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.386+0000 7f90c77fe640 1 -- 172.21.15.106:0/3651776181 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90c00309c0 con 0x7f90c80cb810 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.386+0000 7f90c77fe640 1 -- 172.21.15.106:0/3651776181 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90c0030d50 con 0x7f90c80cb810 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 -- 172.21.15.106:0/3651776181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80cb810 msgr2=0x7f90c80cbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3651776181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80cb810 0x7f90c80cbcc0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f90c804e8b0 tx=0x7f90c002b9a0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.390 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 -- 172.21.15.106:0/3651776181 shutdown_connections 2023-11-24T08:13:54.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3651776181 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80cb810 0x7f90c80cbcc0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3651776181 >> v2:172.21.15.156:3300/0 conn(0x7f90c806a500 0x7f90c806d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 -- 172.21.15.106:0/3651776181 >> 172.21.15.106:0/3651776181 conn(0x7f90c80d01b0 msgr2=0x7f90c80d05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:54.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 -- 172.21.15.106:0/3651776181 shutdown_connections 2023-11-24T08:13:54.391 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.387+0000 7f90ce87c640 1 -- 172.21.15.106:0/3651776181 wait complete. 2023-11-24T08:13:54.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.389+0000 7f90ce87c640 1 Processor -- start 2023-11-24T08:13:54.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.389+0000 7f90ce87c640 1 -- start start 2023-11-24T08:13:54.392 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90ce87c640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c806a500 0x7f90c81eed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90ce87c640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 0x7f90c80b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90ce87c640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90c80cb810 0x7f90c80c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90ce87c640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80b4c20 con 0x7f90c80c37c0 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90ce87c640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80b3d00 con 0x7f90c806a500 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90ce87c640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90c80b3fa0 con 0x7f90c80cb810 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 0x7f90c80b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90bf7fe640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 0x7f90c80b5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46076/0 (socket says 172.21.15.106:46076) 2023-11-24T08:13:54.393 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.390+0000 7f90bf7fe640 1 -- 172.21.15.106:0/3830703675 learned_addr learned my addr 172.21.15.106:0/3830703675 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:54.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90bf7fe640 1 -- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90c80cb810 msgr2=0x7f90c80c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90bf7fe640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90c80cb810 0x7f90c80c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90c7fff640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c806a500 0x7f90c81eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90bf7fe640 1 -- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c806a500 msgr2=0x7f90c81eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90bf7fe640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c806a500 0x7f90c81eed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.395 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90bf7fe640 1 -- 172.21.15.106:0/3830703675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90c00094d0 con 0x7f90c80c37c0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90bf7fe640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 0x7f90c80b5040 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c440 tx=0x7f90b0001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b00052d0 con 0x7f90c80c37c0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b0005430 con 0x7f90c80c37c0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0007260 con 0x7f90c80c37c0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.391+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90c80b4280 con 0x7f90c80c37c0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.392+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90c80b4840 con 0x7f90c80c37c0 2023-11-24T08:13:54.396 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.393+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90c80b4d60 con 0x7f90c80c37c0 2023-11-24T08:13:54.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.393+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f90b00073c0 con 0x7f90c80c37c0 2023-11-24T08:13:54.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.394+0000 7f90c57fa640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f90a806b470 0x7f90a806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.394+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f90b00892e0 con 0x7f90c80c37c0 2023-11-24T08:13:54.397 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.395+0000 7f90c7fff640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f90a806b470 0x7f90a806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.398 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.395+0000 7f90c7fff640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f90a806b470 0x7f90a806d930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009f30 tx=0x7f90b800b040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:54.402 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.400+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f90c80b4d60 con 0x7f90c80c37c0 2023-11-24T08:13:54.588 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.586+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 --> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f90c8166080 con 0x7f90a806b470 2023-11-24T08:13:54.594 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.592+0000 7f90c57fa640 1 -- 172.21.15.106:0/3830703675 <== mgr.4110 v2:172.21.15.106:6832/2073981325 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+48464 (secure 0 0 0) 0x7f90b8009b20 con 0x7f90a806b470 2023-11-24T08:13:54.595 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:54.595 INFO:teuthology.orchestra.run.smithi106.stderr:dumped all 2023-11-24T08:13:54.597 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f90a806b470 msgr2=0x7f90a806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f90a806b470 0x7f90a806d930 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f90b8009f30 tx=0x7f90b800b040 comp rx=0 tx=0).stop 2023-11-24T08:13:54.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 msgr2=0x7f90c80b5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 0x7f90c80b5040 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f90b000c440 tx=0x7f90b0001e70 comp rx=0 tx=0).stop 2023-11-24T08:13:54.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 shutdown_connections 2023-11-24T08:13:54.598 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f90a806b470 0x7f90a806d930 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.594+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f90c80cb810 0x7f90c80c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.595+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f90c80c37c0 0x7f90c80b5040 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.595+0000 7f90ce87c640 1 --2- 172.21.15.106:0/3830703675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90c806a500 0x7f90c81eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.595+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 >> 172.21.15.106:0/3830703675 conn(0x7f90c80d01b0 msgr2=0x7f90c80ce0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:54.599 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.595+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 shutdown_connections 2023-11-24T08:13:54.600 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.595+0000 7f90ce87c640 1 -- 172.21.15.106:0/3830703675 wait complete. 2023-11-24T08:13:54.617 INFO:teuthology.orchestra.run.smithi106.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2023-11-24T08:13:53.367018+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":259480,"kb_used_data":10024,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715240,"statfs":{"total":773094113280,"available":772828405760,"internally_reserved":0,"allocated":10264576,"data_stored":5057200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":255315504},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003325"},"pg_stats":[{"pgid":"1.0","version":"92'119","reported_seq":232,"reported_epoch":92,"state":"active+clean","last_fresh":"2023-11-24T08:13:43.902504+0000","last_change":"2023-11-24T08:13:43.902569+0000","last_active":"2023-11-24T08:13:43.902504+0000","last_peered":"2023-11-24T08:13:43.902504+0000","last_clean":"2023-11-24T08:13:43.902504+0000","last_became_active":"2023-11-24T08:08:43.018366+0000","last_became_peered":"2023-11-24T08:08:43.018366+0000","last_unstale":"2023-11-24T08:13:43.902504+0000","last_undegraded":"2023-11-24T08:13:43.902504+0000","last_fullsized":"2023-11-24T08:13:43.902504+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"92'119","last_scrub_stamp":"2023-11-24T08:13:43.902308+0000","last_deep_scrub":"92'119","last_deep_scrub_stamp":"2023-11-24T08:13:43.902308+0000","last_clean_scrub_stamp":"2023-11-24T08:13:43.902308+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-25T15:53:13.661331+0000","scrub_duration":0.012927413,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542210,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1728512,"data_stored":1074926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.0629999999999999,"15min":1.028},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":2.488,"5min":2.5329999999999999,"15min":2.5329999999999999},"last":1.9370000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.107,"15min":1.024},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.7429999999999999,"5min":2.7989999999999999,"15min":2.7989999999999999},"last":0.995}]},{"osd":1,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.1559999999999999,"15min":1.1339999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":2.3039999999999998,"5min":2.3039999999999998,"15min":2.3039999999999998},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.1879999999999999,"15min":1.1699999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":2.6819999999999999,"5min":2.6819999999999999,"15min":2.6819999999999999},"last":1.369}]},{"osd":2,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1539999999999999,"5min":1.157,"15min":1.081},"min":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"max":{"1min":1.9950000000000001,"5min":3.1829999999999998,"15min":3.1829999999999998},"last":1.133},{"interface":"front","average":{"1min":1.3620000000000001,"5min":1.256,"15min":1.22},"min":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":2.4350000000000001,"5min":2.4350000000000001,"15min":2.4350000000000001},"last":1.6619999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.238,"15min":1.234},"min":{"1min":0.72799999999999998,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.877,"5min":3.387,"15min":3.387},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.3140000000000001,"15min":1.2430000000000001},"min":{"1min":0.84399999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":2.367,"5min":2.4020000000000001,"15min":2.4020000000000001},"last":1.7609999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.3129999999999999,"15min":1.238},"min":{"1min":0.78900000000000003,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":3.0630000000000002,"5min":3.0630000000000002,"15min":3.0630000000000002},"last":1.5920000000000001},{"interface":"front","average":{"1min":1.3759999999999999,"5min":1.3959999999999999,"15min":1.3779999999999999},"min":{"1min":0.81799999999999995,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.774,"5min":3.2869999999999999,"15min":3.2869999999999999},"last":1.4410000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.5349999999999999,"5min":1.421,"15min":1.4330000000000001},"min":{"1min":0.754,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.5419999999999998,"5min":2.5419999999999998,"15min":2.5419999999999998},"last":1.8540000000000001},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.444,"15min":1.3799999999999999},"min":{"1min":0.87,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":3.577,"5min":3.577,"15min":3.577},"last":1.708}]},{"osd":6,"last update":"Fri Nov 24 08:13:45 2023","interfaces":[{"interface":"back","average":{"1min":1.8700000000000001,"5min":1.6220000000000001,"15min":1.522},"min":{"1min":1.054,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":6.3300000000000001,"5min":6.3300000000000001,"15min":6.3300000000000001},"last":1.804},{"interface":"front","average":{"1min":1.8029999999999999,"5min":1.5660000000000001,"15min":1.504},"min":{"1min":0.94799999999999995,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":6.6399999999999997,"5min":6.6399999999999997,"15min":6.6399999999999997},"last":1.6359999999999999}]}]},{"osd":6,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.0629999999999999,"15min":1.0900000000000001},"min":{"1min":0.58899999999999997,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.6060000000000001,"5min":2.085,"15min":2.085},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.206,"15min":1.234},"min":{"1min":0.77300000000000002,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.671,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":1.544}]},{"osd":1,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.125,"15min":1.129},"min":{"1min":0.86699999999999999,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.665,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":1.038},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.125,"15min":1.1559999999999999},"min":{"1min":0.751,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.4239999999999999,"5min":1.98,"15min":1.98},"last":1.3779999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1910000000000001,"15min":1.2090000000000001},"min":{"1min":0.81000000000000005,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.6379999999999999,"5min":1.778,"15min":1.778},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.115,"15min":1.1759999999999999},"min":{"1min":0.71499999999999997,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.484,"5min":1.7709999999999999,"15min":1.7709999999999999},"last":1.4510000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.2629999999999999,"15min":1.331},"min":{"1min":0.84899999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"max":{"1min":1.7529999999999999,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.3049999999999999,"5min":1.365,"15min":1.423},"min":{"1min":0.92700000000000005,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":1.9350000000000001,"5min":2.4510000000000001,"15min":2.4510000000000001},"last":1.72}]},{"osd":4,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.276,"15min":1.3460000000000001},"min":{"1min":0.68000000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.6859999999999999,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.571,"5min":1.5329999999999999,"15min":1.579},"min":{"1min":1.089,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":2.1429999999999998,"5min":2.661,"15min":2.661},"last":1.782}]},{"osd":5,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.522,"5min":1.524,"15min":1.6739999999999999},"min":{"1min":1.198,"5min":0.72399999999999998,"15min":0.72399999999999998},"max":{"1min":2.181,"5min":3.2829999999999999,"15min":3.2829999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.4850000000000001,"15min":1.6160000000000001},"min":{"1min":1.0389999999999999,"5min":0.505,"15min":0.505},"max":{"1min":2.0790000000000002,"5min":3.5950000000000002,"15min":3.5950000000000002},"last":1.502}]},{"osd":7,"last update":"Fri Nov 24 08:13:49 2023","interfaces":[{"interface":"back","average":{"1min":1.621,"5min":1.655,"15min":1.7569999999999999},"min":{"1min":1.2689999999999999,"5min":1.0680000000000001,"15min":1.0680000000000001},"max":{"1min":2.3239999999999998,"5min":2.5830000000000002,"15min":2.5830000000000002},"last":1.863},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.621,"15min":1.7509999999999999},"min":{"1min":1.232,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":2.0339999999999998,"5min":2.657,"15min":2.657},"last":1.893}]}]},{"osd":5,"up_from":14,"seq":60129542208,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32288,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339552,"statfs":{"total":96636764160,"available":96603701248,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.0529999999999999,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.76900000000000002,"5min":0.627,"15min":0.627},"max":{"1min":1.4199999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.129,"5min":1.125,"15min":1.155},"min":{"1min":0.73999999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.5700000000000001,"5min":1.837,"15min":1.837},"last":1.387}]},{"osd":1,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.179,"5min":1.1339999999999999,"15min":1.1200000000000001},"min":{"1min":0.65700000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.7190000000000001,"5min":1.7190000000000001,"15min":1.7190000000000001},"last":1.1359999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.1899999999999999,"15min":1.198},"min":{"1min":0.70399999999999996,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.827,"5min":1.827,"15min":1.827},"last":1.097}]},{"osd":2,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2330000000000001,"15min":1.3029999999999999},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.706,"5min":2.085,"15min":2.085},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.3129999999999999,"15min":1.355},"min":{"1min":0.75900000000000001,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.841,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.872}]},{"osd":3,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.3680000000000001,"15min":1.4239999999999999},"min":{"1min":0.871,"5min":0.83599999999999997,"15min":0.83599999999999997},"max":{"1min":1.8440000000000001,"5min":2.0379999999999998,"15min":2.0379999999999998},"last":1.482},{"interface":"front","average":{"1min":1.351,"5min":1.429,"15min":1.504},"min":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"max":{"1min":1.8540000000000001,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.2589999999999999}]},{"osd":4,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.4079999999999999,"15min":1.446},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.7609999999999999,"5min":3.577,"15min":3.577},"last":1.6619999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.5820000000000001,"15min":1.6819999999999999},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.8280000000000001,"5min":4.0279999999999996,"15min":4.0279999999999996},"last":1.5940000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.51,"5min":1.556,"15min":1.599},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":2,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.462,"5min":1.534,"15min":1.536},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.9570000000000001,"5min":2.512,"15min":2.512},"last":1.73}]},{"osd":7,"last update":"Fri Nov 24 08:12:51 2023","interfaces":[{"interface":"back","average":{"1min":1.5669999999999999,"5min":1.6499999999999999,"15min":1.7050000000000001},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2,"5min":2.7400000000000002,"15min":2.7400000000000002},"last":1.702},{"interface":"front","average":{"1min":1.607,"5min":1.679,"15min":1.7270000000000001},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":2.2919999999999998,"5min":2.9689999999999999,"15min":2.9689999999999999},"last":1.631}]}]},{"osd":4,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.0580000000000001,"15min":1.0029999999999999},"min":{"1min":0.71799999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.48,"5min":1.6950000000000001,"15min":1.6950000000000001},"last":0.98699999999999999},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.141,"15min":1.107},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6859999999999999,"5min":2.0099999999999998,"15min":2.0099999999999998},"last":1.0960000000000001}]},{"osd":1,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.163,"15min":1.157},"min":{"1min":0.57599999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.6819999999999999,"5min":1.97,"15min":1.97},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.109,"5min":1.2010000000000001,"15min":1.167},"min":{"1min":0.77800000000000002,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.5700000000000001,"5min":2.2749999999999999,"15min":2.2749999999999999},"last":0.90200000000000002}]},{"osd":2,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1559999999999999,"5min":1.25,"15min":1.2689999999999999},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.722,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.0900000000000001,"15min":1.1020000000000001},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.6930000000000001,"5min":2.0600000000000001,"15min":2.0600000000000001},"last":0.80900000000000005}]},{"osd":3,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.202,"5min":1.252,"15min":1.28},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.639,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.3740000000000001,"15min":1.407},"min":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.758,"5min":2.746,"15min":2.746},"last":0.873}]},{"osd":5,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.2849999999999999,"15min":1.3140000000000001},"min":{"1min":0.59799999999999998,"5min":0.58199999999999996,"15min":0.58199999999999996},"max":{"1min":1.8029999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":1.446,"5min":1.466,"15min":1.462},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.8069999999999999,"5min":2.6930000000000001,"15min":2.6930000000000001},"last":1.359}]},{"osd":6,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.379,"15min":1.4299999999999999},"min":{"1min":0.73199999999999998,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.7509999999999999,"5min":2.5030000000000001,"15min":2.5030000000000001},"last":0.95399999999999996},{"interface":"front","average":{"1min":1.484,"5min":1.5249999999999999,"15min":1.5529999999999999},"min":{"1min":0.72299999999999998,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":2.0499999999999998,"5min":2.79,"15min":2.79},"last":1.1639999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.3720000000000001,"5min":1.498,"15min":1.5069999999999999},"min":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.8480000000000001,"5min":2.548,"15min":2.548},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.4830000000000001,"5min":1.514,"15min":1.542},"min":{"1min":0.68100000000000005,"5min":0.65100000000000002,"15min":0.65100000000000002},"max":{"1min":1.865,"5min":2.8359999999999999,"15min":2.8359999999999999},"last":0.73799999999999999}]}]},{"osd":3,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.97799999999999998,"15min":1.1220000000000001},"min":{"1min":0.48399999999999999,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.4139999999999999,"5min":6.6879999999999997,"15min":6.6879999999999997},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.88,"5min":0.998,"15min":1.1220000000000001},"min":{"1min":0.56100000000000005,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.274,"5min":8.0530000000000008,"15min":8.0530000000000008},"last":1.224}]},{"osd":1,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.92300000000000004,"15min":0.98299999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.429,"5min":3.7309999999999999,"15min":3.7309999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.93200000000000005,"5min":1.0329999999999999,"15min":1.103},"min":{"1min":0.53300000000000003,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.5740000000000001,"5min":2.9220000000000002,"15min":2.9220000000000002},"last":0.85399999999999998}]},{"osd":2,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.107,"15min":1.2030000000000001},"min":{"1min":0.57099999999999995,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.544,"5min":5.7400000000000002,"15min":5.7400000000000002},"last":0.95199999999999996},{"interface":"front","average":{"1min":1.121,"5min":1.1140000000000001,"15min":1.145},"min":{"1min":0.56999999999999995,"5min":0.505,"15min":0.505},"max":{"1min":1.5029999999999999,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.002}]},{"osd":4,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.3149999999999999,"15min":1.4350000000000001},"min":{"1min":0.72399999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":2.0659999999999998,"5min":8.8409999999999993,"15min":8.8409999999999993},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.2989999999999999,"15min":1.361},"min":{"1min":0.88,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":1.6719999999999999,"5min":3.8620000000000001,"15min":3.8620000000000001},"last":1.387}]},{"osd":5,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.4470000000000001,"15min":1.548},"min":{"1min":0.92700000000000005,"5min":0.72799999999999998,"15min":0.72799999999999998},"max":{"1min":1.829,"5min":6.2729999999999997,"15min":6.2729999999999997},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.351,"5min":1.444,"15min":1.522},"min":{"1min":1.0169999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7410000000000001,"5min":3.339,"15min":3.339},"last":1.2989999999999999}]},{"osd":6,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.4750000000000001,"15min":1.637},"min":{"1min":0.89000000000000001,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":1.8440000000000001,"5min":9.0579999999999998,"15min":9.0579999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4850000000000001,"15min":1.5449999999999999},"min":{"1min":0.85099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"max":{"1min":2.0030000000000001,"5min":4.4800000000000004,"15min":4.4800000000000004},"last":1.1699999999999999}]},{"osd":7,"last update":"Fri Nov 24 08:12:55 2023","interfaces":[{"interface":"back","average":{"1min":1.411,"5min":1.5449999999999999,"15min":1.655},"min":{"1min":1.077,"5min":0.90900000000000003,"15min":0.90900000000000003},"max":{"1min":2.105,"5min":5.1399999999999997,"15min":5.1399999999999997},"last":1.141},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.6839999999999999,"15min":1.966},"min":{"1min":1.01,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":2.0470000000000002,"5min":16.952999999999999,"15min":16.952999999999999},"last":1.3540000000000001}]}]},{"osd":2,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32296,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339544,"statfs":{"total":96636764160,"available":96603693056,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.77700000000000002,"15min":0.76600000000000001},"min":{"1min":0.35399999999999998,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1519999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.83899999999999997,"15min":0.78400000000000003},"min":{"1min":0.48599999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.6679999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.96599999999999997}]},{"osd":1,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.84799999999999998,"15min":0.87},"min":{"1min":0.54000000000000004,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.2110000000000001,"5min":1.6479999999999999,"15min":1.6479999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.93400000000000005},"min":{"1min":0.49299999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.0449999999999999}]},{"osd":3,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.996,"15min":0.97799999999999998},"min":{"1min":0.47499999999999998,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.47,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.4690000000000001},{"interface":"front","average":{"1min":0.97199999999999998,"5min":0.97799999999999998,"15min":0.998},"min":{"1min":0.52200000000000002,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.472,"5min":1.7769999999999999,"15min":1.7769999999999999},"last":1.2010000000000001}]},{"osd":4,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.212,"15min":1.2150000000000001},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.698,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.244,"15min":1.2290000000000001},"min":{"1min":0.80000000000000004,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.8939999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.528}]},{"osd":5,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.3140000000000001,"15min":1.415},"min":{"1min":0.83599999999999997,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.585,"5min":4.0129999999999999,"15min":4.0129999999999999},"last":1.3819999999999999},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.70399999999999996,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.6459999999999999,"5min":1.839,"15min":1.839},"last":1.3120000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.361,"15min":1.409},"min":{"1min":0.83099999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.7290000000000001,"5min":2.7509999999999999,"15min":2.7509999999999999},"last":1.347},{"interface":"front","average":{"1min":1.325,"5min":1.3220000000000001,"15min":1.3360000000000001},"min":{"1min":0.78200000000000003,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.696,"5min":1.958,"15min":1.958},"last":1.6830000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:12:49 2023","interfaces":[{"interface":"back","average":{"1min":1.395,"5min":1.4139999999999999,"15min":1.47},"min":{"1min":0.76200000000000001,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.8640000000000001,"5min":5.1379999999999999,"15min":5.1379999999999999},"last":1.431},{"interface":"front","average":{"1min":1.385,"5min":1.411,"15min":1.4379999999999999},"min":{"1min":0.85499999999999998,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.8200000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":1.635}]}]},{"osd":1,"up_from":14,"seq":60129542209,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32292,"kb_used_data":1108,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339548,"statfs":{"total":96636764160,"available":96603697152,"internally_reserved":0,"allocated":1134592,"data_stored":484558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.81200000000000006,"15min":0.82999999999999996},"min":{"1min":0.40699999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3799999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.85199999999999998,"15min":0.879},"min":{"1min":0.41799999999999998,"5min":0.37,"15min":0.37},"max":{"1min":1.5229999999999999,"5min":1.5740000000000001,"15min":1.5740000000000001},"last":1.506}]},{"osd":2,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":0.84299999999999997},"min":{"1min":0.49299999999999999,"5min":0.375,"15min":0.375},"max":{"1min":1.0049999999999999,"5min":2.0659999999999998,"15min":2.0659999999999998},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.98699999999999999,"5min":1.02,"15min":1.01},"min":{"1min":0.60199999999999998,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.427,"5min":1.925,"15min":1.925},"last":1.0940000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0940000000000001,"15min":1.1419999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.5549999999999999,"5min":1.907,"15min":1.907},"last":1.4750000000000001},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.96099999999999997,"15min":1.0269999999999999},"min":{"1min":0.48799999999999999,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.5620000000000001,"5min":2.1240000000000001,"15min":2.1240000000000001},"last":1.042}]},{"osd":4,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.105,"5min":1.155,"15min":1.161},"min":{"1min":0.83599999999999997,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.577,"5min":2.29,"15min":2.29},"last":1.248},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2909999999999999,"15min":1.319},"min":{"1min":0.82699999999999996,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.502,"5min":2.1160000000000001,"15min":2.1160000000000001},"last":1.2909999999999999}]},{"osd":5,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2330000000000001,"15min":1.2909999999999999},"min":{"1min":0.80900000000000005,"5min":0.77200000000000002,"15min":0.77200000000000002},"max":{"1min":1.7989999999999999,"5min":2.661,"15min":2.661},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.121,"5min":1.2450000000000001,"15min":1.3129999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5449999999999999,"5min":2.9710000000000001,"15min":2.9710000000000001},"last":1.425}]},{"osd":6,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.298,"5min":1.3080000000000001,"15min":1.3440000000000001},"min":{"1min":0.84899999999999998,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.827,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.7729999999999999},{"interface":"front","average":{"1min":1.3069999999999999,"5min":1.335,"15min":1.333},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.8460000000000001,"5min":2.5760000000000001,"15min":2.5760000000000001},"last":1.5720000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:13:47 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.369,"15min":1.391},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.544,"5min":2.222,"15min":2.222},"last":1.544},{"interface":"front","average":{"1min":1.3340000000000001,"5min":1.3440000000000001,"15min":1.3600000000000001},"min":{"1min":0.76700000000000002,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.873,"5min":2.008,"15min":2.008},"last":1.873}]}]},{"osd":0,"up_from":14,"seq":60129542209,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32864,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338976,"statfs":{"total":96636764160,"available":96603111424,"internally_reserved":0,"allocated":1728512,"data_stored":1074926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.83299999999999996,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.538,"5min":1.6299999999999999,"15min":1.6299999999999999},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.022,"5min":0.97099999999999997,"15min":0.96199999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.5669999999999999,"5min":1.7070000000000001,"15min":1.7070000000000001},"last":1.2769999999999999}]},{"osd":2,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.84399999999999997,"15min":0.83499999999999996},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.601,"5min":1.601,"15min":1.601},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.90900000000000003,"15min":0.92800000000000005},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.4379999999999999,"5min":1.986,"15min":1.986},"last":1.0920000000000001}]},{"osd":3,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.054,"15min":1.0740000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.5329999999999999,"5min":2.1389999999999998,"15min":2.1389999999999998},"last":1.556},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.88100000000000001,"15min":0.94899999999999995},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.5569999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.1579999999999999,"15min":1.1679999999999999},"min":{"1min":0.64700000000000002,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.504},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.286,"15min":1.29},"min":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"max":{"1min":1.7270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4790000000000001}]},{"osd":5,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.1970000000000001,"15min":1.2190000000000001},"min":{"1min":0.81100000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"max":{"1min":1.6180000000000001,"5min":2.0390000000000001,"15min":2.0390000000000001},"last":1.4259999999999999},{"interface":"front","average":{"1min":1.3,"5min":1.258,"15min":1.268},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":1.7609999999999999,"5min":2.1459999999999999,"15min":2.1459999999999999},"last":1.3140000000000001}]},{"osd":6,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.25,"15min":1.266},"min":{"1min":0.74199999999999999,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6870000000000001,"5min":2.0190000000000001,"15min":2.0190000000000001},"last":1.3600000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.286,"15min":1.2869999999999999},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.895,"5min":2.149,"15min":2.149},"last":1.6180000000000001}]},{"osd":7,"last update":"Fri Nov 24 08:13:44 2023","interfaces":[{"interface":"back","average":{"1min":1.4139999999999999,"5min":1.375,"15min":1.425},"min":{"1min":0.79300000000000004,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.974,"5min":2.4260000000000002,"15min":2.4260000000000002},"last":1.403},{"interface":"front","average":{"1min":1.429,"5min":1.365,"15min":1.4059999999999999},"min":{"1min":0.94699999999999995,"5min":0.77300000000000002,"15min":0.77300000000000002},"max":{"1min":1.8420000000000001,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":1.2350000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-24T08:13:54.620 DEBUG:teuthology.orchestra.run.smithi106:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-24T08:13:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.808+0000 7fdea8023640 1 Processor -- start 2023-11-24T08:13:54.811 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.808+0000 7fdea8023640 1 -- start start 2023-11-24T08:13:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.810+0000 7fdea8023640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fdea00c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.810+0000 7fdea8023640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea0161fa0 con 0x7fdea0161b10 2023-11-24T08:13:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.810+0000 7fdea8023640 1 -- --> v1:172.21.15.156:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdea015de00 con 0x7fdea00c65f0 2023-11-24T08:13:54.815 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.810+0000 7fdea8023640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00c6a70 con 0x7fdea0140520 2023-11-24T08:13:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.810+0000 7fdea65c0640 1 --1- >> v1:172.21.15.106:6789/0 conn(0x7fdea0161b10 0x7fdea006ff70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.106:6789/0 says I am v1:172.21.15.106:54072/0 (socket says 172.21.15.106:54072) 2023-11-24T08:13:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.810+0000 7fdea65c0640 1 -- 172.21.15.106:0/1156606534 learned_addr learned my addr 172.21.15.106:0/1156606534 (peer_addr_for_me v1:172.21.15.106:0/0) 2023-11-24T08:13:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea5dbf640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fdea00c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.816 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 <== mon.0 v1:172.21.15.106:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865654564 0 0) 0x7fdea0161fa0 con 0x7fdea0161b10 2023-11-24T08:13:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 --> v1:172.21.15.106:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde8c003520 con 0x7fdea0161b10 2023-11-24T08:13:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 <== mon.0 v1:172.21.15.106:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 133096307 0 0) 0x7fde8c003520 con 0x7fdea0161b10 2023-11-24T08:13:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 msgr2=0x7fdea00c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fdea00c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 >> v1:172.21.15.156:6789/0 conn(0x7fdea00c65f0 legacy=0x7fdea0161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.817 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 --> v1:172.21.15.106:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea015e0e0 con 0x7fdea0161b10 2023-11-24T08:13:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 <== mon.0 v1:172.21.15.106:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1683670756 0 0) 0x7fde980019f0 con 0x7fdea0161b10 2023-11-24T08:13:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 >> v1:172.21.15.106:6789/0 conn(0x7fdea0161b10 legacy=0x7fdea006ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0039a0 0x7fde8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde8c0047c0 0x7fde8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.818 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fde8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00c6a70 con 0x7fde8c0047c0 2023-11-24T08:13:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c00ba60 con 0x7fde8c0039a0 2023-11-24T08:13:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.811+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8c00bc20 con 0x7fdea0140520 2023-11-24T08:13:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea55be640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde8c0047c0 0x7fde8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.819 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea65c0640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fde8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea55be640 1 -- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 msgr2=0x7fde8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea55be640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fde8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea55be640 1 -- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0039a0 msgr2=0x7fde8c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea55be640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0039a0 0x7fde8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.820 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea55be640 1 -- 172.21.15.106:0/1156606534 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde8c003520 con 0x7fde8c0047c0 2023-11-24T08:13:54.821 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.812+0000 7fdea65c0640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0140520 0x7fde8c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.813+0000 7fdea55be640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde8c0047c0 0x7fde8c004c40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fde90001720 tx=0x7fde8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527997340f87b2e server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.813+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900052d0 con 0x7fde8c0047c0 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.813+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9002e050 con 0x7fde8c0047c0 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.813+0000 7fdea4dbd640 1 -- 172.21.15.106:0/1156606534 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde900035f0 con 0x7fde8c0047c0 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea8023640 1 -- 172.21.15.106:0/1156606534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde8c0047c0 msgr2=0x7fde8c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea8023640 1 --2- 172.21.15.106:0/1156606534 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fde8c0047c0 0x7fde8c004c40 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fde90001720 tx=0x7fde8c0057f0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea65c0640 1 -- 172.21.15.106:0/1156606534 reap_dead start 2023-11-24T08:13:54.825 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea8023640 1 -- 172.21.15.106:0/1156606534 shutdown_connections 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea8023640 1 -- 172.21.15.106:0/1156606534 >> 172.21.15.106:0/1156606534 conn(0x7fdea00b2e40 msgr2=0x7fdea00b3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea8023640 1 -- 172.21.15.106:0/1156606534 shutdown_connections 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.814+0000 7fdea8023640 1 -- 172.21.15.106:0/1156606534 wait complete. 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.816+0000 7fdea8023640 1 Processor -- start 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.816+0000 7fdea8023640 1 -- start start 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea8023640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0047c0 0x7fdea015d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.826 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea8023640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0156880 0x7fdea015fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea8023640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 0x7fdea0160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea8023640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea00cac80 con 0x7fdea00cae20 2023-11-24T08:13:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea8023640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdea01f67e0 con 0x7fde8c0047c0 2023-11-24T08:13:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea8023640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdea01f6a30 con 0x7fdea0156880 2023-11-24T08:13:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea65c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 0x7fdea0160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.827 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea65c0640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 0x7fdea0160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46102/0 (socket says 172.21.15.106:46102) 2023-11-24T08:13:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.817+0000 7fdea65c0640 1 -- 172.21.15.106:0/1933885508 learned_addr learned my addr 172.21.15.106:0/1933885508 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea55be640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0156880 0x7fdea015fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea5dbf640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0047c0 0x7fdea015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea65c0640 1 -- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0156880 msgr2=0x7fdea015fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.828 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea65c0640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0156880 0x7fdea015fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea65c0640 1 -- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0047c0 msgr2=0x7fdea015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea65c0640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0047c0 0x7fdea015d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea65c0640 1 -- 172.21.15.106:0/1933885508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdea01f6d10 con 0x7fdea00cae20 2023-11-24T08:13:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fdea65c0640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 0x7fdea0160150 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ceb50 tx=0x7fde9800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:54.829 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98004d60 con 0x7fdea00cae20 2023-11-24T08:13:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.818+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9800bb80 con 0x7fdea00cae20 2023-11-24T08:13:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.819+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdea01f6ff0 con 0x7fdea00cae20 2023-11-24T08:13:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.819+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde98007110 con 0x7fdea00cae20 2023-11-24T08:13:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.819+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdea015b710 con 0x7fdea00cae20 2023-11-24T08:13:54.830 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.820+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdea01f7510 con 0x7fdea00cae20 2023-11-24T08:13:54.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.821+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7fde98007270 con 0x7fdea00cae20 2023-11-24T08:13:54.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.822+0000 7fde967fc640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fde8406b4a0 0x7fde8406d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:54.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.822+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fde9808a2c0 con 0x7fdea00cae20 2023-11-24T08:13:54.831 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.822+0000 7fdea5dbf640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fde8406b4a0 0x7fde8406d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:54.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.823+0000 7fdea5dbf640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fde8406b4a0 0x7fde8406d960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0047b0 tx=0x7fde9c008040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:54.832 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:54.826+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7fdea01f7510 con 0x7fdea00cae20 2023-11-24T08:13:55.020 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.017+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fdea01f7510 con 0x7fdea00cae20 2023-11-24T08:13:55.025 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.023+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v20)=0 v20) v1 ==== 123+0+0 (secure 0 0 0) 0x7fdea01f7510 con 0x7fdea00cae20 2023-11-24T08:13:55.027 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.025+0000 7fde967fc640 1 -- 172.21.15.106:0/1933885508 <== mon.0 v2:172.21.15.106:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fde98088000 con 0x7fdea00cae20 2023-11-24T08:13:55.028 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.026+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fde8406b4a0 msgr2=0x7fde8406d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.026+0000 7fdea8023640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fde8406b4a0 0x7fde8406d960 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fde9c0047b0 tx=0x7fde9c008040 comp rx=0 tx=0).stop 2023-11-24T08:13:55.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.026+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 msgr2=0x7fdea0160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.029 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.026+0000 7fdea8023640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 0x7fdea0160150 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ceb50 tx=0x7fde9800b4a0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 shutdown_connections 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7fdea00cae20 0x7fdea0160150 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7fde8406b4a0 0x7fde8406d960 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7fdea0156880 0x7fdea015fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 --2- 172.21.15.106:0/1933885508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fde8c0047c0 0x7fdea015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 >> 172.21.15.106:0/1933885508 conn(0x7fdea00b2e40 msgr2=0x7fdea00b1300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:55.030 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 shutdown_connections 2023-11-24T08:13:55.031 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.027+0000 7fdea8023640 1 -- 172.21.15.106:0/1933885508 wait complete. 2023-11-24T08:13:55.048 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-24T08:13:55.049 DEBUG:teuthology.orchestra.run.smithi106:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-24T08:13:55.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.229+0000 7f2c5be3d640 1 Processor -- start 2023-11-24T08:13:55.232 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.229+0000 7f2c5be3d640 1 -- start start 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c5be3d640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c540aedf0 0x7f2c540cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c5be3d640 1 --2- >> v2:172.21.15.156:3300/0 conn(0x7f2c5406b510 0x7f2c5406b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c5be3d640 1 -- --> v1:172.21.15.106:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c540ce360 con 0x7f2c5415b310 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c5be3d640 1 -- --> v2:172.21.15.106:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c5406bf00 con 0x7f2c540aedf0 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c5be3d640 1 -- --> v2:172.21.15.156:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c540ce600 con 0x7f2c5406b510 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c593d8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c540aedf0 0x7f2c540cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:55.235 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c593d8640 1 --2- >> v2:172.21.15.106:3300/0 conn(0x7f2c540aedf0 0x7f2c540cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46112/0 (socket says 172.21.15.106:46112) 2023-11-24T08:13:55.236 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.232+0000 7f2c593d8640 1 -- 172.21.15.106:0/3803576362 learned_addr learned my addr 172.21.15.106:0/3803576362 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.233+0000 7f2c5a3da640 1 --2- 172.21.15.106:0/3803576362 >> v2:172.21.15.156:3300/0 conn(0x7f2c5406b510 0x7f2c5406b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.233+0000 7f2c593d8640 1 -- 172.21.15.106:0/3803576362 >> v2:172.21.15.156:3300/0 conn(0x7f2c5406b510 msgr2=0x7f2c5406b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.233+0000 7f2c593d8640 1 --2- 172.21.15.106:0/3803576362 >> v2:172.21.15.156:3300/0 conn(0x7f2c5406b510 0x7f2c5406b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.233+0000 7f2c593d8640 1 -- 172.21.15.106:0/3803576362 >> v1:172.21.15.106:6789/0 conn(0x7f2c5415b310 legacy=0x7f2c540cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.233+0000 7f2c593d8640 1 -- 172.21.15.106:0/3803576362 --> v2:172.21.15.106:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5415b6f0 con 0x7f2c540aedf0 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.233+0000 7f2c5a3da640 1 --2- 172.21.15.106:0/3803576362 >> v2:172.21.15.156:3300/0 conn(0x7f2c5406b510 0x7f2c5406b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-24T08:13:55.237 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.234+0000 7f2c593d8640 1 --2- 172.21.15.106:0/3803576362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c540cc290 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2c540b48e0 tx=0x7f2c4402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e67b15999d580 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.234+0000 7f2c58bd7640 1 -- 172.21.15.106:0/3803576362 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44030860 con 0x7f2c540aedf0 2023-11-24T08:13:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.234+0000 7f2c58bd7640 1 -- 172.21.15.106:0/3803576362 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2c440309c0 con 0x7f2c540aedf0 2023-11-24T08:13:55.238 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.234+0000 7f2c58bd7640 1 -- 172.21.15.106:0/3803576362 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44030d70 con 0x7f2c540aedf0 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 -- 172.21.15.106:0/3803576362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 msgr2=0x7f2c540cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/3803576362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c540cc290 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f2c540b48e0 tx=0x7f2c4402b9a0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 -- 172.21.15.106:0/3803576362 shutdown_connections 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/3803576362 >> v2:172.21.15.156:3300/0 conn(0x7f2c5406b510 0x7f2c5406b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/3803576362 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c540cc290 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 -- 172.21.15.106:0/3803576362 >> 172.21.15.106:0/3803576362 conn(0x7f2c540cfeb0 msgr2=0x7f2c54070db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:55.239 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 -- 172.21.15.106:0/3803576362 shutdown_connections 2023-11-24T08:13:55.240 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.235+0000 7f2c5be3d640 1 -- 172.21.15.106:0/3803576362 wait complete. 2023-11-24T08:13:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.237+0000 7f2c5be3d640 1 Processor -- start 2023-11-24T08:13:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.237+0000 7f2c5be3d640 1 -- start start 2023-11-24T08:13:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c5be3d640 1 --2- >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5406b510 0x7f2c5414d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c5be3d640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c5414bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c5be3d640 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5415b310 0x7f2c54151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:55.242 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c5be3d640 1 -- --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5414d560 con 0x7f2c540aedf0 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c5be3d640 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5415d290 con 0x7f2c5415b310 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c5be3d640 1 -- --> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c5414c110 con 0x7f2c5406b510 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c593d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c5414bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c593d8640 1 --2- >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c5414bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.106:3300/0 says I am v2:172.21.15.106:46126/0 (socket says 172.21.15.106:46126) 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.238+0000 7f2c593d8640 1 -- 172.21.15.106:0/361512779 learned_addr learned my addr 172.21.15.106:0/361512779 (peer_addr_for_me v2:172.21.15.106:0/0) 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c59bd9640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5406b510 0x7f2c5414d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c5a3da640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5415b310 0x7f2c54151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:55.243 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c593d8640 1 -- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5406b510 msgr2=0x7f2c5414d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c593d8640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5406b510 0x7f2c5414d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c593d8640 1 -- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5415b310 msgr2=0x7f2c54151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c593d8640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5415b310 0x7f2c54151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c593d8640 1 -- 172.21.15.106:0/361512779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c440094d0 con 0x7f2c540aedf0 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c593d8640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c5414bbd0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2c44000c00 tx=0x7f2c44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44044480 con 0x7f2c540aedf0 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f2c440445e0 con 0x7f2c540aedf0 2023-11-24T08:13:55.244 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c440449b0 con 0x7f2c540aedf0 2023-11-24T08:13:55.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.239+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c54152850 con 0x7f2c540aedf0 2023-11-24T08:13:55.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.240+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5415d500 con 0x7f2c540aedf0 2023-11-24T08:13:55.245 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.241+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c5415d6f0 con 0x7f2c540aedf0 2023-11-24T08:13:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.242+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 4 ==== mgrmap(e 13) v1 ==== 89792+0+0 (secure 0 0 0) 0x7f2c4404d3b0 con 0x7f2c540aedf0 2023-11-24T08:13:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.242+0000 7f2c4a7fc640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c3806b470 0x7f2c3806d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-24T08:13:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.242+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2c4403e070 con 0x7f2c540aedf0 2023-11-24T08:13:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.243+0000 7f2c59bd9640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c3806b470 0x7f2c3806d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-24T08:13:55.246 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.243+0000 7f2c59bd9640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c3806b470 0x7f2c3806d930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2c5414ccf0 tx=0x7f2c50009040 comp rx=0 tx=0).ready entity=mgr.4110 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-24T08:13:55.249 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.247+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179825 (secure 0 0 0) 0x7f2c440777e0 con 0x7f2c540aedf0 2023-11-24T08:13:55.479 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.477+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 --> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c54067b80 con 0x7f2c540aedf0 2023-11-24T08:13:55.480 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.478+0000 7f2c4a7fc640 1 -- 172.21.15.106:0/361512779 <== mon.0 v2:172.21.15.106:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f2c44049090 con 0x7f2c540aedf0 2023-11-24T08:13:55.481 INFO:teuthology.orchestra.run.smithi106.stdout: 2023-11-24T08:13:55.481 INFO:teuthology.orchestra.run.smithi106.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6834/3756666211","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":3756666211},{"type":"v1","addr":"172.21.15.156:6834","nonce":3756666211}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":5449,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/913392401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6833","nonce":913392401},{"type":"v1","addr":"172.21.15.156:6835","nonce":913392401}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":48},{"gid":7017,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6836/524283010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6834","nonce":524283010},{"type":"v1","addr":"172.21.15.106:6836","nonce":524283010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":47},{"gid":7383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.106:6837/2771191151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.106:6835","nonce":2771191151},{"type":"v1","addr":"172.21.15.106:6837","nonce":2771191151}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":56}],"filesystems":[]} 2023-11-24T08:13:55.481 INFO:teuthology.orchestra.run.smithi106.stderr:dumped fsmap epoch 56 2023-11-24T08:13:55.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.480+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c3806b470 msgr2=0x7f2c3806d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.480+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c3806b470 0x7f2c3806d930 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2c5414ccf0 tx=0x7f2c50009040 comp rx=0 tx=0).stop 2023-11-24T08:13:55.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 msgr2=0x7f2c5414bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-24T08:13:55.483 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c5414bbd0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f2c44000c00 tx=0x7f2c44004730 comp rx=0 tx=0).stop 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 shutdown_connections 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c5415b310 0x7f2c54151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:3300/0,v1:172.21.15.106:6789/0] conn(0x7f2c540aedf0 0x7f2c5414bbd0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.106:6832/2073981325,v1:172.21.15.106:6833/2073981325] conn(0x7f2c3806b470 0x7f2c3806d930 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 --2- 172.21.15.106:0/361512779 >> [v2:172.21.15.156:3301/0,v1:172.21.15.156:6790/0] conn(0x7f2c5406b510 0x7f2c5414d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 >> 172.21.15.106:0/361512779 conn(0x7f2c540cfeb0 msgr2=0x7f2c540b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 shutdown_connections 2023-11-24T08:13:55.484 INFO:teuthology.orchestra.run.smithi106.stderr:2023-11-24T08:13:55.481+0000 7f2c5be3d640 1 -- 172.21.15.106:0/361512779 wait complete. 2023-11-24T08:13:55.501 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-24T08:13:55.502 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-24T08:13:55.502 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-24T08:13:55.502 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.520 INFO:tasks.ceph.mds.a:Stopped 2023-11-24T08:13:55.521 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-24T08:13:55.521 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.584 INFO:tasks.ceph.mds.c:Stopped 2023-11-24T08:13:55.585 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-24T08:13:55.585 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.606 INFO:tasks.ceph.mds.b:Stopped 2023-11-24T08:13:55.606 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-24T08:13:55.606 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.673 INFO:tasks.ceph.mds.d:Stopped 2023-11-24T08:13:55.673 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-24T08:13:55.674 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-24T08:13:55.674 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.749 INFO:tasks.ceph.osd.0:Stopped 2023-11-24T08:13:55.749 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-24T08:13:55.749 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.801 INFO:tasks.ceph.osd.1:Stopped 2023-11-24T08:13:55.802 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-24T08:13:55.802 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.870 INFO:tasks.ceph.osd.2:Stopped 2023-11-24T08:13:55.871 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-24T08:13:55.871 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.923 INFO:tasks.ceph.osd.3:Stopped 2023-11-24T08:13:55.923 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-24T08:13:55.923 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:55.995 INFO:tasks.ceph.osd.4:Stopped 2023-11-24T08:13:55.995 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-24T08:13:55.996 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.078 INFO:tasks.ceph.osd.5:Stopped 2023-11-24T08:13:56.078 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-24T08:13:56.078 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.140 INFO:tasks.ceph.osd.6:Stopped 2023-11-24T08:13:56.140 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-24T08:13:56.140 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.204 INFO:tasks.ceph.osd.7:Stopped 2023-11-24T08:13:56.204 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-24T08:13:56.205 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-24T08:13:56.205 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.280 INFO:tasks.ceph.mgr.y:Stopped 2023-11-24T08:13:56.280 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-24T08:13:56.280 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.371 INFO:tasks.ceph.mgr.x:Stopped 2023-11-24T08:13:56.371 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-24T08:13:56.371 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-24T08:13:56.372 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.411 INFO:tasks.ceph.mon.a:Stopped 2023-11-24T08:13:56.411 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-24T08:13:56.411 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.448 INFO:tasks.ceph.mon.b:Stopped 2023-11-24T08:13:56.448 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-24T08:13:56.448 INFO:teuthology.orchestra.run:waiting for 300 2023-11-24T08:13:56.537 INFO:tasks.ceph.mon.c:Stopped 2023-11-24T08:13:56.538 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-24T08:13:56.538 DEBUG:teuthology.orchestra.run.smithi106:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2023-11-24T08:13:56.655 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:13:56.656 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-24T08:13:57.008 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:13:57.009 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-24T08:13:57.179 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:13:57.179 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-24T08:13:57.339 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:13:57.340 DEBUG:teuthology.orchestra.run.smithi106:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-24T08:13:57.514 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:13:57.515 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-24T08:13:57.778 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:13:57.778 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-24T08:13:57.937 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:13:57.937 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-24T08:13:58.102 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:13:58.103 DEBUG:teuthology.orchestra.run.smithi156:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-24T08:13:58.242 INFO:tasks.ceph:Archiving mon data... 2023-11-24T08:13:58.244 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/data/mon.a.tgz 2023-11-24T08:13:58.244 DEBUG:teuthology.orchestra.run.smithi106:> mktemp 2023-11-24T08:13:58.303 INFO:teuthology.orchestra.run.smithi106.stdout:/tmp/tmp.Cqp1EkfehK 2023-11-24T08:13:58.305 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Cqp1EkfehK 2023-11-24T08:13:58.592 DEBUG:teuthology.orchestra.run.smithi106:> sudo chmod 0666 /tmp/tmp.Cqp1EkfehK 2023-11-24T08:13:58.749 DEBUG:teuthology.orchestra.remote:smithi106:/tmp/tmp.Cqp1EkfehK is 747KB 2023-11-24T08:13:58.864 DEBUG:teuthology.orchestra.run.smithi106:> rm -fr /tmp/tmp.Cqp1EkfehK 2023-11-24T08:13:58.922 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/data/mon.b.tgz 2023-11-24T08:13:58.922 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-11-24T08:13:58.983 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.doJOFh8Igt 2023-11-24T08:13:58.984 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.doJOFh8Igt 2023-11-24T08:13:59.292 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.doJOFh8Igt 2023-11-24T08:13:59.447 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.doJOFh8Igt is 1014KB 2023-11-24T08:13:59.574 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.doJOFh8Igt 2023-11-24T08:13:59.634 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/data/mon.c.tgz 2023-11-24T08:13:59.634 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-11-24T08:13:59.688 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.le715ldv1C 2023-11-24T08:13:59.689 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.le715ldv1C 2023-11-24T08:13:59.977 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.le715ldv1C 2023-11-24T08:14:00.133 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.le715ldv1C is 1014KB 2023-11-24T08:14:00.252 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.le715ldv1C 2023-11-24T08:14:00.311 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-24T08:14:00.311 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:14:00.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-24T08:14:00.420 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-24T08:14:00.421 DEBUG:teuthology.orchestra.run.smithi106:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:14:00.430 DEBUG:teuthology.orchestra.run.smithi156:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-24T08:14:00.528 INFO:teuthology.orchestra.run.smithi106.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:14:00.536 INFO:teuthology.orchestra.run.smithi156.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-24T08:14:00.548 INFO:tasks.ceph:Archiving crash dumps... 2023-11-24T08:14:00.550 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/remote/smithi106/crash 2023-11-24T08:14:00.551 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:14:00.638 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/crash to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/remote/smithi156/crash 2023-11-24T08:14:00.639 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-24T08:14:00.739 INFO:tasks.ceph:Shutting down logrotate 2023-11-24T08:14:00.739 DEBUG:teuthology.orchestra.run.smithi106:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:14:00.748 DEBUG:teuthology.orchestra.run.smithi156:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-24T08:14:00.888 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:14:00.989 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-24T08:14:01.087 INFO:tasks.ceph:Compressing logs... 2023-11-24T08:14:01.087 DEBUG:teuthology.orchestra.run.smithi106:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:14:01.096 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-24T08:14:01.184 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97246.log 2023-11-24T08:14:01.185 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112081.log 2023-11-24T08:14:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97246.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97246.log.gz 2023-11-24T08:14:01.186 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97498.log 2023-11-24T08:14:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112081.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112081.log.gz 2023-11-24T08:14:01.188 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95985.log 2023-11-24T08:14:01.189 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112790.log 2023-11-24T08:14:01.189 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97498.log.gz 2023-11-24T08:14:01.190 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95985.log.gz 2023-11-24T08:14:01.190 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99295.log 2023-11-24T08:14:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112790.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112790.log.gz 2023-11-24T08:14:01.192 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102449.log 2023-11-24T08:14:01.192 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88780.log 2023-11-24T08:14:01.193 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110065.log 2023-11-24T08:14:01.193 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99295.log.gz 2023-11-24T08:14:01.194 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-24T08:14:01.195 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102449.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101623.log 2023-11-24T08:14:01.195 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102449.log.gz 2023-11-24T08:14:01.196 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93855.log 2023-11-24T08:14:01.196 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.110065.log.gz 2023-11-24T08:14:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88660.log 2023-11-24T08:14:01.198 INFO:teuthology.orchestra.run.smithi156.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88780.log.gz 2023-11-24T08:14:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88740.log 2023-11-24T08:14:01.198 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88660.log: 87.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.88660.log.gz 2023-11-24T08:14:01.198 INFO:teuthology.orchestra.run.smithi156.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.88900.log 2023-11-24T08:14:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88740.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88740.log.gz 2023-11-24T08:14:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.95651.log 2023-11-24T08:14:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88900.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:14:01.201 INFO:teuthology.orchestra.run.smithi156.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88900.log.gz 2023-11-24T08:14:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.1.95651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.95651.log.gz 2023-11-24T08:14:01.201 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-24T08:14:01.203 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108509.log 2023-11-24T08:14:01.203 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101623.log.gz 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93855.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93855.log.gz 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105008.log 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108509.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108509.log.gz 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106404.log 88.7% 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105008.log.gz 2023-11-24T08:14:01.204 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94179.log 2023-11-24T08:14:01.205 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101707.log 2023-11-24T08:14:01.205 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106404.log.gz 2023-11-24T08:14:01.205 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105223.log 2023-11-24T08:14:01.205 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94179.log.gz 2023-11-24T08:14:01.208 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101707.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101707.log.gz 2023-11-24T08:14:01.208 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.114274.log 2023-11-24T08:14:01.208 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94933.log 2023-11-24T08:14:01.208 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105223.log.gz 2023-11-24T08:14:01.209 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114274.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.114274.log.gz 2023-11-24T08:14:01.209 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102678.log 2023-11-24T08:14:01.209 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105511.log 2023-11-24T08:14:01.209 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94933.log.gz 2023-11-24T08:14:01.210 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88860.log 2023-11-24T08:14:01.210 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-24T08:14:01.210 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88860.log: 92.9% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-24T08:14:01.211 INFO:teuthology.orchestra.run.smithi156.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.88860.log.gz 2023-11-24T08:14:01.211 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88700.log 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr: 93.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88820.log 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88700.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88700.log.gz 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88820.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.88820.log.gz 2023-11-24T08:14:01.212 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94170.log 2023-11-24T08:14:01.214 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.93167.log 2023-11-24T08:14:01.214 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.1.94170.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.94170.log.gz 2023-11-24T08:14:01.214 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88940.log 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110369.log 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102678.log.gz 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109215.log 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105511.log.gz 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99379.log 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110369.log.gz 2023-11-24T08:14:01.217 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94609.log 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109215.log.gz 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99379.log: gzip -5 88.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.99379.log.gz -- 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr: /var/log/ceph/ceph-client.admin.99043.log 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102762.log 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94609.log.gz 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114454.log 2023-11-24T08:14:01.218 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99043.log.gz 2023-11-24T08:14:01.219 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113607.log 2023-11-24T08:14:01.219 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102762.log.gz 2023-11-24T08:14:01.222 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.1.93167.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.1.93167.log.gz -5 2023-11-24T08:14:01.222 INFO:teuthology.orchestra.run.smithi156.stderr: --verbose -- /var/log/ceph/ceph-client.1.95156.log 2023-11-24T08:14:01.222 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.admin.88940.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88940.log.gz 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.1.95156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.95156.log.gz 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.96128.log 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94679.log 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.1.96128.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.96128.log.gz 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:14:01.223 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.1.94679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.94679.log.gz 2023-11-24T08:14:01.225 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99505.log 2023-11-24T08:14:01.225 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114454.log.gz 2023-11-24T08:14:01.225 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113607.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113607.log.gz 2023-11-24T08:14:01.225 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115375.log 2023-11-24T08:14:01.225 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103933.log 2023-11-24T08:14:01.225 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99505.log.gz 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115375.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115375.log.gz 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105181.log 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102888.log 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103933.log.gz 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105181.log: gzip 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105181.log.gz 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111222.log 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114963.log 2023-11-24T08:14:01.226 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102888.log.gz 2023-11-24T08:14:01.227 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111222.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111222.log.gz 2023-11-24T08:14:01.227 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108305.log 2023-11-24T08:14:01.229 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-24T08:14:01.229 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-24T08:14:01.229 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-mon.b.log: 92.2% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:14:01.229 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-mds.d.log: 93.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-24T08:14:01.230 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.93578.log 2023-11-24T08:14:01.231 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98056.log 2023-11-24T08:14:01.231 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114963.log.gz 2023-11-24T08:14:01.231 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108305.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108305.log.gz 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101528.log 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98056.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112291.log 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98056.log.gz 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101528.log: 88.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.101528.log.gz 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.111138.log 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110555.log 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112291.log.gz 2023-11-24T08:14:01.232 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110597.log 2023-11-24T08:14:01.233 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111138.log.gz 2023-11-24T08:14:01.234 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-24T08:14:01.234 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-client.3.93578.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.93578.log.gz 2023-11-24T08:14:01.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110555.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110555.log.gz 2023-11-24T08:14:01.235 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115021.log 2023-11-24T08:14:01.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95607.log 2023-11-24T08:14:01.235 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.110597.log.gz 2023-11-24T08:14:01.235 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115021.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115021.log.gzgzip 2023-11-24T08:14:01.235 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.98251.log 2023-11-24T08:14:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102407.log 2023-11-24T08:14:01.236 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95607.log.gz 2023-11-24T08:14:01.236 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107846.log 2023-11-24T08:14:01.237 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98251.log.gz 2023-11-24T08:14:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102407.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102407.log.gz 2023-11-24T08:14:01.237 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95274.log 2023-11-24T08:14:01.238 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95278.log 2023-11-24T08:14:01.238 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107846.log.gz 2023-11-24T08:14:01.239 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95274.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95274.log.gzgzip 2023-11-24T08:14:01.239 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.106703.log 2023-11-24T08:14:01.240 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112748.log 2023-11-24T08:14:01.240 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95278.log.gz 2023-11-24T08:14:01.241 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106703.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108431.log 2023-11-24T08:14:01.241 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.106703.log.gz 2023-11-24T08:14:01.242 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107762.log 2023-11-24T08:14:01.243 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112748.log.gz 2023-11-24T08:14:01.243 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108431.log: gzip -5 --verbose -- 87.8% /var/log/ceph/ceph-client.admin.113341.log 2023-11-24T08:14:01.243 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108431.log.gz 2023-11-24T08:14:01.244 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107305.log 2023-11-24T08:14:01.244 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107762.log.gz 2023-11-24T08:14:01.245 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113341.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.113341.log.gz 2023-11-24T08:14:01.245 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106152.log 2023-11-24T08:14:01.246 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114795.log 2023-11-24T08:14:01.246 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107305.log.gz 2023-11-24T08:14:01.247 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106152.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106152.log.gz 2023-11-24T08:14:01.248 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103271.log 2023-11-24T08:14:01.248 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.107684.log 2023-11-24T08:14:01.248 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114795.log.gz 2023-11-24T08:14:01.249 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104474.log 2023-11-24T08:14:01.249 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103271.log.gz 2023-11-24T08:14:01.250 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.107684.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.107684.log.gz 2023-11-24T08:14:01.251 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111855.log 2023-11-24T08:14:01.251 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103577.log 2023-11-24T08:14:01.251 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104474.log.gz 2023-11-24T08:14:01.253 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97540.log 2023-11-24T08:14:01.253 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111855.log.gz 2023-11-24T08:14:01.253 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103577.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103577.log.gz 2023-11-24T08:14:01.254 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103849.log 2023-11-24T08:14:01.255 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110157.log 2023-11-24T08:14:01.255 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97540.log.gz 2023-11-24T08:14:01.256 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115879.log 2023-11-24T08:14:01.256 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103849.log.gz 2023-11-24T08:14:01.257 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110157.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110157.log.gz 2023-11-24T08:14:01.257 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107431.log 2023-11-24T08:14:01.259 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110639.log 2023-11-24T08:14:01.259 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115879.log.gz 2023-11-24T08:14:01.259 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107431.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.107431.log.gz 2023-11-24T08:14:01.260 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112706.log 2023-11-24T08:14:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107263.log 2023-11-24T08:14:01.261 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.110639.log.gz 2023-11-24T08:14:01.261 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105469.log 2023-11-24T08:14:01.262 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112706.log.gz 2023-11-24T08:14:01.263 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104516.log 2023-11-24T08:14:01.263 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107263.log.gz 2023-11-24T08:14:01.264 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111693.log 2023-11-24T08:14:01.265 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105469.log.gz 2023-11-24T08:14:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104516.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104516.log.gz 2023-11-24T08:14:01.265 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95649.log 2023-11-24T08:14:01.266 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88773.log 2023-11-24T08:14:01.266 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111693.log.gz 2023-11-24T08:14:01.267 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105965.log 2023-11-24T08:14:01.267 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95649.log.gz 2023-11-24T08:14:01.268 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.88773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108987.log 2023-11-24T08:14:01.268 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.88773.log.gz 2023-11-24T08:14:01.269 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105965.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105965.log.gz 2023-11-24T08:14:01.270 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107009.log 2023-11-24T08:14:01.271 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113733.log 2023-11-24T08:14:01.272 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107009.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108987.log.gz 2023-11-24T08:14:01.273 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.107009.log.gz 2023-11-24T08:14:01.273 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98140.log 2023-11-24T08:14:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116593.log 2023-11-24T08:14:01.275 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113733.log.gz 2023-11-24T08:14:01.275 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99001.log 2023-11-24T08:14:01.275 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98140.log.gz 2023-11-24T08:14:01.276 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100447.log 2023-11-24T08:14:01.276 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116593.log.gz 2023-11-24T08:14:01.277 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99001.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99001.log.gz 2023-11-24T08:14:01.278 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112874.log 2023-11-24T08:14:01.278 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116820.log 2023-11-24T08:14:01.279 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100447.log.gz 2023-11-24T08:14:01.282 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112874.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.112874.log.gz 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114717.log 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99547.log 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116820.log.gz 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114717.log: gzip 89.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.114717.log.gz 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.112207.log 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94945.log 2023-11-24T08:14:01.283 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99547.log.gz 2023-11-24T08:14:01.284 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112207.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112207.log.gz 2023-11-24T08:14:01.284 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104881.log 2023-11-24T08:14:01.284 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/ceph-mon.c.log: 94.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-24T08:14:01.285 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98581.log 2023-11-24T08:14:01.286 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94945.log.gz 2023-11-24T08:14:01.286 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113775.log 2023-11-24T08:14:01.286 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104881.log.gz 2023-11-24T08:14:01.286 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98581.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98581.log.gz 2023-11-24T08:14:01.286 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116005.log 2023-11-24T08:14:01.287 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113299.log 2023-11-24T08:14:01.287 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113775.log.gz 2023-11-24T08:14:01.288 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116635.log 2023-11-24T08:14:01.288 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116005.log.gz 2023-11-24T08:14:01.289 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99937.log 2023-11-24T08:14:01.289 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.113299.log.gz 2023-11-24T08:14:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116635.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116635.log.gz 2023-11-24T08:14:01.290 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116383.log 2023-11-24T08:14:01.291 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107804.log 88.7% 2023-11-24T08:14:01.291 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99937.log.gz 2023-11-24T08:14:01.292 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107137.log 2023-11-24T08:14:01.292 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116383.log.gz 2023-11-24T08:14:01.293 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115604.log 2023-11-24T08:14:01.293 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107804.log.gz 2023-11-24T08:14:01.294 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108068.log 2023-11-24T08:14:01.294 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107137.log.gz 2023-11-24T08:14:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115604.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115604.log.gz 2023-11-24T08:14:01.295 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107221.log 2023-11-24T08:14:01.296 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116299.log 2023-11-24T08:14:01.296 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108068.log.gz 2023-11-24T08:14:01.297 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107221.log: gzip 88.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.107221.log.gz --verbose 2023-11-24T08:14:01.297 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.104017.log 2023-11-24T08:14:01.298 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98797.log 2023-11-24T08:14:01.298 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116299.log.gz 2023-11-24T08:14:01.299 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104017.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104017.log.gz 2023-11-24T08:14:01.299 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95282.log 2023-11-24T08:14:01.300 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101665.log 2023-11-24T08:14:01.300 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98797.log.gz 2023-11-24T08:14:01.301 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95282.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95282.log.gz 2023-11-24T08:14:01.301 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106109.log 2023-11-24T08:14:01.302 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116089.log 2023-11-24T08:14:01.302 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101665.log.gz 2023-11-24T08:14:01.303 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106109.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106109.log.gz 2023-11-24T08:14:01.303 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104656.log 2023-11-24T08:14:01.304 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108735.log 2023-11-24T08:14:01.304 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.116089.log.gz 2023-11-24T08:14:01.305 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104656.log: gzip 88.1% -5 -- replaced with /var/log/ceph/ceph-client.admin.104656.log.gz 2023-11-24T08:14:01.305 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.110681.log 2023-11-24T08:14:01.306 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108735.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108735.log.gz 2023-11-24T08:14:01.307 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103663.log 2023-11-24T08:14:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110681.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110681.log.gz 2023-11-24T08:14:01.308 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111431.log 2023-11-24T08:14:01.309 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113901.log 2023-11-24T08:14:01.309 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103663.log.gz 2023-11-24T08:14:01.310 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103125.log 2023-11-24T08:14:01.310 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111431.log.gz 2023-11-24T08:14:01.311 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105139.log 2023-11-24T08:14:01.312 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103125.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.113901.log.gz 2023-11-24T08:14:01.312 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103125.log.gz 2023-11-24T08:14:01.312 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103975.log 2023-11-24T08:14:01.313 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93599.log 2023-11-24T08:14:01.313 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105139.log.gz 2023-11-24T08:14:01.314 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106959.log 89.0% 2023-11-24T08:14:01.314 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.103975.log.gz 2023-11-24T08:14:01.315 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103187.log 2023-11-24T08:14:01.316 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93599.log.gz 2023-11-24T08:14:01.316 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110411.log 2023-11-24T08:14:01.316 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106959.log.gz 2023-11-24T08:14:01.318 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94600.log 2023-11-24T08:14:01.318 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103187.log.gz 2023-11-24T08:14:01.319 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110411.log: 88.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.110411.log.gz -5 2023-11-24T08:14:01.319 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.96069.log 2023-11-24T08:14:01.320 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102281.log 2023-11-24T08:14:01.320 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94600.log.gz 2023-11-24T08:14:01.321 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.96069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94937.log 2023-11-24T08:14:01.321 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.96069.log.gz 2023-11-24T08:14:01.322 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97625.log 2023-11-24T08:14:01.323 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102281.log.gz 2023-11-24T08:14:01.323 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102491.log 2023-11-24T08:14:01.323 INFO:teuthology.orchestra.run.smithi106.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94937.log.gz 2023-11-24T08:14:01.324 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94947.log 2023-11-24T08:14:01.325 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97625.log.gz 2023-11-24T08:14:01.325 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102491.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102491.log.gz 2023-11-24T08:14:01.325 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100191.log 2023-11-24T08:14:01.326 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100275.log 2023-11-24T08:14:01.327 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94947.log.gz 2023-11-24T08:14:01.327 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100191.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.100191.log.gz 2023-11-24T08:14:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88900.log 2023-11-24T08:14:01.328 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105821.log 2023-11-24T08:14:01.329 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100275.log.gz 2023-11-24T08:14:01.329 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.88900.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.88900.log.gz 2023-11-24T08:14:01.330 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103229.log 2023-11-24T08:14:01.331 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108263.log 2023-11-24T08:14:01.331 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105821.log.gz 2023-11-24T08:14:01.332 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108389.log 2023-11-24T08:14:01.332 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103229.log.gz 2023-11-24T08:14:01.333 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105385.log 2023-11-24T08:14:01.333 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108263.log.gz 2023-11-24T08:14:01.334 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94943.log 2023-11-24T08:14:01.335 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108389.log.gz 2023-11-24T08:14:01.335 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103807.log 2023-11-24T08:14:01.336 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105385.log.gz 2023-11-24T08:14:01.336 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97582.log 2023-11-24T08:14:01.337 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94943.log.gz 2023-11-24T08:14:01.337 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103807.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103807.log.gz 2023-11-24T08:14:01.338 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105307.log 2023-11-24T08:14:01.339 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.114154.log 2023-11-24T08:14:01.339 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97582.log.gz 2023-11-24T08:14:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105307.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105307.log.gz 2023-11-24T08:14:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115207.log 2023-11-24T08:14:01.340 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.114154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.114154.log.gz 2023-11-24T08:14:01.341 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109935.log 2023-11-24T08:14:01.342 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115207.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97671.log 2023-11-24T08:14:01.342 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115207.log.gz 2023-11-24T08:14:01.343 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113479.log 2023-11-24T08:14:01.343 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109935.log.gz 2023-11-24T08:14:01.344 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109426.log 2023-11-24T08:14:01.344 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.97671.log.gz 2023-11-24T08:14:01.345 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99855.log 2023-11-24T08:14:01.345 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.113479.log.gz 2023-11-24T08:14:01.346 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109426.log: gzip 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109426.log.gz 2023-11-24T08:14:01.346 INFO:teuthology.orchestra.run.smithi106.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.110453.log 2023-11-24T08:14:01.347 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99855.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-24T08:14:01.347 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99855.log.gz 2023-11-24T08:14:01.348 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94053.log 2023-11-24T08:14:01.348 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110453.log.gz 2023-11-24T08:14:01.351 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99211.log 2023-11-24T08:14:01.351 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100403.log 2023-11-24T08:14:01.351 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94053.log.gz 2023-11-24T08:14:01.351 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-24T08:14:01.351 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102533.log 2023-11-24T08:14:01.352 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99211.log.gz 2023-11-24T08:14:01.353 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100403.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100403.log.gz 2023-11-24T08:14:01.353 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103313.log 2023-11-24T08:14:01.354 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102533.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102533.log.gz 2023-11-24T08:14:01.355 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113091.log 2023-11-24T08:14:01.355 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113257.log 2023-11-24T08:14:01.355 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103313.log.gz 2023-11-24T08:14:01.356 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99085.log 2023-11-24T08:14:01.356 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.113091.log.gz 2023-11-24T08:14:01.356 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108861.log 2023-11-24T08:14:01.357 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113257.log.gz 2023-11-24T08:14:01.358 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96579.log 2023-11-24T08:14:01.358 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99085.log.gz 2023-11-24T08:14:01.358 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108861.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108861.log.gz 2023-11-24T08:14:01.359 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106049.log 2023-11-24T08:14:01.359 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.96579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96579.log.gz 2023-11-24T08:14:01.360 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113214.log 2023-11-24T08:14:01.360 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99733.log 2023-11-24T08:14:01.360 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106049.log.gz 2023-11-24T08:14:01.361 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114921.log 2023-11-24T08:14:01.361 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.113214.log.gz 2023-11-24T08:14:01.362 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103355.log 2023-11-24T08:14:01.362 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99733.log.gz 2023-11-24T08:14:01.363 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101389.log 2023-11-24T08:14:01.363 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.114921.log.gz 2023-11-24T08:14:01.364 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111276.log 2023-11-24T08:14:01.364 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103355.log.gz 2023-11-24T08:14:01.365 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.104354.log 2023-11-24T08:14:01.365 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.101389.log.gz 2023-11-24T08:14:01.366 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106621.log 2023-11-24T08:14:01.366 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111276.log.gz 2023-11-24T08:14:01.367 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.104354.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.104354.log.gz 2023-11-24T08:14:01.367 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94604.log 2023-11-24T08:14:01.368 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107930.log 2023-11-24T08:14:01.368 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.106621.log.gz 2023-11-24T08:14:01.369 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94604.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94604.log.gz 2023-11-24T08:14:01.369 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95817.log 2023-11-24T08:14:01.370 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102197.log 2023-11-24T08:14:01.370 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107930.log.gz 2023-11-24T08:14:01.371 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106541.log 2023-11-24T08:14:01.371 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95817.log.gz 2023-11-24T08:14:01.372 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114879.log 2023-11-24T08:14:01.372 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102197.log.gz 2023-11-24T08:14:01.373 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93518.log 2023-11-24T08:14:01.373 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106541.log.gz 2023-11-24T08:14:01.374 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95481.log 2023-11-24T08:14:01.374 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114879.log.gz 2023-11-24T08:14:01.375 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108109.log 2023-11-24T08:14:01.375 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93518.log.gz 2023-11-24T08:14:01.376 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95270.log 2023-11-24T08:14:01.376 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95481.log.gz 2023-11-24T08:14:01.377 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99127.log 2023-11-24T08:14:01.377 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.108109.log.gz 2023-11-24T08:14:01.378 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95270.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95270.log.gz 2023-11-24T08:14:01.378 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110243.log 2023-11-24T08:14:01.379 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115105.log 2023-11-24T08:14:01.379 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.99127.log.gz 2023-11-24T08:14:01.380 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110021.log 2023-11-24T08:14:01.380 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110243.log.gz 2023-11-24T08:14:01.381 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106829.log 2023-11-24T08:14:01.381 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115105.log.gz 2023-11-24T08:14:01.382 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98335.log 2023-11-24T08:14:01.382 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.110021.log.gz 2023-11-24T08:14:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106829.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106829.log.gz 2023-11-24T08:14:01.383 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105265.log 2023-11-24T08:14:01.384 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110789.log 2023-11-24T08:14:01.384 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98335.log.gz 2023-11-24T08:14:01.385 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103443.log 2023-11-24T08:14:01.385 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105265.log.gz 2023-11-24T08:14:01.386 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115249.log 2023-11-24T08:14:01.386 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110789.log.gz 2023-11-24T08:14:01.387 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114591.log 2023-11-24T08:14:01.387 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.103443.log.gz 2023-11-24T08:14:01.388 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115688.log 2023-11-24T08:14:01.388 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.115249.log.gz 2023-11-24T08:14:01.389 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113967.log 2023-11-24T08:14:01.389 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114591.log.gz 2023-11-24T08:14:01.390 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109727.log 2023-11-24T08:14:01.390 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115688.log.gz 2023-11-24T08:14:01.391 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100836.log 2023-11-24T08:14:01.391 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113967.log.gz 2023-11-24T08:14:01.392 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109727.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109727.log.gz 2023-11-24T08:14:01.392 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114400.log 2023-11-24T08:14:01.394 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100836.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100836.log.gz 2023-11-24T08:14:01.394 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98923.log 2023-11-24T08:14:01.395 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114400.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114400.log.gz 2023-11-24T08:14:01.395 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95943.log 2023-11-24T08:14:01.396 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102592.log 2023-11-24T08:14:01.397 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98923.log.gz 2023-11-24T08:14:01.397 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102365.log 2023-11-24T08:14:01.398 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95943.log.gz 2023-11-24T08:14:01.398 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106194.log 2023-11-24T08:14:01.398 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102592.log.gz 2023-11-24T08:14:01.399 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94347.log 2023-11-24T08:14:01.399 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102365.log.gz 2023-11-24T08:14:01.400 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104740.log 2023-11-24T08:14:01.400 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.106194.log.gz 2023-11-24T08:14:01.401 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104558.log 2023-11-24T08:14:01.401 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.94347.log.gz 2023-11-24T08:14:01.402 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104740.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104740.log.gz 2023-11-24T08:14:01.402 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95901.log 2023-11-24T08:14:01.403 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95775.log 2023-11-24T08:14:01.403 INFO:teuthology.orchestra.run.smithi106.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.104558.log.gz 2023-11-24T08:14:01.404 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100625.log 2023-11-24T08:14:01.404 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95901.log.gz 2023-11-24T08:14:01.405 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95775.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95775.log.gz 2023-11-24T08:14:01.405 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109071.log 2023-11-24T08:14:01.406 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100625.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100625.log.gz 2023-11-24T08:14:01.407 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105881.log 2023-11-24T08:14:01.407 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109071.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109071.log.gz 2023-11-24T08:14:01.407 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104101.log 2023-11-24T08:14:01.408 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105881.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105881.log.gz 2023-11-24T08:14:01.408 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107179.log 2023-11-24T08:14:01.409 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100359.log 2023-11-24T08:14:01.410 INFO:teuthology.orchestra.run.smithi106.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.104101.log.gz 2023-11-24T08:14:01.410 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107179.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107179.log.gz 2023-11-24T08:14:01.411 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-11-24T08:14:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98014.log 2023-11-24T08:14:01.412 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.1.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100359.log.gz 2023-11-24T08:14:01.413 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106446.log 2023-11-24T08:14:01.414 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98014.log: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.98014.log.gz 2023-11-24T08:14:01.415 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102720.log 2023-11-24T08:14:01.416 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109850.log 2023-11-24T08:14:01.416 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106446.log.gz 2023-11-24T08:14:01.417 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-24T08:14:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102720.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102720.log.gz 2023-11-24T08:14:01.418 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112832.log 2023-11-24T08:14:01.419 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95523.log 2023-11-24T08:14:01.419 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109850.log.gz 2023-11-24T08:14:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112832.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.112832.log.gz 2023-11-24T08:14:01.420 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100986.log 2023-11-24T08:14:01.422 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93710.log 2023-11-24T08:14:01.422 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95523.log.gz 2023-11-24T08:14:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100986.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100986.log.gz 2023-11-24T08:14:01.423 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100233.log 2023-11-24T08:14:01.425 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101252.log 2023-11-24T08:14:01.425 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100233.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.93710.log.gz 2023-11-24T08:14:01.426 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100233.log.gz 2023-11-24T08:14:01.426 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113563.log 2023-11-24T08:14:01.428 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95565.log 2023-11-24T08:14:01.428 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101252.log.gz 2023-11-24T08:14:01.429 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113563.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113563.log.gz 2023-11-24T08:14:01.429 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98539.log 2023-11-24T08:14:01.431 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94612.log 2023-11-24T08:14:01.431 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98539.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.95565.log.gz 2023-11-24T08:14:01.432 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98539.log.gz 2023-11-24T08:14:01.433 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115147.log 2023-11-24T08:14:01.434 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110115.log 2023-11-24T08:14:01.435 INFO:teuthology.orchestra.run.smithi106.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94612.log.gz 2023-11-24T08:14:01.435 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115147.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.115147.log.gz 2023-11-24T08:14:01.436 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109552.log 2023-11-24T08:14:01.437 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97761.log 2023-11-24T08:14:01.437 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.110115.log.gz 2023-11-24T08:14:01.438 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109552.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109552.log.gz 2023-11-24T08:14:01.439 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113385.log 2023-11-24T08:14:01.440 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110723.log 2023-11-24T08:14:01.440 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97761.log.gz 2023-11-24T08:14:01.441 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98098.log 2023-11-24T08:14:01.441 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113385.log.gz 2023-11-24T08:14:01.442 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106787.log 2023-11-24T08:14:01.443 INFO:teuthology.orchestra.run.smithi106.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110723.log.gz 2023-11-24T08:14:01.444 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98098.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98098.log.gz 2023-11-24T08:14:01.444 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94221.log 2023-11-24T08:14:01.445 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106787.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106787.log.gz 2023-11-24T08:14:01.445 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104698.log 2023-11-24T08:14:01.446 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94221.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94221.log.gz 2023-11-24T08:14:01.446 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112561.log 2023-11-24T08:14:01.447 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104698.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104698.log.gz 2023-11-24T08:14:01.448 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106278.log 2023-11-24T08:14:01.448 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112561.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112561.log.gz 2023-11-24T08:14:01.449 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108593.log 2023-11-24T08:14:01.449 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106278.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106278.log.gz 2023-11-24T08:14:01.450 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109300.log 2023-11-24T08:14:01.451 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108593.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108593.log.gz 2023-11-24T08:14:01.451 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88516.log 2023-11-24T08:14:01.452 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109300.log: gzip 89.2% -5 -- replaced with /var/log/ceph/ceph-client.admin.109300.log.gz 2023-11-24T08:14:01.452 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.100021.log 2023-11-24T08:14:01.453 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.88516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95285.log 2023-11-24T08:14:01.454 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.88516.log.gz 2023-11-24T08:14:01.454 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100021.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.100021.log.gz 2023-11-24T08:14:01.455 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105553.log 2023-11-24T08:14:01.456 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99589.log 2023-11-24T08:14:01.456 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95285.log.gz 2023-11-24T08:14:01.457 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105553.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.105553.log.gz 2023-11-24T08:14:01.457 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97888.log 2023-11-24T08:14:01.458 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100752.log 2023-11-24T08:14:01.458 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99589.log.gz 2023-11-24T08:14:01.459 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97888.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.97888.log.gz 2023-11-24T08:14:01.459 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116131.log 2023-11-24T08:14:01.460 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113817.log 2023-11-24T08:14:01.461 INFO:teuthology.orchestra.run.smithi106.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.100752.log.gz 2023-11-24T08:14:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116131.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116131.log.gz 2023-11-24T08:14:01.462 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101347.log 2023-11-24T08:14:01.463 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97144.log 2023-11-24T08:14:01.463 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113817.log.gz 2023-11-24T08:14:01.464 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101347.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101347.log.gz 2023-11-24T08:14:01.464 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98755.log 2023-11-24T08:14:01.465 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109510.log 2023-11-24T08:14:01.466 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.97144.log.gz 2023-11-24T08:14:01.466 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98755.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98755.log.gz 2023-11-24T08:14:01.467 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-24T08:14:01.467 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102155.log 2023-11-24T08:14:01.468 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113521.log 2023-11-24T08:14:01.468 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.109510.log.gz 2023-11-24T08:14:01.469 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102155.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102155.log.gz 2023-11-24T08:14:01.469 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101305.log 2023-11-24T08:14:01.470 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109647.log 2023-11-24T08:14:01.470 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113521.log.gz 2023-11-24T08:14:01.471 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101305.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101305.log.gz 2023-11-24T08:14:01.472 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116215.log 2023-11-24T08:14:01.473 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115417.log 2023-11-24T08:14:01.473 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109647.log.gz 2023-11-24T08:14:01.474 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116215.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.116215.log.gz 2023-11-24T08:14:01.474 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109468.log 2023-11-24T08:14:01.475 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116719.log 2023-11-24T08:14:01.476 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115417.log.gz 2023-11-24T08:14:01.476 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109468.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.109468.log.gz 2023-11-24T08:14:01.477 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106007.log 2023-11-24T08:14:01.478 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112375.log 2023-11-24T08:14:01.478 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116719.log.gz 2023-11-24T08:14:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106007.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106007.log.gz 2023-11-24T08:14:01.479 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112604.log 2023-11-24T08:14:01.480 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112375.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2023-11-24T08:14:01.481 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.112375.log.gz 2023-11-24T08:14:01.481 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112604.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112604.log.gz 2023-11-24T08:14:01.482 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107984.log 2023-11-24T08:14:01.483 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100710.log 2023-11-24T08:14:01.484 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107984.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.107984.log.gz 2023-11-24T08:14:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111371.log 2023-11-24T08:14:01.485 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100710.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100710.log.gz 2023-11-24T08:14:01.486 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105611.log 2023-11-24T08:14:01.487 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95733.log 2023-11-24T08:14:01.488 INFO:teuthology.orchestra.run.smithi106.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.111371.log.gz 2023-11-24T08:14:01.488 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105611.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105611.log.gz 2023-11-24T08:14:01.489 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106745.log 2023-11-24T08:14:01.490 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95691.log 88.7% 2023-11-24T08:14:01.490 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.95733.log.gz 2023-11-24T08:14:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106745.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106745.log.gz 2023-11-24T08:14:01.492 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99421.log 2023-11-24T08:14:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95691.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95691.log.gz 2023-11-24T08:14:01.493 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110199.log 2023-11-24T08:14:01.494 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112501.log 2023-11-24T08:14:01.495 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99421.log.gz 2023-11-24T08:14:01.495 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110199.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110199.log.gz 2023-11-24T08:14:01.496 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116467.log 2023-11-24T08:14:01.497 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112501.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112501.log.gz 2023-11-24T08:14:01.497 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99337.log 2023-11-24T08:14:01.498 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116047.log 2023-11-24T08:14:01.499 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116467.log.gz 2023-11-24T08:14:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99337.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99337.log.gz 2023-11-24T08:14:01.500 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97972.log 2023-11-24T08:14:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116047.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116047.log.gz 2023-11-24T08:14:01.501 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112249.log 2023-11-24T08:14:01.502 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109155.log 2023-11-24T08:14:01.503 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112249.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97972.log.gz 2023-11-24T08:14:01.504 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112249.log.gz 2023-11-24T08:14:01.504 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95250.log 2023-11-24T08:14:01.505 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109155.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.109155.log.gz 2023-11-24T08:14:01.506 INFO:teuthology.orchestra.run.smithi156.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-24T08:14:01.506 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111981.log 2023-11-24T08:14:01.507 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93766.log 2023-11-24T08:14:01.507 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95250.log.gz 2023-11-24T08:14:01.508 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111981.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111981.log.gz 2023-11-24T08:14:01.508 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106915.log 2023-11-24T08:14:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93766.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93766.log.gz 2023-11-24T08:14:01.510 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94934.log 2023-11-24T08:14:01.511 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106915.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106915.log.gz 2023-11-24T08:14:01.511 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97456.log 2023-11-24T08:14:01.512 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94934.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94934.log.gz 2023-11-24T08:14:01.513 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107389.log 2023-11-24T08:14:01.514 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109808.log 2023-11-24T08:14:01.515 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97456.log.gz 2023-11-24T08:14:01.515 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107389.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107389.log.gz 2023-11-24T08:14:01.516 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100063.log 2023-11-24T08:14:01.517 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109808.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.109808.log.gz 2023-11-24T08:14:01.517 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115921.log 2023-11-24T08:14:01.518 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100063.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100063.log.gz 2023-11-24T08:14:01.519 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98623.log 2023-11-24T08:14:01.520 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109258.log 2023-11-24T08:14:01.520 INFO:teuthology.orchestra.run.smithi106.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115921.log.gz 2023-11-24T08:14:01.521 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98623.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98623.log.gz 2023-11-24T08:14:01.521 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94095.log 2023-11-24T08:14:01.522 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109258.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.109258.log.gz 2023-11-24T08:14:01.523 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104600.log 2023-11-24T08:14:01.524 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94305.log 2023-11-24T08:14:01.524 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94095.log.gz 2023-11-24T08:14:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104600.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104600.log.gz 2023-11-24T08:14:01.525 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97204.log 2023-11-24T08:14:01.526 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94305.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94305.log.gz 2023-11-24T08:14:01.527 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97372.log 2023-11-24T08:14:01.528 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97204.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101869.log 2023-11-24T08:14:01.529 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97204.log.gz 2023-11-24T08:14:01.529 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97372.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97372.log.gz 2023-11-24T08:14:01.530 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101995.log 2023-11-24T08:14:01.530 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101869.log: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.101869.log.gz 2023-11-24T08:14:01.531 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97330.log 2023-11-24T08:14:01.532 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112417.log 2023-11-24T08:14:01.533 INFO:teuthology.orchestra.run.smithi156.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-24T08:14:01.533 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.101995.log.gz 2023-11-24T08:14:01.533 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97330.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97330.log.gz 2023-11-24T08:14:01.534 INFO:teuthology.orchestra.run.smithi156.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-24T08:14:01.534 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98665.log 2023-11-24T08:14:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112417.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.112417.log.gz 2023-11-24T08:14:01.535 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108347.log 2023-11-24T08:14:01.536 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98665.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98665.log.gz 2023-11-24T08:14:01.537 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113859.log 2023-11-24T08:14:01.538 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94610.log 2023-11-24T08:14:01.538 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108347.log.gz 2023-11-24T08:14:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113859.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113859.log.gz 2023-11-24T08:14:01.539 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110495.log 2023-11-24T08:14:01.541 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94949.log 2023-11-24T08:14:01.541 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94610.log.gz 2023-11-24T08:14:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110495.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.110495.log.gz 2023-11-24T08:14:01.542 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93813.log 2023-11-24T08:14:01.543 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94596.log 2023-11-24T08:14:01.544 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94949.log.gz 2023-11-24T08:14:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93813.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.93813.log.gz 2023-11-24T08:14:01.545 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105779.log 2023-11-24T08:14:01.546 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94596.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94596.log.gz 2023-11-24T08:14:01.547 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103705.log 2023-11-24T08:14:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105779.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105779.log.gz 2023-11-24T08:14:01.548 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100106.log 2023-11-24T08:14:01.549 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103705.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103705.log.gz 2023-11-24T08:14:01.549 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94946.log 2023-11-24T08:14:01.550 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115291.log 2023-11-24T08:14:01.551 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100106.log.gz 2023-11-24T08:14:01.551 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94946.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.94946.log.gz 2023-11-24T08:14:01.552 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95284.log 2023-11-24T08:14:01.553 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115562.log 2023-11-24T08:14:01.554 INFO:teuthology.orchestra.run.smithi106.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.115291.log.gz 2023-11-24T08:14:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95284.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.95284.log.gz 2023-11-24T08:14:01.555 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113429.log 2023-11-24T08:14:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115562.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115562.log.gz 2023-11-24T08:14:01.556 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104432.log 2023-11-24T08:14:01.557 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115333.log 2023-11-24T08:14:01.558 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113429.log.gz 2023-11-24T08:14:01.558 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104432.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104432.log.gz 2023-11-24T08:14:01.559 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113649.log 2023-11-24T08:14:01.560 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115333.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.115333.log.gz 2023-11-24T08:14:01.560 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110285.log 2023-11-24T08:14:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113649.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113649.log.gz 2023-11-24T08:14:01.562 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99463.log 2023-11-24T08:14:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110285.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.110285.log.gz 2023-11-24T08:14:01.563 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108819.log 2023-11-24T08:14:01.564 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109029.log 2023-11-24T08:14:01.565 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108819.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99463.log.gz 2023-11-24T08:14:01.565 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.108819.log.gz 2023-11-24T08:14:01.566 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115459.log 2023-11-24T08:14:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109029.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109029.log.gz 2023-11-24T08:14:01.567 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106320.log 2023-11-24T08:14:01.568 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115459.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-24T08:14:01.569 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.115459.log.gz 2023-11-24T08:14:01.570 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106320.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.106320.log.gz 2023-11-24T08:14:01.570 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101953.log 2023-11-24T08:14:01.571 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101431.log 2023-11-24T08:14:01.573 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108677.log 2023-11-24T08:14:01.573 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101953.log.gz 2023-11-24T08:14:01.574 INFO:teuthology.orchestra.run.smithi156.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-24T08:14:01.574 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108635.log 2023-11-24T08:14:01.575 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101431.log.gz 2023-11-24T08:14:01.576 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97719.log 88.8% 2023-11-24T08:14:01.576 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108677.log.gz 2023-11-24T08:14:01.578 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98839.log 2023-11-24T08:14:01.578 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108635.log.gz 2023-11-24T08:14:01.579 INFO:teuthology.orchestra.run.smithi156.stderr: 2023-11-24T08:14:01.579 INFO:teuthology.orchestra.run.smithi156.stderr:real 0m0.428s 2023-11-24T08:14:01.579 INFO:teuthology.orchestra.run.smithi156.stderr:user 0m1.703s 2023-11-24T08:14:01.579 INFO:teuthology.orchestra.run.smithi156.stderr:sys 0m0.284s 2023-11-24T08:14:01.580 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102804.log 2023-11-24T08:14:01.580 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.97719.log.gz 2023-11-24T08:14:01.581 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94011.log 2023-11-24T08:14:01.581 INFO:teuthology.orchestra.run.smithi106.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98839.log.gz 2023-11-24T08:14:01.582 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111180.log 2023-11-24T08:14:01.582 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.102804.log.gz 2023-11-24T08:14:01.583 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94011.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.94011.log.gz -5 2023-11-24T08:14:01.583 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.105737.log 2023-11-24T08:14:01.585 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111180.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111180.log.gz 2023-11-24T08:14:01.585 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114358.log 2023-11-24T08:14:01.587 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102323.log 2023-11-24T08:14:01.587 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105737.log.gz 2023-11-24T08:14:01.588 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97804.log 2023-11-24T08:14:01.588 INFO:teuthology.orchestra.run.smithi106.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.114358.log.gz 2023-11-24T08:14:01.590 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105923.log 2023-11-24T08:14:01.590 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102323.log.gz 2023-11-24T08:14:01.591 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109342.log 2023-11-24T08:14:01.591 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.97804.log.gz 2023-11-24T08:14:01.593 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111777.log 2023-11-24T08:14:01.593 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105923.log.gz 2023-11-24T08:14:01.594 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102037.log 2023-11-24T08:14:01.594 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109342.log.gz 2023-11-24T08:14:01.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96027.log 2023-11-24T08:14:01.598 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111777.log.gz 2023-11-24T08:14:01.598 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98377.log 2023-11-24T08:14:01.598 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102037.log.gz 2023-11-24T08:14:01.599 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.96027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93656.log 2023-11-24T08:14:01.599 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.96027.log.gz 2023-11-24T08:14:01.600 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98419.log 2023-11-24T08:14:01.600 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98377.log.gz 2023-11-24T08:14:01.601 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93656.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.82432.log 2023-11-24T08:14:01.602 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93656.log.gz 2023-11-24T08:14:01.603 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112123.log 2023-11-24T08:14:01.603 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.98419.log.gz 2023-11-24T08:14:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.tmp-client.admin.82432.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.82432.log.gz 2023-11-24T08:14:01.604 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103621.log 2023-11-24T08:14:01.606 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100878.log 2023-11-24T08:14:01.606 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112123.log.gz 2023-11-24T08:14:01.607 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111939.log 2023-11-24T08:14:01.608 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103621.log.gz 2023-11-24T08:14:01.609 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100317.log 2023-11-24T08:14:01.609 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.100878.log.gz 2023-11-24T08:14:01.611 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108026.log 2023-11-24T08:14:01.611 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111939.log.gz 2023-11-24T08:14:01.612 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103399.log 2023-11-24T08:14:01.612 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.100317.log.gz 2023-11-24T08:14:01.613 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116779.log 2023-11-24T08:14:01.614 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108026.log.gz 2023-11-24T08:14:01.616 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112165.log 2023-11-24T08:14:01.616 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103399.log.gz 2023-11-24T08:14:01.618 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113691.log 2023-11-24T08:14:01.618 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116779.log.gz 2023-11-24T08:14:01.619 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116865.log 2023-11-24T08:14:01.619 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112165.log.gz 2023-11-24T08:14:01.620 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103493.log 2023-11-24T08:14:01.620 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.113691.log.gz 2023-11-24T08:14:01.621 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100920.log 2023-11-24T08:14:01.621 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.116865.log.gz 2023-11-24T08:14:01.622 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101791.log 2023-11-24T08:14:01.623 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103493.log.gz 2023-11-24T08:14:01.624 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113172.log 2023-11-24T08:14:01.624 INFO:teuthology.orchestra.run.smithi106.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.100920.log.gz 2023-11-24T08:14:01.625 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101791.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-24T08:14:01.625 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101791.log.gz 2023-11-24T08:14:01.627 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105695.log 2023-11-24T08:14:01.627 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.113172.log.gz 2023-11-24T08:14:01.629 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109384.log 2023-11-24T08:14:01.630 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105653.log 2023-11-24T08:14:01.630 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105695.log.gz 2023-11-24T08:14:01.631 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98881.log 2023-11-24T08:14:01.632 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.109384.log.gz 2023-11-24T08:14:01.633 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100149.log 2023-11-24T08:14:01.633 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105653.log.gz 2023-11-24T08:14:01.634 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112646.log 2023-11-24T08:14:01.635 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98881.log.gz 2023-11-24T08:14:01.636 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100149.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100149.log.gz 2023-11-24T08:14:01.636 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116173.log 2023-11-24T08:14:01.637 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112646.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109113.log 2023-11-24T08:14:01.638 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.112646.log.gz 2023-11-24T08:14:01.639 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107095.log 2023-11-24T08:14:01.639 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116173.log.gz 2023-11-24T08:14:01.640 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109113.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.109113.log.gz 2023-11-24T08:14:01.641 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104801.log 2023-11-24T08:14:01.642 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107095.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107095.log.gz 2023-11-24T08:14:01.642 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94389.log 2023-11-24T08:14:01.644 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110327.log 2023-11-24T08:14:01.644 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104801.log.gz 2023-11-24T08:14:01.645 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.97014.log 2023-11-24T08:14:01.646 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94389.log.gz 2023-11-24T08:14:01.647 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.110327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111609.log 2023-11-24T08:14:01.647 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110327.log.gz 2023-11-24T08:14:01.648 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.2.97014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.97014.log.gz 2023-11-24T08:14:01.648 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104059.log 2023-11-24T08:14:01.650 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103891.log 2023-11-24T08:14:01.650 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.111609.log.gz 2023-11-24T08:14:01.651 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115063.log 2023-11-24T08:14:01.652 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104059.log.gz 2023-11-24T08:14:01.653 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116425.log 2023-11-24T08:14:01.653 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.103891.log.gz 2023-11-24T08:14:01.654 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93422.log 2023-11-24T08:14:01.655 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115063.log.gz 2023-11-24T08:14:01.656 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116425.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97846.log 2023-11-24T08:14:01.656 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.116425.log.gz 2023-11-24T08:14:01.657 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112023.log 2023-11-24T08:14:01.658 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93422.log.gz 2023-11-24T08:14:01.659 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97846.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-24T08:14:01.659 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97846.log.gz 2023-11-24T08:14:01.661 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112023.log: gzip 88.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.112023.log.gz --verbose 2023-11-24T08:14:01.661 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.94431.log 2023-11-24T08:14:01.662 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114675.log 2023-11-24T08:14:01.664 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94431.log: 86.1% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-24T08:14:01.664 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94431.log.gz 2023-11-24T08:14:01.664 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105427.log 2023-11-24T08:14:01.665 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114675.log: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-24T08:14:01.665 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114675.log.gz 2023-11-24T08:14:01.666 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101911.log 2023-11-24T08:14:01.667 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.105427.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105427.log.gz 2023-11-24T08:14:01.668 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104167.log 2023-11-24T08:14:01.669 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116551.log 2023-11-24T08:14:01.670 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101911.log.gz 2023-11-24T08:14:01.671 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104167.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-24T08:14:01.671 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.104167.log.gz 2023-11-24T08:14:01.673 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108551.log 2023-11-24T08:14:01.673 INFO:teuthology.orchestra.run.smithi106.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.116551.log.gz 2023-11-24T08:14:01.674 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98497.log 2023-11-24T08:14:01.676 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111514.log 2023-11-24T08:14:01.676 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108551.log.gz 2023-11-24T08:14:01.677 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99169.log 2023-11-24T08:14:01.677 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98497.log.gz 2023-11-24T08:14:01.679 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102930.log 2023-11-24T08:14:01.679 INFO:teuthology.orchestra.run.smithi106.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.111514.log.gz 2023-11-24T08:14:01.680 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96714.log 2023-11-24T08:14:01.680 INFO:teuthology.orchestra.run.smithi106.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99169.log.gz 2023-11-24T08:14:01.682 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102930.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-24T08:14:01.682 INFO:teuthology.orchestra.run.smithi106.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102930.log.gz 2023-11-24T08:14:01.683 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.96714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95859.log 2023-11-24T08:14:01.684 INFO:teuthology.orchestra.run.smithi106.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.96714.log.gz 2023-11-24T08:14:01.685 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99979.log 2023-11-24T08:14:01.686 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95859.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95859.log.gz 2023-11-24T08:14:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111096.log 2023-11-24T08:14:01.687 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99979.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99979.log.gz 2023-11-24T08:14:01.688 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112459.log 2023-11-24T08:14:01.689 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111096.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111096.log.gz 2023-11-24T08:14:01.691 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112459.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112459.log.gz 2023-11-24T08:14:01.692 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114316.log 2023-11-24T08:14:01.694 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100794.log 2023-11-24T08:14:01.695 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114316.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114316.log.gz 2023-11-24T08:14:01.696 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107888.log 2023-11-24T08:14:01.697 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100794.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100794.log.gz 2023-11-24T08:14:01.698 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102113.log/var/log/ceph/ceph-client.admin.107888.log: 2023-11-24T08:14:01.699 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107888.log.gz 2023-11-24T08:14:01.700 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93969.log 2023-11-24T08:14:01.701 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102113.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102113.log.gz 2023-11-24T08:14:01.702 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106236.log 2023-11-24T08:14:01.703 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93969.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93969.log.gz 2023-11-24T08:14:01.704 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106871.log 2023-11-24T08:14:01.705 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106236.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106236.log.gz 2023-11-24T08:14:01.706 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97288.log 2023-11-24T08:14:01.707 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106871.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106871.log.gz 2023-11-24T08:14:01.708 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101487.log 2023-11-24T08:14:01.709 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97288.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.97288.log.gz 2023-11-24T08:14:01.710 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109977.log 2023-11-24T08:14:01.711 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101487.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.101487.log.gz 2023-11-24T08:14:01.712 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96111.log 2023-11-24T08:14:01.713 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109977.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109977.log.gz 2023-11-24T08:14:01.714 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103747.log 2023-11-24T08:14:01.716 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.96111.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.96111.log.gz 2023-11-24T08:14:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111735.log 2023-11-24T08:14:01.717 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103747.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103747.log.gz 2023-11-24T08:14:01.719 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114232.log 2023-11-24T08:14:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111735.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111735.log.gz 2023-11-24T08:14:01.720 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94137.log 2023-11-24T08:14:01.722 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114232.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114232.log.gz 2023-11-24T08:14:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99631.log 2023-11-24T08:14:01.723 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94137.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94137.log.gz 2023-11-24T08:14:01.725 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99631.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99631.log.gz 2023-11-24T08:14:01.726 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98293.log 2023-11-24T08:14:01.728 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102239.log 2023-11-24T08:14:01.729 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.98293.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98293.log.gz 2023-11-24T08:14:01.730 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114549.log 2023-11-24T08:14:01.731 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102239.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102239.log.gz 2023-11-24T08:14:01.732 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.114549.log: /var/log/ceph/ceph-client.admin.100539.log 2023-11-24T08:14:01.732 INFO:teuthology.orchestra.run.smithi106.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-24T08:14:01.733 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.114549.log.gz 2023-11-24T08:14:01.733 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112333.log 2023-11-24T08:14:01.735 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100539.log: gzip -5 --verbose 89.1% -- replaced with /var/log/ceph/ceph-client.admin.100539.log.gz 2023-11-24T08:14:01.735 INFO:teuthology.orchestra.run.smithi106.stderr: -- /var/log/ceph/ceph-client.admin.103025.log 2023-11-24T08:14:01.736 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112333.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112333.log.gz 2023-11-24T08:14:01.737 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99673.log 2023-11-24T08:14:01.738 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100668.log 2023-11-24T08:14:01.738 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103025.log.gz 2023-11-24T08:14:01.740 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99253.log 2023-11-24T08:14:01.740 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99673.log.gz 2023-11-24T08:14:01.741 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.110976.log 2023-11-24T08:14:01.742 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100668.log.gz 2023-11-24T08:14:01.743 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108903.log 2023-11-24T08:14:01.743 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99253.log.gz 2023-11-24T08:14:01.744 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.110976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.110976.log.gz 2023-11-24T08:14:01.744 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97930.log 2023-11-24T08:14:01.745 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108903.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115963.log 2023-11-24T08:14:01.746 INFO:teuthology.orchestra.run.smithi106.stderr: -- replaced with /var/log/ceph/ceph-client.admin.108903.log.gz 2023-11-24T08:14:01.747 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116341.log 2023-11-24T08:14:01.747 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97930.log.gz 2023-11-24T08:14:01.749 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93925.log 2023-11-24T08:14:01.749 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.115963.log.gz 2023-11-24T08:14:01.750 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97414.log 2023-11-24T08:14:01.750 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116341.log.gz 2023-11-24T08:14:01.751 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.93925.log: gzip 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93925.log.gz -5 2023-11-24T08:14:01.752 INFO:teuthology.orchestra.run.smithi106.stderr: --verbose -- /var/log/ceph/ceph-client.admin.111651.log 2023-11-24T08:14:01.753 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.97414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101749.log 2023-11-24T08:14:01.753 INFO:teuthology.orchestra.run.smithi106.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97414.log.gz 2023-11-24T08:14:01.755 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102846.log 2023-11-24T08:14:01.755 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111651.log.gz 2023-11-24T08:14:01.756 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.101749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96249.log 2023-11-24T08:14:01.756 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101749.log.gz 2023-11-24T08:14:01.757 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102846.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102846.log.gz 2023-11-24T08:14:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102636.log 2023-11-24T08:14:01.758 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.96249.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.96249.log.gz 2023-11-24T08:14:01.759 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115519.log 2023-11-24T08:14:01.760 INFO:teuthology.orchestra.run.smithi106.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-24T08:14:01.761 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.102636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103535.log 2023-11-24T08:14:01.761 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102636.log.gz 2023-11-24T08:14:01.762 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108777.log 2023-11-24T08:14:01.762 INFO:teuthology.orchestra.run.smithi106.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115519.log.gz 2023-11-24T08:14:01.763 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.103535.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.103535.log.gz 2023-11-24T08:14:01.763 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111054.log 2023-11-24T08:14:01.764 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94613.log 2023-11-24T08:14:01.765 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108777.log.gz 2023-11-24T08:14:01.765 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111054.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111054.log.gz 2023-11-24T08:14:01.766 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109893.log 2023-11-24T08:14:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94613.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94613.log.gz 2023-11-24T08:14:01.767 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112916.log 2023-11-24T08:14:01.768 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.109893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99775.log 2023-11-24T08:14:01.769 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109893.log.gz 2023-11-24T08:14:01.769 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.112916.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112916.log.gz 2023-11-24T08:14:01.770 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95280.log 2023-11-24T08:14:01.771 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.99775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100583.log 2023-11-24T08:14:01.771 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99775.log.gz 2023-11-24T08:14:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.95280.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95280.log.gz 2023-11-24T08:14:01.772 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100497.log 2023-11-24T08:14:01.773 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114633.log 2023-11-24T08:14:01.774 INFO:teuthology.orchestra.run.smithi106.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100583.log.gz 2023-11-24T08:14:01.774 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.100497.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.100497.log.gz 2023-11-24T08:14:01.775 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94263.log 2023-11-24T08:14:01.776 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108945.log 2023-11-24T08:14:01.777 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94263.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.114633.log.gz 2023-11-24T08:14:01.777 INFO:teuthology.orchestra.run.smithi106.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.94263.log.gz 2023-11-24T08:14:01.777 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107347.log 2023-11-24T08:14:01.778 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.108945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104942.log 2023-11-24T08:14:01.778 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108945.log.gz 2023-11-24T08:14:01.779 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107347.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107347.log.gz 2023-11-24T08:14:01.780 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106362.log 2023-11-24T08:14:01.781 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.104942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.101173.log 2023-11-24T08:14:01.781 INFO:teuthology.orchestra.run.smithi106.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.104942.log.gz 2023-11-24T08:14:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.106362.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.106362.log.gz 2023-11-24T08:14:01.782 INFO:teuthology.orchestra.run.smithi106.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-24T08:14:01.783 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.0.101173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107051.log 2023-11-24T08:14:01.783 INFO:teuthology.orchestra.run.smithi106.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.101173.log.gz 2023-11-24T08:14:01.785 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113011.log 2023-11-24T08:14:01.786 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107497.log 2023-11-24T08:14:01.787 INFO:teuthology.orchestra.run.smithi106.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107051.log.gz 2023-11-24T08:14:01.788 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.113011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116509.log 2023-11-24T08:14:01.788 INFO:teuthology.orchestra.run.smithi106.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.113011.log.gz 2023-11-24T08:14:01.789 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.107497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94606.log 2023-11-24T08:14:01.790 INFO:teuthology.orchestra.run.smithi106.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107497.log.gz 2023-11-24T08:14:01.791 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116257.log 2023-11-24T08:14:01.791 INFO:teuthology.orchestra.run.smithi106.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116509.log.gz 2023-11-24T08:14:01.792 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.94606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115646.log 2023-11-24T08:14:01.793 INFO:teuthology.orchestra.run.smithi106.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94606.log.gz 2023-11-24T08:14:01.794 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116677.log 2023-11-24T08:14:01.794 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116257.log.gz 2023-11-24T08:14:01.795 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.115646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111897.log 2023-11-24T08:14:01.795 INFO:teuthology.orchestra.run.smithi106.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.115646.log.gz 2023-11-24T08:14:01.797 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.116677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114837.log 2023-11-24T08:14:01.797 INFO:teuthology.orchestra.run.smithi106.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.116677.log.gz 2023-11-24T08:14:01.798 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.111897.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.111897.log.gz 2023-11-24T08:14:01.801 INFO:teuthology.orchestra.run.smithi106.stderr:/var/log/ceph/ceph-client.admin.114837.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.114837.log.gz 2023-11-24T08:14:01.826 INFO:teuthology.orchestra.run.smithi106.stderr: 93.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-24T08:14:01.891 INFO:teuthology.orchestra.run.smithi106.stderr: 92.8% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-24T08:14:01.928 INFO:teuthology.orchestra.run.smithi106.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-24T08:14:01.973 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-24T08:14:01.992 INFO:teuthology.orchestra.run.smithi106.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-24T08:14:01.995 INFO:teuthology.orchestra.run.smithi106.stderr: 2023-11-24T08:14:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:real 0m0.854s 2023-11-24T08:14:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:user 0m2.325s 2023-11-24T08:14:01.996 INFO:teuthology.orchestra.run.smithi106.stderr:sys 0m1.446s 2023-11-24T08:14:01.997 INFO:tasks.ceph:Archiving logs... 2023-11-24T08:14:01.997 DEBUG:teuthology.misc:Transferring archived files from smithi106:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/remote/smithi106/log 2023-11-24T08:14:01.998 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:14:02.691 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/remote/smithi156/log 2023-11-24T08:14:02.692 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-24T08:14:03.025 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-24T08:14:03.048 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-24T08:14:03.048 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:14:03.058 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-24T08:14:03.206 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-24T08:14:03.215 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-24T08:14:03.215 DEBUG:teuthology.orchestra.run.smithi106:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:14:03.224 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-24T08:14:03.271 INFO:teuthology.orchestra.run.smithi106.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:14:03.281 INFO:teuthology.orchestra.run.smithi106.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:14:03.281 INFO:teuthology.orchestra.run.smithi106.stdout:=============================================================================== 2023-11-24T08:14:03.281 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 33 +2418us[+2426us] +/- 83ms 2023-11-24T08:14:03.281 INFO:teuthology.orchestra.run.smithi106.stdout:^* hv02.front.sepia.ceph.com 3 6 377 31 -757us[ -749us] +/- 61ms 2023-11-24T08:14:03.282 INFO:teuthology.orchestra.run.smithi106.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -1140us[-1140us] +/- 73ms 2023-11-24T08:14:03.282 INFO:teuthology.orchestra.run.smithi106.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:14:03.282 INFO:teuthology.orchestra.run.smithi156.stderr:bash: line 1: ntpq: command not found 2023-11-24T08:14:03.292 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-24T08:14:03.292 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-11-24T08:14:03.292 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 32 +2437us[+2442us] +/- 83ms 2023-11-24T08:14:03.292 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -756us[ -751us] +/- 61ms 2023-11-24T08:14:03.292 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 30 -1159us[-1154us] +/- 73ms 2023-11-24T08:14:03.293 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-24T08:14:03.294 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-24T08:14:03.302 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-24T08:14:03.303 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-24T08:14:03.326 DEBUG:teuthology.orchestra.run.smithi106:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:14:03.595 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-24T08:14:03.858 DEBUG:teuthology.orchestra.run.smithi106:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:14:03.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:14:03.969 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-24T08:14:04.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:14:04.080 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-24T08:14:04.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-24T08:14:04.135 INFO:teuthology.task.internal:Duration was 1092.330474 seconds 2023-11-24T08:14:04.136 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-24T08:14:04.170 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-24T08:14:04.170 DEBUG:teuthology.orchestra.run.smithi106:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:14:04.179 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-24T08:14:04.341 INFO:teuthology.orchestra.run.smithi106.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:14:04.345 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-24T08:14:04.793 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-24T08:14:04.793 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi106.front.sepia.ceph.com 2023-11-24T08:14:04.794 DEBUG:teuthology.orchestra.run.smithi106:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:14:04.855 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-11-24T08:14:04.856 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-24T08:14:04.940 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-24T08:14:04.940 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:14:04.949 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-24T08:14:05.044 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-24T08:14:05.045 DEBUG:teuthology.orchestra.run.smithi106:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:14:05.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-24T08:14:05.240 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-24T08:14:05.250 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-24T08:14:05.250 DEBUG:teuthology.orchestra.run.smithi106:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:14:05.345 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-24T08:14:05.445 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-24T08:14:05.455 DEBUG:teuthology.orchestra.run.smithi106:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:14:05.465 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:14:05.571 INFO:teuthology.orchestra.run.smithi106.stdout:kernel.core_pattern = core 2023-11-24T08:14:05.573 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-11-24T08:14:05.622 DEBUG:teuthology.orchestra.run.smithi106:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:14:05.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:14:05.662 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-24T08:14:05.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-24T08:14:05.722 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-24T08:14:05.733 INFO:teuthology.task.internal:Transferring archived files... 2023-11-24T08:14:05.735 DEBUG:teuthology.misc:Transferring archived files from smithi106:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/remote/smithi106 2023-11-24T08:14:05.735 DEBUG:teuthology.orchestra.run.smithi106:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:14:05.843 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585/remote/smithi156 2023-11-24T08:14:05.843 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-24T08:14:05.951 INFO:teuthology.task.internal:Removing archive directory... 2023-11-24T08:14:05.952 DEBUG:teuthology.orchestra.run.smithi106:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:14:05.962 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-24T08:14:06.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-24T08:14:06.032 INFO:teuthology.task.internal:Not uploading archives. 2023-11-24T08:14:06.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-24T08:14:06.064 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-24T08:14:06.065 DEBUG:teuthology.orchestra.run.smithi106:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:14:06.074 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-24T08:14:06.124 INFO:teuthology.orchestra.run.smithi106.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:14 /home/ubuntu/cephtest 2023-11-24T08:14:06.134 INFO:teuthology.orchestra.run.smithi156.stdout: 393944 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 24 08:14 /home/ubuntu/cephtest 2023-11-24T08:14:06.138 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-24T08:14:06.149 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-24T08:14:06.203 INFO:teuthology.nuke:Checking targets against current locks 2023-11-24T08:14:06.230 DEBUG:teuthology.nuke:shortname: smithi106 2023-11-24T08:14:06.230 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:14:06.251 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi106.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:46:29.092082', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDeEurGEUMjEgvIefvhfRLqIsBnEA6le92tQs8FB72yfJmmwh1/UqB6PkW7dcVCmTDQStpYE1qXQP8mt1ol3+FI='} 2023-11-24T08:14:06.255 DEBUG:teuthology.nuke:shortname: smithi156 2023-11-24T08:14:06.255 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-24T08:14:06.276 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-11-24_07:31:48-fs:volumes-rishabh-mgr-vol-misc-testing-default-smithi/7465585', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-24 07:46:29.090411', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPPAasaiQ3DwUeCiBZ8JtwfQIq+3w7DB62xjr7R10augtjzRlAoThP/oLGKfHN4PsXUMZlqfzp0C6b7q90NfQJU='} 2023-11-24T08:14:06.302 INFO:teuthology.orchestra.console.smithi106:Power off 2023-11-24T08:14:06.302 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:14:06.326 DEBUG:teuthology.orchestra.console.smithi106:power off output: Chassis Power Control: Down/Off 2023-11-24T08:14:06.326 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:06.341 INFO:teuthology.orchestra.console.smithi156:Power off 2023-11-24T08:14:06.342 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-24T08:14:06.353 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2023-11-24T08:14:06.366 DEBUG:teuthology.orchestra.console.smithi156:power off output: Chassis Power Control: Down/Off 2023-11-24T08:14:06.367 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:06.392 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-11-24T08:14:10.354 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:10.393 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:10.465 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2023-11-24T08:14:10.503 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is on 2023-11-24T08:14:14.468 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:14.504 DEBUG:teuthology.orchestra.console.smithi156:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:14.578 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is on 2023-11-24T08:14:14.615 DEBUG:teuthology.orchestra.console.smithi156:check power output: Chassis Power is off 2023-11-24T08:14:14.716 INFO:teuthology.orchestra.console.smithi156:Power off completed 2023-11-24T08:14:18.580 DEBUG:teuthology.orchestra.console.smithi106:pexpect command: ipmitool -H smithi106.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-24T08:14:18.691 DEBUG:teuthology.orchestra.console.smithi106:check power output: Chassis Power is off 2023-11-24T08:14:18.791 INFO:teuthology.orchestra.console.smithi106:Power off completed 2023-11-24T08:14:18.897 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 client.1-kernel-sha1: 8247bbb5b470f7e2a1f3eefcc16e24cf1338f606 description: fs:volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/volumes/{overrides test/snapshot}} duration: 1092.3304743766785 failure_reason: 'Test failure: test_subvolume_inherited_snapshot_ls (tasks.cephfs.test_volumes.TestSubvolumeSnapshots)' flavor: default owner: scheduled_rishabh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=5b9b261e36a94fc79d61d4275dd63228 status: fail success: false 2023-11-24T08:14:18.897 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-24T08:14:18.981 INFO:teuthology.run:FAIL